LiteLLM

Trace LiteLLM chat, embedding, stream, and Router calls from Python.

The integration records LiteLLM calls across model providers. Make a project at telemetry.dev. Copy an API key from the project setup page. Keys use the td_live_... format.

What it captures

The integration records:

• Chat messages, models, sampling parameters, response formats, LiteLLM metadata, and embedding input
• Output messages, finish reasons, errors, time to first chunk, and calculated request cost
• Input, output, total, cache-read, cache-creation, and reasoning token usage
• Embedding token usage without embedding vectors
• Router calls as parent spans, with each deployment attempt as a child span.

Install

pip install telemetry-dev telemetry-dev-litellm

Python 3.14 does not match the package requirement.

Quickstart

import os

import litellm
import telemetry_dev
from telemetry_dev import propagate_attributes
from telemetry_dev_litellm import instrument_litellm

instrument_litellm()
telemetry_dev.init(service_name="litellm-app")

try:
    with propagate_attributes(
        user_id="example-user",
        session_id="litellm-01",
    ):
        response = litellm.completion(
            model=os.getenv("MODEL", "gpt-4o-mini"),
            messages=[
                {
                    "role": "user",
                    "content": "Explain telemetry.dev in one sentence.",
                },
            ],
            max_tokens=200,
        )
        print(response.choices[0].message.content)
finally:
    telemetry_dev.flush()
    telemetry_dev.shutdown()

API

The integration API has no options.

instrument_litellm() instruments completion, acompletion, embedding, and aembedding. More calls to instrument_litellm() have no more effect.

You can use the package drop-ins when import order is difficult to control:

from telemetry_dev_litellm import completion

response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)

Provider names

The integration maps common LiteLLM provider names to OpenTelemetry provider names:

An unknown provider keeps the LiteLLM provider string.

Router spans

wrap_router() instruments the Router instance methods for chat and embeddings. If global instrumentation is active, the Router span becomes the parent of each deployment-attempt span.

Streaming

The stream proxy keeps synchronous and asynchronous iteration, context managers, attributes, close(), and aclose(). It does not change request arguments or add stream_options.include_usage.

The integration rebuilds the final response with litellm.stream_chunk_builder(). Completion, early close, and stream errors close the span one time.

Flush

For a short script, call telemetry_dev.flush() and telemetry_dev.shutdown() before exit. In serverless code, call flush() before the runtime freezes.

Open the trace explorer to examine the spans. Refer to the quickstart for API-key setup.