Logging

Log Proxy input, output, and exceptions using:

• Langfuse
• OpenTelemetry
• GCS, s3, Azure (Blob) Buckets
• AWS SQS
• Lunary
• MLflow
• Deepeval
• Custom Callbacks - Custom code and API endpoints
• Langsmith
• DataDog
• Azure Sentinel
• DynamoDB
• etc.

Getting the LiteLLM Call ID

LiteLLM generates a unique call_id for each request. This call_id can be used to track the request across the system. This can be very useful for finding the info for a particular request in a logging system like one of the systems mentioned in this page.

curl -i -sSL --location 'http://0.0.0.0:4000/chat/completions' \\
    --header 'Authorization: Bearer sk-1234' \\
    --header 'Content-Type: application/json' \\
    --data '{
      "model": "gpt-3.5-turbo",
      "messages": [{"role": "user", "content": "what llm are you"}]
    }' | grep 'x-litellm'

The output of this is:

x-litellm-call-id: b980db26-9512-45cc-b1da-c511a363b83f
x-litellm-model-id: cb41bc03f4c33d310019bae8c5afdb1af0a8f97b36a234405a9807614988457c
x-litellm-model-api-base: https://x-example-1234.openai.azure.com
x-litellm-version: 1.40.21
x-litellm-response-cost: 2.85e-05
x-litellm-key-tpm-limit: None
x-litellm-key-rpm-limit: None

A number of these headers could be useful for troubleshooting, but the x-litellm-call-id is the one that is most useful for tracking a request across components in your system, including in logging tools.

Logging Features

Redact Messages, Response Content

Set litellm.turn_off_message_logging=True This will prevent the messages and responses from being logged to your logging provider, but request metadata - e.g. spend, will still be tracked. Useful for privacy/compliance when handling sensitive data.

1. Setup config.yaml

model_list:
 - model_name: gpt-3.5-turbo
    litellm_params:
      model: gpt-3.5-turbo
litellm_settings:
  success_callback: ["langfuse"]
  turn_off_message_logging: True # 👈 Key Change

2. Send request

curl --location 'http://0.0.0.0:4000/chat/completions' \\
    --header 'Content-Type: application/json' \\
    --data '{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
        "role": "user",
        "content": "what llm are you"
        }
    ]
}'

:::info

Dynamic request message redaction is in BETA.

:::

Pass in a request header to enable message redaction for a request.

x-litellm-enable-message-redaction: true

Example config.yaml

**1. Setup config.yaml **

model_list:
 - model_name: gpt-3.5-turbo
    litellm_params:
      model: gpt-3.5-turbo

2. Setup per request header

curl -L -X POST 'http://0.0.0.0:4000/v1/chat/completions' \\
-H 'Content-Type: application/json' \\
-H 'Authorization: Bearer sk-zV5HlSIm8ihj1F9C_ZbB1g' \\
-H 'x-litellm-enable-message-redaction: true' \\
-d '{
  "model": "gpt-3.5-turbo-testing",
  "messages": [
    {
      "role": "user",
      "content": "Hey, how'\\''s it going 1234?"
    }
  ]
}'

3. Check Logging Tool + Spend Logs

Logging Tool

Spend Logs

Redacting UserAPIKeyInfo

Redact information about the user api key (hashed token, user_id, team id, etc.), from logs.

Currently supported for Langfuse, OpenTelemetry, Logfire, ArizeAI logging.

litellm_settings: 
  callbacks: ["langfuse"]
  redact_user_api_key_info: true

Disable Message Redaction

If you have litellm.turn_on_message_logging turned on, you can override it for specific requests by setting a request header LiteLLM-Disable-Message-Redaction: true.

curl --location 'http://0.0.0.0:4000/chat/completions' \\
    --header 'Content-Type: application/json' \\
    --header 'LiteLLM-Disable-Message-Redaction: true' \\
    --data '{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
        "role": "user",
        "content": "what llm are you"
        }
    ]
}'

Turn off all tracking/logging

For some use cases, you may want to turn off all tracking/logging. You can do this by passing no-log=True in the request body.

:::info

Disable this by setting global_disable_no_log_param:true in your config.yaml file.

litellm_settings:
  global_disable_no_log_param: True

:::

curl -L -X POST 'http://0.0.0.0:4000/v1/chat/completions' \\
-H 'Content-Type: application/json' \\
-H 'Authorization: Bearer <litellm-api-key>' \\
-d '{
    "model": "openai/gpt-3.5-turbo",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "What'\\''s in this image?"
          }
        ]
      }
    ],
    "max_tokens": 300,
    "no-log": true # 👈 Key Change
}'

client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

# request sent to model set on litellm proxy, `litellm --model`
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ],
    extra_body={
      "no-log": True # 👈 Key Change
    }
)

print(response)

Expected Console Log

LiteLLM.Info: "no-log request, skipping logging"

✨ Dynamically Disable specific callbacks

:::info

This is an enterprise feature.

Proceed with LiteLLM Enterprise

:::

For some use cases, you may want to disable specific callbacks for a request. You can do this by passing x-litellm-disable-callbacks: <callback_name> in the request headers.

Send the list of callbacks to disable in the request header x-litellm-disable-callbacks.

curl --location 'http://0.0.0.0:4000/chat/completions' \\
    --header 'Content-Type: application/json' \\
    --header 'Authorization: Bearer sk-1234' \\
    --header 'x-litellm-disable-callbacks: langfuse' \\
    --data '{
    "model": "claude-sonnet-4-20250514",
    "messages": [
        {
        "role": "user",
        "content": "what llm are you"
        }
    ]
}'

client = openai.OpenAI(
    api_key="sk-1234",
    base_url="http://0.0.0.0:4000"
)

response = client.chat.completions.create(
    model="claude-sonnet-4-20250514",
    messages=[
        {
            "role": "user",
            "content": "what llm are you"
        }
    ],
    extra_headers={
        "x-litellm-disable-callbacks": "langfuse"
    }
)

print(response)

✨ Conditional Logging by Virtual Keys, Teams

Use this to:

1. Conditionally enable logging for some virtual keys/teams
2. Set different logging providers for different virtual keys/teams

👉 Get Started - Team/Key Based Logging

What gets logged?

Found under kwargs["standard_logging_object"]. This is a standard payload, logged for every response.

👉 Standard Logging Payload Specification

Langfuse

We will use the --config to set litellm.success_callback = ["langfuse"] this will log all successful LLM calls to langfuse. Make sure to set LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY in your environment

Step 1 Install langfuse

uv add langfuse>=2.0.0

Step 2: Create a config.yaml file and set litellm_settings: success_callback

model_list:
 - model_name: gpt-3.5-turbo
    litellm_params:
      model: gpt-3.5-turbo
litellm_settings:
  success_callback: ["langfuse"]

Step 3: Set required env variables for logging to langfuse

# Optional, defaults to https://cloud.langfuse.com

Step 4: Start the proxy, make a test request

Start proxy

litellm --config config.yaml --debug

Test Request

litellm --test

Expected output on Langfuse

Logging Metadata to Langfuse

Pass metadata as part of the request body

curl --location 'http://0.0.0.0:4000/chat/completions' \\
    --header 'Content-Type: application/json' \\
    --data '{
    "model": "gpt-3.5-turbo",
    "messages": [
        {
        "role": "user",
        "content": "what llm are you"
        }
    ],
    "metadata": {
        "generation_name": "ishaan-test-generation",
        "generation_id": "gen-id22",
        "trace_id": "trace-id22",
        "trace_user_id": "user-id2"
    }
}'

Set extra_body={"metadata": { }} to metadata you want to pass

client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

# request sent to model set on litellm proxy, `litellm --model`
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ],
    extra_body={
        "metadata": {
            "generation_name": "ishaan-generation-openai-client",
            "generation_id": "openai-client-gen-id22",
            "trace_id": "openai-client-trace-id22",
            "trace_user_id": "openai-client-user-id2"
        }
    }
)

print(response)

from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import (
    ChatPromptTemplate,
    HumanMessagePromptTemplate,
    SystemMessagePromptTemplate,
)
from langchain.schema import HumanMessage, SystemMessage

chat = ChatOpenAI(
    openai_api_base="http://0.0.0.0:4000",
    model = "gpt-3.5-turbo",
    temperature=0.1,
    extra_body={
        "metadata": {
            "generation_name": "ishaan-generation-langchain-client",
            "generation_id": "langchain-client-gen-id22",
            "trace_id": "langchain-client-trace-id22",
            "trace_user_id": "langchain-client-user-id2"
        }
    }
)

messages = [
    SystemMessage(
        content="You are a helpful assistant that im using to make a test request to."
    ),
    HumanMessage(
        content="test from litellm. tell me why it's amazing in 1 sentence"
    ),
]
response = chat(messages)

print(response)

Custom Tags

Set tags as part of your request body

client = openai.OpenAI(
    api_key="sk-1234",
    base_url="http://0.0.0.0:4000"
)

response = client.chat.completions.create(
    model="llama3",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ],
    user="palantir",
    extra_body={
        "metadata": {
            "tags": ["jobID:214590dsff09fds", "taskName:run_page_classification"]
        }
    }
)

print(response)

Pass metadata as part of the request body

curl --location 'http://0.0.0.0:4000/chat/completions' \\
    --header 'Content-Type: application/json' \\
    --header 'Authorization: Bearer sk-1234' \\
    --data '{
    "model": "llama3",
    "messages": [
        {
        "role": "user",
        "content": "what llm are you"
        }
    ],
    "user": "palantir",
    "metadata": {
        "tags": ["jobID:214590dsff09fds", "taskName:run_page_classification"]
    }
}'

from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import (
    ChatPromptTemplate,
    HumanMessagePromptTemplate,
    SystemMessagePromptTemplate,
)
from langchain.schema import HumanMessage, SystemMessage

os.environ["OPENAI_API_KEY"] = "sk-1234"

chat = ChatOpenAI(
    openai_api_base="http://0.0.0.0:4000",
    model = "llama3",
    user="palantir",
    extra_body={
        "metadata": {
            "tags": ["jobID:214590dsff09fds", "taskName:run_page_classification"]
        }
    }
)

messages = [
    SystemMessage(
        content="You are a helpful assistant that im using to make a test request to."
    ),
    HumanMessage(
        content="test from litellm. tell me why it's amazing in 1 sentence"
    ),
]
response = chat(messages)

print(response)

LiteLLM Tags - cache_hit, cache_key

Use this if you want to control which LiteLLM-specific fields are logged as tags by the LiteLLM proxy. By default LiteLLM Proxy logs no LiteLLM-specific fields

Usage

Specify langfuse_default_tags to control what litellm fields get logged on Langfuse

Example config.yaml

model_list:
  - model_name: gpt-4
    litellm_params:
      model: openai/fake
      api_key: fake-key
      api_base: https://exampleopenaiendpoint-production.up.railway.app/