Submit
All skills
Skillintermediate

Getting Started Tutorial

import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Image from '@theme/IdealImage';

Claude Code Knowledge Pack7/10/2026

Overview

Getting Started Tutorial

End-to-End tutorial for LiteLLM Proxy to:

  • Add an Azure OpenAI model
  • Make a successful /chat/completion call
  • Generate a virtual key
  • Set RPM limit on virtual key

Quick Install (Recommended for local / beginners)

New to LiteLLM? This is the easiest way to get started locally. One command installs LiteLLM and walks you through setup interactively — no config files to write by hand.

1. Install

curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/install.sh | sh

This detects your OS, installs litellm[proxy], and drops you straight into the setup wizard.

2. Follow the wizard

$ litellm --setup

  Welcome to LiteLLM

  Choose your LLM providers
  ○ 1. OpenAI        GPT-4o, GPT-4o-mini, o1
  ○ 2. Anthropic     Claude Opus, Sonnet, Haiku
  ○ 3. Azure OpenAI  GPT-4o via Azure
  ○ 4. Google Gemini Gemini 2.0 Flash, 1.5 Pro
  ○ 5. AWS Bedrock   Claude, Llama via AWS
  ○ 6. Ollama        Local models

  ❯ Provider(s): 1,2

  ❯ OpenAI API key: sk-...
  ❯ Anthropic API key: sk-ant-...

  ❯ Port [4000]:
  ❯ Master key [auto-generate]:

  ✔ Config saved → ./litellm_config.yaml

  ❯ Start the proxy now? (Y/n):

The wizard walks you through:

  1. Pick your LLM providers (OpenAI, Anthropic, Azure, Bedrock, Gemini, Ollama)
  2. Enter API keys for each provider
  3. Set a port and master key (or accept the defaults)
  4. Config is saved to ./litellm_config.yaml and the proxy starts immediately

3. Make a call

Your proxy is running on http://0.0.0.0:4000. Test it:

curl -X POST 'http://0.0.0.0:4000/chat/completions' \\
-H 'Content-Type: application/json' \\
-H 'Authorization: Bearer <your-master-key>' \\
-d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}]
}'

:::tip Already have uv installed? You can skip the curl install and run litellm --setup directly after uv tool install 'litellm[proxy]'. :::

Pre-Requisites

Choose your install method. Docker Compose users complete their full setup inside the tab and are done. Docker and LiteLLM CLI users continue with the steps below the tabs.

docker pull docker.litellm.ai/berriai/litellm:main-latest

See all docker images

$ uv tool install 'litellm[proxy]'

Docker Compose bundles LiteLLM with a Postgres database. Follow the steps below — the proxy will be fully running by the end.

Step 1 — Pull the LiteLLM database image

LiteLLM provides a dedicated litellm-database image for proxy deployments that connect to Postgres.

docker pull ghcr.io/berriai/litellm-database:main-latest

See all available tags on the GitHub Container Registry.

Step 2 — Set up a database

Complete all three config files before running docker compose up. The proxy server will not start correctly if any of these are missing.

2.1 — Get docker-compose.yml and create .env

# Get the docker compose file
curl -O https://raw.githubusercontent.com/BerriAI/litellm/main/docker-compose.yml

# Add the master key - you can change this after setup
echo 'LITELLM_MASTER_KEY="sk-1234"' > .env

# Add the litellm salt key — cannot be changed after adding a model
# Used to encrypt/decrypt your LLM API key credentials
# Generate a strong random value: https://1password.com/password-generator/
echo 'LITELLM_SALT_KEY="sk-1234"' >> .env

# Add your model credentials
echo 'AZURE_API_BASE="https://openai-***********/"' >> .env
echo 'AZURE_API_KEY="your-azure-api-key"' >> .env

2.2 — Create config.yaml

The default docker-compose.yml starts a Postgres container at db:5432. Your config.yaml must include database_url pointing to it:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: azure/my_azure_deployment
      api_base: os.environ/AZURE_API_BASE
      api_key: os.environ/AZURE_API_KEY
      api_version: "2025-01-01-preview"

general_settings:
  master_key: sk-1234 # 🔑 your proxy admin key (must start with sk-)
  database_url: "postgresql://llmproxy:dbpassword9090@db:5432/litellm"

:::tip database_url enables virtual keys, spend tracking, and the UI. Replace it with your Supabase or Neon connection string if you prefer a managed database. :::

2.3 — Create prometheus.yml

This file must exist as a file before docker compose up. If it is missing, Docker auto-creates it as an empty directory and the Prometheus container fails to start.

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: "litellm"
    static_configs:
      - targets: ["litellm:4000"]

Also verify that the config.yaml volume mount and --config flag are not commented out in docker-compose.yml:

services:
  litellm:
    volumes:
      - ./config.yaml:/app/config.yaml # ✅ must be uncommented
    command:
      - "--config=/app/config.yaml" # ✅ must be uncommented

:::warning All three files (.env, config.yaml, prometheus.yml) must be present before running docker compose up. See Troubleshooting if you run into issues. :::

Step 3 — Start the proxy server and test it

After config.yaml, prometheus.yml, and .env are complete, start the proxy:

docker compose up

Once running, test it with a curl request:

curl -X POST 'http://0.0.0.0:4000/chat/completions' \\
  -H 'Content-Type: application/json' \\
  -H 'Authorization: Bearer sk-1234' \\
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Expected response:

{
  "id": "chatcmpl-abcd",
  "created": 1773817678,
  "model": "gpt-4o",
  "object": "chat.completion",
  "system_fingerprint": "fp_6b1ef07cda",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "Hello! How can I assist you today?",
        "role": "assistant",
        "annotations": []
      }
    }
  ],
  "usage": {
    "completion_tokens": 9,
    "prompt_tokens": 9,
    "total_tokens": 18,
    "completion_tokens_details": {
      "accepted_prediction_tokens": 0,
      "audio_tokens": 0,
      "reasoning_tokens": 0,
      "rejected_prediction_tokens": 0
    },
    "prompt_tokens_details": {
      "audio_tokens": 0,
      "cached_tokens": 0
    }
  },
  "service_tier": "default"
}

Optional — Navigate to the LiteLLM UI and generate a virtual key

Open http://localhost:4000/ui in your browser and log in with your master key (sk-1234).

Navigate to Virtual Keys and click + Create New Key:

Virtual keys let you track spend, set rate limits, and control model access per user or team.

:::note Docker Compose users Your setup is complete — the steps below are for Docker and LiteLLM CLI users only. :::

Step 1 — Add a model

Control LiteLLM Proxy with a config.yaml file. Create one with your Azure model:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: azure/my_azure_deployment
      api_base: os.environ/AZURE_API_BASE
      api_key: "os.environ/AZURE_API_KEY"
      api_version: "2025-01-01-preview" # [OPTIONAL] litellm uses the latest azure api_version by default

Model List Specification

You can read more about how model resolution works in the Model Configuration section.

  • model_name (str) - This field should contain the name of the model as received.
  • litellm_params (dict) See All LiteLLM Params
    • model (str) - Specifies the model name to be sent to litellm.acompletion / litellm.aembedding, etc. This is the identifier used by LiteLLM to route to the correct model + provider logic on the backend.
    • api_key (str) - The API key required for authentication. It can be retrieved from an environment variable using os.environ/.
    • api_base (str) - The API base for your azure deployment.
    • api_version (str) - The API Version to use when calling Azure's OpenAI API. Get the latest Inference API version here.

Useful Links

2. Make a successful /chat/completion call

LiteLLM Proxy is 100% OpenAI-compatible. Test your azure model via the /chat/completions route.

2.1 Start Proxy

Save your config.yaml from step 1. as litellm_config.yaml.

docker run \\
    -v $(pwd)/litellm_config.yaml:/app/config.yaml \\
    -e AZURE_API_KEY=d6*********** \\
    -e AZURE_API_BASE=https://openai-***********/ \\
    -p 4000:4000 \\
    docker.litellm.ai/berriai/litellm:main-latest \\
    --config /app/config.yaml --detailed_debug

# RUNNING on http://0.0.0.0:4000

$ litellm --config /app/config.yaml --detailed_debug

Confirm your config was loaded correctly — you should see this in the logs:

Loaded config YAML (api_key and environment_variables are not shown):
{
  "model_list": [
    {
      "model_name": ...

2.2 Make Call

LiteLLM Proxy is 100% OpenAI-compatible. Test your model via /chat/completions:

curl -X POST 'http://0.0.0.0:4000/chat/completions' \\
-H 'Content-Type: application/json' \\
-H 'Authorization: Bearer sk-1234' \\
-d '{
    "model": "gpt-4o",
    "messages": [
      {
        "role": "system",
        "content": "You are an LLM named gpt-4o"
      },
      {
        "role": "user",
        "content": "what is your name?"
      }
    ]
}'

Expected Response

{
  "id": "chatcmpl-BcO8tRQmQV6Dfw6onqMufxPkLLkA8",
  "created": 1748488967,
  "model": "gpt-4o-2024-11-20",
  "object": "chat.completion",
  "system_fingerprint": "fp_ee1d74bde0",
  "choices": [
    {
      "finish_reason": "stop",
      "index": 0,
      "message": {
        "content": "My name is **gpt-4o**! How can I assist you today?",
        "role": "assistant",
        "tool_calls": null,
        "function_call": null,
        "annotations": []
      }
    }
  ],
  "usage": {
    "completion_tokens": 19,
    "prompt_tokens": 28,
    "total_tokens": 47,
    "completion_tokens_details": {
      "accepted_prediction_tokens": 0,
      "audio_tokens": 0,
      "reasoning_tokens": 0,
      "rejected_prediction_tokens": 0
    },
    "prompt_tokens_details": {
      "audio_tokens": 0,
      "cached_tokens": 0
    }
  },
  "service_tier": null,
  "prompt_filter_results": [
    {
      "prompt_index": 0,
      "content_filter_results": {
        "hate": {
          "filtered": false,
          "severity": "safe"
        },
        "self_harm": {
          "filtered": false,
          "severity": "safe"
        },
        "sexual": {
          "filtered": false,
          "severity": "safe"
        },
        "violence": {
          "filtered": false,
          "severity": "safe"
        }
      }
    }
  ]
}

Useful Links

Optional: Generate a virtual key

Track spend and control model access via virtual keys for the proxy.

Prerequisite — Set up a database

:::note Docker Compose users Your Postgres container is already running — skip ahead to Create Key w/ RPM Limit below. :::

Docker / LiteLLM CLI users — you need a Postgres database (e.g. Supabase, Neon, or self-hosted). Add general_settings to your config.yaml:

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: azure/my_azure_deployment
      api_base: os.environ/AZURE_API_BASE
      api_key: "os.environ/AZURE_API_KEY"
      api_version: "2025-01-01-preview" # [OPTIONAL] litellm uses the latest azure api_version by default

general_settings: 
  master_key: sk-1234 
  database_url: "postgresql://<user>:<password>@<host>:<port>/<dbname>" # 👈 KEY CHANGE

Save config.yaml as litellm_config.yaml before continuing.

You must finish this setup before starting the proxy server.

What is general_settings?

These are settings for the LiteLLM Proxy Server.

See All General Settings here.

  1. master_key (str)

    • Description:
      • Set a master key, this is your Proxy Admin key - you can use this to create other keys (🚨 must start with sk-).
    • Usage:
      • Set on config.yaml set your master key under general_settings:master_key, example - master_key: sk-1234
      • Set env variable set LITELLM_MASTER_KEY

  2. database_url (str)

    • Description:
      • Set a database_url, this is the connection to your Postgres DB, which is used by litellm for generating keys, users, teams.
    • Usage:
      • Set on config.yaml set your database_url under general_settings:database_url, example - database_url: "postgresql://..."
      • Set DATABASE_URL=postgresql://<user>:<password>@<host>:<port>/<dbname> in your env

Start Proxy

docker run \\
    -v $(pwd)/litellm_config.yaml:/app/config.yaml \\
    -e AZURE_API_KEY=d6*********** \\
    -e AZURE_API_BASE=https://openai-***********/ \\
    -p 4000:4000 \\
    ghcr.io/berriai/litellm-database:main-latest \\
    --config /app/config.yaml --detailed_debug

Create Key w/ RPM Limit

Create a key with rpm_limit: 1. This will only allow 1 request per minute for calls to proxy with this key.

curl -L -X POST 'http://0.0.0.0:4000/key/generate' \\
-H 'Authorization: Bearer sk-