--- title: "Getting Started Tutorial" description: "import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Image from '@theme/IdealImage';" type: skill canonical_url: https://claudary.paisolsolutions.com/skills/docker-quick-start source: "Claudary" difficulty: intermediate author: "Claude Code Knowledge Pack" date: 2026-07-10T11:19:57.519Z license: CC-BY-4.0 attribution: "Getting Started Tutorial — Claudary (https://claudary.paisolsolutions.com/skills/docker-quick-start)" --- # Getting Started Tutorial import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Image from '@theme/IdealImage'; ## Overview import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Image from '@theme/IdealImage'; # Getting Started Tutorial End-to-End tutorial for LiteLLM Proxy to: - Add an Azure OpenAI model - Make a successful /chat/completion call - Generate a virtual key - Set RPM limit on virtual key ## Quick Install (Recommended for local / beginners) New to LiteLLM? This is the easiest way to get started locally. One command installs LiteLLM and walks you through setup interactively — no config files to write by hand. ### 1. Install ```bash curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/install.sh | sh ``` This detects your OS, installs `litellm[proxy]`, and drops you straight into the setup wizard. ### 2. Follow the wizard ``` $ litellm --setup Welcome to LiteLLM Choose your LLM providers ○ 1. OpenAI GPT-4o, GPT-4o-mini, o1 ○ 2. Anthropic Claude Opus, Sonnet, Haiku ○ 3. Azure OpenAI GPT-4o via Azure ○ 4. Google Gemini Gemini 2.0 Flash, 1.5 Pro ○ 5. AWS Bedrock Claude, Llama via AWS ○ 6. Ollama Local models ❯ Provider(s): 1,2 ❯ OpenAI API key: sk-... ❯ Anthropic API key: sk-ant-... ❯ Port [4000]: ❯ Master key [auto-generate]: ✔ Config saved → ./litellm_config.yaml ❯ Start the proxy now? (Y/n): ``` The wizard walks you through: 1. Pick your LLM providers (OpenAI, Anthropic, Azure, Bedrock, Gemini, Ollama) 2. Enter API keys for each provider 3. Set a port and master key (or accept the defaults) 4. Config is saved to `./litellm_config.yaml` and the proxy starts immediately ### 3. Make a call Your proxy is running on `http://0.0.0.0:4000`. Test it: ```bash curl -X POST 'http://0.0.0.0:4000/chat/completions' \\ -H 'Content-Type: application/json' \\ -H 'Authorization: Bearer ' \\ -d '{ "model": "gpt-4o", "messages": [{"role": "user", "content": "Hello!"}] }' ``` :::tip Already have uv installed? You can skip the curl install and run `litellm --setup` directly after `uv tool install 'litellm[proxy]'`. ::: --- ## Pre-Requisites Choose your install method. **Docker Compose** users complete their full setup inside the tab and are done. **Docker** and **LiteLLM CLI** users continue with the steps below the tabs. ```bash docker pull docker.litellm.ai/berriai/litellm:main-latest ``` [**See all docker images**](https://github.com/orgs/BerriAI/packages) ```shell $ uv tool install 'litellm[proxy]' ``` Docker Compose bundles LiteLLM with a Postgres database. Follow the steps below — the proxy will be fully running by the end. ### Step 1 — Pull the LiteLLM database image LiteLLM provides a dedicated `litellm-database` image for proxy deployments that connect to Postgres. ```bash docker pull ghcr.io/berriai/litellm-database:main-latest ``` See all available tags on the [GitHub Container Registry](https://github.com/BerriAI/litellm/pkgs/container/litellm-database). --- ### Step 2 — Set up a database Complete all three config files **before** running `docker compose up`. The proxy server will not start correctly if any of these are missing. #### 2.1 — Get `docker-compose.yml` and create `.env` ```bash # Get the docker compose file curl -O https://raw.githubusercontent.com/BerriAI/litellm/main/docker-compose.yml # Add the master key - you can change this after setup echo 'LITELLM_MASTER_KEY="sk-1234"' > .env # Add the litellm salt key — cannot be changed after adding a model # Used to encrypt/decrypt your LLM API key credentials # Generate a strong random value: https://1password.com/password-generator/ echo 'LITELLM_SALT_KEY="sk-1234"' >> .env # Add your model credentials echo 'AZURE_API_BASE="https://openai-***********/"' >> .env echo 'AZURE_API_KEY="your-azure-api-key"' >> .env ``` #### 2.2 — Create `config.yaml` The default `docker-compose.yml` starts a Postgres container at `db:5432`. Your `config.yaml` must include `database_url` pointing to it: ```yaml model_list: - model_name: gpt-4o litellm_params: model: azure/my_azure_deployment api_base: os.environ/AZURE_API_BASE api_key: os.environ/AZURE_API_KEY api_version: "2025-01-01-preview" general_settings: master_key: sk-1234 # 🔑 your proxy admin key (must start with sk-) database_url: "postgresql://llmproxy:dbpassword9090@db:5432/litellm" ``` :::tip `database_url` enables virtual keys, spend tracking, and the UI. Replace it with your [Supabase](https://supabase.com/) or [Neon](https://neon.tech/) connection string if you prefer a managed database. ::: #### 2.3 — Create `prometheus.yml` This file **must exist as a file** before `docker compose up`. If it is missing, Docker auto-creates it as an empty directory and the Prometheus container fails to start. ```yaml global: scrape_interval: 15s evaluation_interval: 15s scrape_configs: - job_name: "litellm" static_configs: - targets: ["litellm:4000"] ``` Also verify that the `config.yaml` volume mount and `--config` flag are **not commented out** in `docker-compose.yml`: ```yaml services: litellm: volumes: - ./config.yaml:/app/config.yaml # ✅ must be uncommented command: - "--config=/app/config.yaml" # ✅ must be uncommented ``` :::warning All three files (`.env`, `config.yaml`, `prometheus.yml`) must be present before running `docker compose up`. See [Troubleshooting](#troubleshooting) if you run into issues. ::: --- ### Step 3 — Start the proxy server and test it After `config.yaml`, `prometheus.yml`, and `.env` are complete, start the proxy: ```bash docker compose up ``` Once running, test it with a curl request: ```bash curl -X POST 'http://0.0.0.0:4000/chat/completions' \\ -H 'Content-Type: application/json' \\ -H 'Authorization: Bearer sk-1234' \\ -d '{ "model": "gpt-4o", "messages": [{"role": "user", "content": "Hello!"}] }' ``` **Expected response:** ```json { "id": "chatcmpl-abcd", "created": 1773817678, "model": "gpt-4o", "object": "chat.completion", "system_fingerprint": "fp_6b1ef07cda", "choices": [ { "finish_reason": "stop", "index": 0, "message": { "content": "Hello! How can I assist you today?", "role": "assistant", "annotations": [] } } ], "usage": { "completion_tokens": 9, "prompt_tokens": 9, "total_tokens": 18, "completion_tokens_details": { "accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0 }, "prompt_tokens_details": { "audio_tokens": 0, "cached_tokens": 0 } }, "service_tier": "default" } ``` --- ### Optional — Navigate to the LiteLLM UI and generate a virtual key Open [http://localhost:4000/ui](http://localhost:4000/ui) in your browser and log in with your master key (`sk-1234`). Navigate to **Virtual Keys** and click **+ Create New Key**: LiteLLM UI — Create Virtual Key Virtual keys let you track spend, set rate limits, and control model access per user or team. :::note Docker Compose users Your setup is complete — the steps below are for **Docker** and **LiteLLM CLI** users only. ::: --- ## Step 1 — Add a model Control LiteLLM Proxy with a `config.yaml` file. Create one with your Azure model: ```yaml model_list: - model_name: gpt-4o litellm_params: model: azure/my_azure_deployment api_base: os.environ/AZURE_API_BASE api_key: "os.environ/AZURE_API_KEY" api_version: "2025-01-01-preview" # [OPTIONAL] litellm uses the latest azure api_version by default ``` --- ### Model List Specification You can read more about how model resolution works in the [Model Configuration](#understanding-model-configuration) section. - **`model_name`** (`str`) - This field should contain the name of the model as received. - **`litellm_params`** (`dict`) [See All LiteLLM Params](https://github.com/BerriAI/litellm/blob/559a6ad826b5daef41565f54f06c739c8c068b28/litellm/types/router.py#L222) - **`model`** (`str`) - Specifies the model name to be sent to `litellm.acompletion` / `litellm.aembedding`, etc. This is the identifier used by LiteLLM to route to the correct model + provider logic on the backend. - **`api_key`** (`str`) - The API key required for authentication. It can be retrieved from an environment variable using `os.environ/`. - **`api_base`** (`str`) - The API base for your azure deployment. - **`api_version`** (`str`) - The API Version to use when calling Azure's OpenAI API. Get the latest Inference API version [here](https://learn.microsoft.com/en-us/azure/ai-services/openai/api-version-deprecation?source=recommendations#latest-preview-api-releases). --- ### Useful Links - [**All Supported LLM API Providers (OpenAI/Bedrock/Vertex/etc.)**](../providers/) - [**Full Config.Yaml Spec**](./configs.md) - [**Pass provider-specific params**](../completion/provider_specific_params.md#proxy-usage) ## 2. Make a successful /chat/completion call LiteLLM Proxy is 100% OpenAI-compatible. Test your azure model via the `/chat/completions` route. ### 2.1 Start Proxy Save your config.yaml from step 1. as `litellm_config.yaml`. ```bash docker run \\ -v $(pwd)/litellm_config.yaml:/app/config.yaml \\ -e AZURE_API_KEY=d6*********** \\ -e AZURE_API_BASE=https://openai-***********/ \\ -p 4000:4000 \\ docker.litellm.ai/berriai/litellm:main-latest \\ --config /app/config.yaml --detailed_debug # RUNNING on http://0.0.0.0:4000 ``` ```shell $ litellm --config /app/config.yaml --detailed_debug ``` Confirm your config was loaded correctly — you should see this in the logs: ``` Loaded config YAML (api_key and environment_variables are not shown): { "model_list": [ { "model_name": ... ``` ### 2.2 Make Call LiteLLM Proxy is 100% OpenAI-compatible. Test your model via `/chat/completions`: ```bash curl -X POST 'http://0.0.0.0:4000/chat/completions' \\ -H 'Content-Type: application/json' \\ -H 'Authorization: Bearer sk-1234' \\ -d '{ "model": "gpt-4o", "messages": [ { "role": "system", "content": "You are an LLM named gpt-4o" }, { "role": "user", "content": "what is your name?" } ] }' ``` **Expected Response** ```bash { "id": "chatcmpl-BcO8tRQmQV6Dfw6onqMufxPkLLkA8", "created": 1748488967, "model": "gpt-4o-2024-11-20", "object": "chat.completion", "system_fingerprint": "fp_ee1d74bde0", "choices": [ { "finish_reason": "stop", "index": 0, "message": { "content": "My name is **gpt-4o**! How can I assist you today?", "role": "assistant", "tool_calls": null, "function_call": null, "annotations": [] } } ], "usage": { "completion_tokens": 19, "prompt_tokens": 28, "total_tokens": 47, "completion_tokens_details": { "accepted_prediction_tokens": 0, "audio_tokens": 0, "reasoning_tokens": 0, "rejected_prediction_tokens": 0 }, "prompt_tokens_details": { "audio_tokens": 0, "cached_tokens": 0 } }, "service_tier": null, "prompt_filter_results": [ { "prompt_index": 0, "content_filter_results": { "hate": { "filtered": false, "severity": "safe" }, "self_harm": { "filtered": false, "severity": "safe" }, "sexual": { "filtered": false, "severity": "safe" }, "violence": { "filtered": false, "severity": "safe" } } } ] } ``` ### Useful Links - [All Supported LLM API Providers (OpenAI/Bedrock/Vertex/etc.)](../providers/) - [Call LiteLLM Proxy via OpenAI SDK, Langchain, etc.](./user_keys.md#request-format) - [All API Endpoints Swagger](https://litellm-api.up.railway.app/#/chat%2Fcompletions) - [Other/Non-Chat Completion Endpoints](../embedding/supported_embedding.md) - [Pass-through for VertexAI, Bedrock, etc.](../pass_through/vertex_ai.md) ## Optional: Generate a virtual key Track spend and control model access via virtual keys for the proxy. ### Prerequisite — Set up a database :::note Docker Compose users Your Postgres container is already running — skip ahead to [Create Key w/ RPM Limit](#create-key-w-rpm-limit) below. ::: **Docker / LiteLLM CLI users** — you need a Postgres database (e.g. [Supabase](https://supabase.com/), [Neon](https://neon.tech/), or self-hosted). Add `general_settings` to your `config.yaml`: ```yaml model_list: - model_name: gpt-4o litellm_params: model: azure/my_azure_deployment api_base: os.environ/AZURE_API_BASE api_key: "os.environ/AZURE_API_KEY" api_version: "2025-01-01-preview" # [OPTIONAL] litellm uses the latest azure api_version by default general_settings: master_key: sk-1234 database_url: "postgresql://:@:/" # 👈 KEY CHANGE ``` Save config.yaml as `litellm_config.yaml` before continuing. You must finish this setup before starting the proxy server. --- **What is `general_settings`?** These are settings for the LiteLLM Proxy Server. See All General Settings [here](http://localhost:3000/docs/proxy/configs#all-settings). 1. **`master_key`** (`str`) - **Description**: - Set a `master key`, this is your Proxy Admin key - you can use this to create other keys (🚨 must start with `sk-`). - **Usage**: - **Set on config.yaml** set your master key under `general_settings:master_key`, example - `master_key: sk-1234` - **Set env variable** set `LITELLM_MASTER_KEY` 2. **`database_url`** (str) - **Description**: - Set a `database_url`, this is the connection to your Postgres DB, which is used by litellm for generating keys, users, teams. - **Usage**: - **Set on config.yaml** set your `database_url` under `general_settings:database_url`, example - `database_url: "postgresql://..."` - Set `DATABASE_URL=postgresql://:@:/` in your env ### Start Proxy ```bash docker run \\ -v $(pwd)/litellm_config.yaml:/app/config.yaml \\ -e AZURE_API_KEY=d6*********** \\ -e AZURE_API_BASE=https://openai-***********/ \\ -p 4000:4000 \\ ghcr.io/berriai/litellm-database:main-latest \\ --config /app/config.yaml --detailed_debug ``` ### Create Key w/ RPM Limit Create a key with `rpm_limit: 1`. This will only allow 1 request per minute for calls to proxy with this key. ```bash curl -L -X POST 'http://0.0.0.0:4000/key/generate' \\ -H 'Authorization: Bearer sk- --- Source: [Claudary](https://claudary.paisolsolutions.com/skills/docker-quick-start) · https://claudary.paisolsolutions.com