---
title: "Docker, Helm, Terraform"
description: "import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Image from '@theme/IdealImage';"
type: skill
canonical_url: https://claudary.paisolsolutions.com/skills/deploy-2
source: "Claudary"
difficulty: intermediate
author: "Claude Code Knowledge Pack"
date: 2026-07-10T11:19:43.515Z
license: CC-BY-4.0
attribution: "Docker, Helm, Terraform — Claudary (https://claudary.paisolsolutions.com/skills/deploy-2)"
---

# Docker, Helm, Terraform
import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import Image from '@theme/IdealImage';

## Overview

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Image from '@theme/IdealImage';

# Docker, Helm, Terraform

:::info No Limits on LiteLLM OSS
There are **no limits** on the number of users, keys, or teams you can create on LiteLLM OSS.
:::

You can find the Dockerfile to build litellm proxy [here](https://github.com/BerriAI/litellm/blob/main/Dockerfile)

> Note: Production requires at least 4 CPU cores and 8 GB RAM.

## Quick Start

:::info
Facing issues with pulling the docker image? Email us at support@berri.ai.
:::

To start using Litellm, run the following commands in a shell:

<Tabs>

<TabItem value="docker" label="Docker">

```
docker pull docker.litellm.ai/berriai/litellm:main-latest
```

[**See all docker images**](https://github.com/orgs/BerriAI/packages)

</TabItem>

<TabItem value="cli" label="LiteLLM CLI">

```shell
$ uv tool install 'litellm[proxy]'
```

</TabItem>

<TabItem value="docker-compose" label="Docker Compose (Proxy + DB)">

Use this docker compose to spin up the proxy with a postgres database running locally. 

```bash
# Get the docker compose file
curl -O https://raw.githubusercontent.com/BerriAI/litellm/main/docker-compose.yml
curl -O https://raw.githubusercontent.com/BerriAI/litellm/main/prometheus.yml

# Add the master key - you can change this after setup
echo 'LITELLM_MASTER_KEY="sk-1234"' > .env

# Add the litellm salt key - you cannot change this after adding a model
# It is used to encrypt / decrypt your LLM API Key credentials
# We recommend - https://1password.com/password-generator/ 
# password generator to get a random hash for litellm salt key
echo 'LITELLM_SALT_KEY="sk-1234"' >> .env

# Start
docker compose up
```

</TabItem>
</Tabs>

### Verify Docker image signatures

All LiteLLM Docker images are signed with [cosign](https://docs.sigstore.dev/cosign/overview/). Every release is signed with the same key introduced in [commit `0112e53`](https://github.com/BerriAI/litellm/commit/0112e53046018d726492c814b3644b7d376029d0).

**Verify using the pinned commit hash (recommended):**

A commit hash is cryptographically immutable, so this is the strongest way to ensure you are using the original signing key:

```bash
cosign verify \\
  --key https://raw.githubusercontent.com/BerriAI/litellm/0112e53046018d726492c814b3644b7d376029d0/cosign.pub \\
  ghcr.io/berriai/litellm:<release-tag>
```

**Verify using a release tag (convenience):**

Tags are protected in this repository and resolve to the same key. This option is easier to read but relies on tag protection rules:

```bash
cosign verify \\
  --key https://raw.githubusercontent.com/BerriAI/litellm/<release-tag>/cosign.pub \\
  ghcr.io/berriai/litellm:<release-tag>
```

Replace `<release-tag>` with the version you are deploying (e.g. `v1.83.0-stable`).

Expected output:

```
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - The signatures were verified against the specified public key
```

Learn more about LiteLLM's release signing in the [CI/CD v2 announcement](https://docs.litellm.ai/blog/ci-cd-v2-improvements#verify-docker-image-signatures). For a complete guide covering all image variants, CI/CD enforcement, and deployment best practices, see the [Docker Image Security Guide](./docker_image_security.md).

### Docker Run

#### Step 1. CREATE config.yaml 

Example `litellm_config.yaml` 

```yaml
model_list:
  - model_name: azure-gpt-4o
    litellm_params:
      model: azure/<your-azure-model-deployment>
      api_base: os.environ/AZURE_API_BASE # runs os.getenv("AZURE_API_BASE")
      api_key: os.environ/AZURE_API_KEY # runs os.getenv("AZURE_API_KEY")
      api_version: "2025-01-01-preview"
```



#### Step 2. RUN Docker Image

```shell
docker run \\
    -v $(pwd)/litellm_config.yaml:/app/config.yaml \\
    -e AZURE_API_KEY=d6*********** \\
    -e AZURE_API_BASE=https://openai-***********/ \\
    -p 4000:4000 \\
    docker.litellm.ai/berriai/litellm:main-stable \\
    --config /app/config.yaml --detailed_debug
```

Get Latest Image 👉 [here](https://github.com/berriai/litellm/pkgs/container/litellm)

#### Step 3. TEST Request

  Pass `model=azure-gpt-4o` this was set on step 1

  ```shell
  curl --location 'http://0.0.0.0:4000/chat/completions' \\
      --header 'Content-Type: application/json' \\
      --data '{
      "model": "azure-gpt-4o",
      "messages": [
          {
          "role": "user",
          "content": "what llm are you"
          }
      ]
  }'
  ```

### Docker Run - CLI Args

See all supported CLI args [here](https://docs.litellm.ai/docs/proxy/cli): 

Here's how you can run the docker image and pass your config to `litellm`
```shell
docker run docker.litellm.ai/berriai/litellm:main-stable --config your_config.yaml
```

Here's how you can run the docker image and start litellm on port 8002 with `num_workers=8`
```shell
docker run docker.litellm.ai/berriai/litellm:main-stable --port 8002 --num_workers 8
```


### Use litellm as a base image

```shell
# Use the provided base image
FROM docker.litellm.ai/berriai/litellm:main-stable

# Set the working directory to /app
WORKDIR /app

# Copy the configuration file into the container at /app
COPY config.yaml .

# Make sure your docker/entrypoint.sh is executable
RUN chmod +x ./docker/entrypoint.sh

# Expose the necessary port
EXPOSE 4000/tcp

# Override the CMD instruction with your desired command and arguments
# WARNING: FOR PROD DO NOT USE `--detailed_debug` it slows down response times, instead use the following CMD
# CMD ["--port", "4000", "--config", "config.yaml"]

CMD ["--port", "4000", "--config", "config.yaml", "--detailed_debug"]
```

### Build from published LiteLLM packages

Follow these instructions to build a Docker container from published LiteLLM packages. If your company has a strict requirement around security or image provenance, you can follow these steps.

**Note:** Copy the `schema.prisma` file from the [LiteLLM repository](https://github.com/BerriAI/litellm/blob/main/schema.prisma) into your build directory alongside this Dockerfile.

Dockerfile 

```shell
FROM cgr.dev/chainguard/python:latest-dev
ARG UV_IMAGE=ghcr.io/astral-sh/uv:0.10.9

USER root
WORKDIR /app

ENV UV_TOOL_BIN_DIR=/usr/local/bin

# Install runtime dependencies
RUN apk update && \\
    apk add --no-cache gcc python3-dev openssl openssl-dev

COPY --from=$UV_IMAGE /uv /usr/local/bin/uv
COPY --from=$UV_IMAGE /uvx /usr/local/bin/uvx

RUN uv tool install 'litellm[proxy,proxy-runtime,extra_proxy]==1.57.3' \\
    --python python

# Copy Prisma schema file
COPY schema.prisma .

# Generate prisma client
RUN prisma generate

EXPOSE 4000/tcp

ENTRYPOINT ["litellm"]
CMD ["--port", "4000"]
```


Build the docker image

```shell
docker build \\
  -f Dockerfile \\
  -t litellm-proxy-from-package-5 .
```

Run the docker image

```shell
docker run \\
    -v $(pwd)/litellm_config.yaml:/app/config.yaml \\
    -e OPENAI_API_KEY="sk-1222" \\
    -e DATABASE_URL="postgresql://xxxxxxxxx \\
    -p 4000:4000 \\
    litellm-proxy-from-package-5 \\
    --config /app/config.yaml --detailed_debug
```

### Terraform

s/o [Nicholas Cecere](https://www.linkedin.com/in/nicholas-cecere-24243549/) for his LiteLLM User Management Terraform

👉 [Go here for Terraform](https://github.com/BerriAI/terraform-provider-litellm)

### Kubernetes

Deploying a config file based litellm instance just requires a simple deployment that loads
the config.yaml file via a config map. Also it would be a good practice to use the env var
declaration for api keys, and attach the env vars with the api key values as an opaque secret.

```yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: litellm-config-file
data:
  config.yaml: |
      model_list: 
        - model_name: gpt-4o
          litellm_params:
            model: azure/gpt-4o-ca
            api_base: https://my-endpoint-canada-berri992.openai.azure.com/
            api_key: os.environ/CA_AZURE_OPENAI_API_KEY
---
apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: litellm-secrets
data:
  CA_AZURE_OPENAI_API_KEY: bWVvd19pbV9hX2NhdA== # your api key in base64
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: litellm-deployment
  labels:
    app: litellm
spec:
  selector:
    matchLabels:
      app: litellm
  template:
    metadata:
      labels:
        app: litellm
    spec:
      containers:
      - name: litellm
        image: docker.litellm.ai/berriai/litellm:main-stable # it is recommended to fix a version generally
        args:
          - "--config"
          - "/app/proxy_server_config.yaml"
        ports:
        - containerPort: 4000
        volumeMounts:
        - name: config-volume
          mountPath: /app/proxy_server_config.yaml
          subPath: config.yaml
        envFrom:
        - secretRef:
            name: litellm-secrets
      volumes:
        - name: config-volume
          configMap:
            name: litellm-config-file
```

:::info
To avoid issues with predictability, difficulties in rollback, and inconsistent environments, use versioning or SHA digests (for example, `litellm:main-v1.30.3` or `litellm@sha256:12345abcdef...`) instead of `litellm:main-stable`.
:::


### Helm Chart

:::info

[BETA] Helm Chart is BETA. If you run into an issues/have feedback please let us know [https://github.com/BerriAI/litellm/issues](https://github.com/BerriAI/litellm/issues)

:::

Use this when you want to use litellm helm chart as a dependency for other charts. The `litellm-helm` OCI is hosted here [https://github.com/BerriAI/litellm/pkgs/container/litellm-helm](https://github.com/BerriAI/litellm/pkgs/container/litellm-helm)

#### Step 1. Pull the litellm helm chart

```bash
helm pull oci://docker.litellm.ai/berriai/litellm-helm

# Pulled: docker.litellm.ai/berriai/litellm-helm:0.1.2
# Digest: sha256:7d3ded1c99c1597f9ad4dc49d84327cf1db6e0faa0eeea0c614be5526ae94e2a
```

#### Step 2. Unzip litellm helm
Unzip the specific version that was pulled in Step 1

```bash
tar -zxvf litellm-helm-0.1.2.tgz
```

#### Step 3. Install litellm helm

```bash
helm install lite-helm ./litellm-helm
```

#### Step 4. Expose the service to localhost

```bash
kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT
```

Your LiteLLM Proxy Server is now running on `http://127.0.0.1:4000`.

**That's it ! That's the quick start to deploy litellm**

#### Make LLM API Requests

:::info
💡 Go here 👉 [to make your first LLM API Request](user_keys)

LiteLLM is compatible with several SDKs - including OpenAI SDK, Anthropic SDK, Mistral SDK, LLamaIndex, Langchain (Js, Python)

:::

## Deployment Options

| Docs                                                                                              | When to Use                                                                                                                                           |
| ------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- |
| [Quick Start](#quick-start)                                                                       | call 100+ LLMs + Load Balancing                                                                                                                       |
| [Deploy with Database](#deploy-with-database)                                                     | + use Virtual Keys + Track Spend (Note: When deploying with a database providing a `DATABASE_URL` and `LITELLM_MASTER_KEY` are required in your env ) |
| [LiteLLM container + Redis](#litellm-container--redis)                                            | + load balance across multiple litellm containers                                                                                                     |
| [LiteLLM Database container + PostgresDB + Redis](#litellm-database-container--postgresdb--redis) | + use Virtual Keys + Track Spend + load balance across multiple litellm containers                                                                    |

### Deploy with Database
##### Docker, Kubernetes, Helm Chart

:::warning High Traffic Deployments (1000+ RPS)

If you expect high traffic (1000+ requests per second), **Redis is required** to prevent database connection exhaustion and deadlocks.

Add this to your config:
```yaml
general_settings:
  use_redis_transaction_buffer: true

litellm_settings:
  cache: true
  cache_params:
    type: redis
    host: your-redis-host
```

See [Resolve DB Deadlocks](/docs/proxy/db_deadlocks) for details.

:::

Requirements:
- Need a postgres database (e.g. [Supabase](https://supabase.com/), [Neon](https://neon.tech/), etc) Set `DATABASE_URL=postgresql://<user>:<password>@<host>:<port>/<dbname>` in your env 
- Set a `LITELLM_MASTER_KEY`, this is your Proxy Admin key - you can use this to create other keys (🚨 must start with `sk-`)

<Tabs>

<TabItem value="docker-deploy" label="Dockerfile">

We maintain a [separate Dockerfile](https://github.com/BerriAI/litellm/pkgs/container/litellm-database) for reducing build time when running LiteLLM proxy with a connected Postgres Database 

```shell
docker pull docker.litellm.ai/berriai/litellm-database:main-stable
```

```shell
docker run \\
    -v $(pwd)/litellm_config.yaml:/app/config.yaml \\
    -e LITELLM_MASTER_KEY=sk-1234 \\
    -e DATABASE_URL=postgresql://<user>:<password>@<host>:<port>/<dbname> \\
    -e AZURE_API_KEY=d6*********** \\
    -e AZURE_API_BASE=https://openai-***********/ \\
    -p 4000:4000 \\
    docker.litellm.ai/berriai/litellm-database:main-stable \\
    --config /app/config.yaml --detailed_debug
```

Your LiteLLM Proxy Server is now running on `http://0.0.0.0:4000`.

</TabItem>
<TabItem value="kubernetes-deploy" label="Kubernetes">

#### Step 1. Create deployment.yaml

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: litellm-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: litellm
  template:
    metadata:
      labels:
        app: litellm
    spec:
      containers:
        - name: litellm-container
          image: docker.litellm.ai/berriai/litellm:main-stable
          imagePullPolicy: Always
          env:
            - name: AZURE_API_KEY
              value: "d6******"
            - name: AZURE_API_BASE
              value: "https://ope******"
            - name: LITELLM_MASTER_KEY
              value: "sk-1234"
            - name: DATABASE_URL
              value: "po**********"
          args:
            - "--config"
            - "/app/proxy_config.yaml"  # Update the path to mount the config file
          volumeMounts:                 # Define volume mount for proxy_config.yaml
            - name: config-volume
              mountPath: /app/proxy_config.yaml
              subPath: config.yaml      # Specify the field under data of the ConfigMap litellm-config
              readOnly: true
          livenessProbe:

---

Source: [Claudary](https://claudary.paisolsolutions.com/skills/deploy-2) · https://claudary.paisolsolutions.com