Submit
All skills
Skillintermediate

LiveKit Voice Agent with LiteLLM Gateway

Simple example showing how to use LiveKit's xAI realtime plugin with LiteLLM as a proxy. This lets you switch between xAI, OpenAI, and Azure realtime APIs without changing your code.

Claude Code Knowledge Pack7/10/2026

Overview

LiveKit Voice Agent with LiteLLM Gateway

Simple example showing how to use LiveKit's xAI realtime plugin with LiteLLM as a proxy. This lets you switch between xAI, OpenAI, and Azure realtime APIs without changing your code.

Quick Start

1. Install dependencies

pip install livekit-agents[xai] websockets

2. Start LiteLLM proxy

# With xAI

litellm --config config.yaml --port 4000

3. Run the voice agent

python main.py

Type your message and get a voice response from Grok!

Configuration

Set these environment variables if needed:

Or use the defaults - connects to http://localhost:4000 by default.

Example Config File

Create a config.yaml with your realtime models:

model_list:
  - model_name: grok-voice-agent
    litellm_params:
      model: xai/grok-2-vision-1212
      api_key: os.environ/XAI_API_KEY
    model_info:
      mode: realtime

  - model_name: openai-voice-agent
    litellm_params:
      model: gpt-4o-realtime-preview
      api_key: os.environ/OPENAI_API_KEY
    model_info:
      mode: realtime

general_settings:
  master_key: sk-1234

Then start: litellm --config config.yaml --port 4000

How It Works

LiveKit's xAI plugin connects through LiteLLM proxy by setting base_url:

from livekit.plugins import xai

model = xai.realtime.RealtimeModel(
    voice="ara",
    api_key="sk-1234",              # LiteLLM proxy key
    base_url="http://localhost:4000", # Point to LiteLLM
)

Switching Providers

Just change the model in your config - no code changes needed:

xAI Grok:

model: xai/grok-2-vision-1212

OpenAI:

model: gpt-4o-realtime-preview

Azure OpenAI:

model: azure/gpt-4o-realtime-preview
api_base: https://your-endpoint.openai.azure.com/

Why Use LiteLLM?

  • ✅ Switch providers without changing agent code
  • ✅ Cost tracking across all voice sessions
  • ✅ Rate limiting and budgets
  • ✅ Load balancing across multiple API keys
  • ✅ Fallbacks to backup models

Learn More