Skillintermediate

FriendliAI

:::info **We support ALL FriendliAI models, just set `friendliai/` as a prefix when sending completion requests** :::

Claude Code Knowledge Pack7/10/2026

Overview

FriendliAI

:::info We support ALL FriendliAI models, just set friendliai/ as a prefix when sending completion requests :::

Property	Details
Description	The fastest and most efficient inference engine to build production-ready, compound AI systems.
Provider Route on LiteLLM	`friendliai/`
Provider Doc	FriendliAI ↗
Supported OpenAI Endpoints	`/chat/completions`, `/completions`

API Key

# env variable
os.environ['FRIENDLI_TOKEN']

Sample Usage

from litellm import completion

os.environ['FRIENDLI_TOKEN'] = ""
response = completion(
    model="friendliai/meta-llama-3.1-8b-instruct",
    messages=[
       {"role": "user", "content": "hello from litellm"}
   ],
)
print(response)

Sample Usage - Streaming

from litellm import completion

os.environ['FRIENDLI_TOKEN'] = ""
response = completion(
    model="friendliai/meta-llama-3.1-8b-instruct",
    messages=[
       {"role": "user", "content": "hello from litellm"}
   ],
    stream=True
)

for chunk in response:
    print(chunk)

Supported Models

We support ALL FriendliAI AI models, just set friendliai/ as a prefix when sending completion requests

Model Name	Function Call
meta-llama-3.1-8b-instruct	`completion(model="friendliai/meta-llama-3.1-8b-instruct", messages)`
meta-llama-3.1-70b-instruct	`completion(model="friendliai/meta-llama-3.1-70b-instruct", messages)`