--- title: "Guardrails - Quick Start" description: "import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';" type: skill canonical_url: https://claudary.paisolsolutions.com/skills/quick-start-1-2 source: "Claudary" difficulty: intermediate author: "Claude Code Knowledge Pack" date: 2026-07-10T11:37:26.215Z license: CC-BY-4.0 attribution: "Guardrails - Quick Start — Claudary (https://claudary.paisolsolutions.com/skills/quick-start-1-2)" --- # Guardrails - Quick Start import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; ## Overview import Image from '@theme/IdealImage'; import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; # Guardrails - Quick Start Setup Prompt Injection Detection, PII Masking on LiteLLM Proxy (AI Gateway) ## 1. Define guardrails on your LiteLLM config.yaml Set your guardrails under the `guardrails` section ```yaml model_list: - model_name: gpt-3.5-turbo litellm_params: model: openai/gpt-3.5-turbo api_key: os.environ/OPENAI_API_KEY guardrails: - guardrail_name: general-guard litellm_params: guardrail: aim mode: [pre_call, post_call] api_key: os.environ/AIM_API_KEY api_base: os.environ/AIM_API_BASE default_on: true # Optional - guardrail_name: "aporia-pre-guard" litellm_params: guardrail: aporia # supported values: "aporia", "lakera" mode: "during_call" api_key: os.environ/APORIA_API_KEY_1 api_base: os.environ/APORIA_API_BASE_1 - guardrail_name: "aporia-post-guard" litellm_params: guardrail: aporia # supported values: "aporia", "lakera" mode: "post_call" api_key: os.environ/APORIA_API_KEY_2 api_base: os.environ/APORIA_API_BASE_2 guardrail_info: # Optional field, info is returned on GET /guardrails/list # you can enter any fields under info for consumers of your guardrail params: - name: "toxicity_score" type: "float" description: "Score between 0-1 indicating content toxicity level" - name: "pii_detection" type: "boolean" # Example Presidio guardrail config with entity actions + confidence score thresholds - guardrail_name: "presidio-pii" litellm_params: guardrail: presidio mode: "pre_call" presidio_language: "en" pii_entities_config: CREDIT_CARD: "MASK" EMAIL_ADDRESS: "MASK" US_SSN: "MASK" presidio_score_thresholds: # minimum confidence scores for keeping detections CREDIT_CARD: 0.8 EMAIL_ADDRESS: 0.6 # Example Pillar Security config via Generic Guardrail API - guardrail_name: "pillar-security" litellm_params: guardrail: generic_guardrail_api mode: [pre_call, post_call] api_base: https://api.pillar.security/api/v1/integrations/litellm api_key: os.environ/PILLAR_API_KEY additional_provider_specific_params: plr_mask: true plr_evidence: true plr_scanners: true ``` For generic guardrail APIs you can also set **static headers** (`headers`: key/value sent on every request) and **dynamic headers** (`extra_headers`: list of client header names to forward). See [Generic Guardrail API - Static and dynamic headers](/docs/adding_provider/generic_guardrail_api#static-and-dynamic-headers). ### Supported values for `mode` (Event Hooks) - `pre_call` Run **before** LLM call, on **input** - `post_call` Run **after** LLM call, on **input & output** - `during_call` Run **during** LLM call, on **input** Same as `pre_call` but runs in parallel as LLM call. Response not returned until guardrail check completes - A list of the above values to run multiple modes, e.g. `mode: [pre_call, post_call]` ### Skip system messages in guardrail evaluation You can stop **unified** guardrails from scanning `role: system` content while still sending the full `messages` list to the model. **Global** — in `litellm_settings`: ```yaml litellm_settings: skip_system_message_in_guardrail: true ``` **Per guardrail** — under that guardrail’s `litellm_params`: set `skip_system_message_in_guardrail: true` or `false`. If omitted, the global `litellm_settings` value is used; per-guardrail `false` forces system messages to be included even when the global flag is `true`. **Via LiteLLM UI** — when **creating** or **editing** a guardrail in the LiteLLM Admin Dashboard, set **Skip system messages in guardrail** (under Basic Info on create, or in the edit / guardrail settings flows): | UI option | Effect | | ------------------------------------- | -------------------------------------------------------------------------------------- | | **Use global default** | Uses `litellm_settings.skip_system_message_in_guardrail` from your proxy config | | **Yes — exclude from guardrail scan** | Sets per-guardrail `skip_system_message_in_guardrail: true` | | **No — always include in scan** | Sets per-guardrail `skip_system_message_in_guardrail: false` (overrides a global skip) | Create guardrail: Skip system messages in guardrail dropdown with Use global default, Yes exclude from guardrail scan, and No always include in scan **Where this applies:** Only the **unified** guardrail path (providers that implement `apply_guardrail` and run through LiteLLM’s message translation layer) on **OpenAI Chat Completions** (`/v1/chat/completions`) and **Anthropic Messages** (`/v1/messages`). Examples include Presidio, Bedrock guardrails, `litellm_content_filter`, OpenAI Moderation, Generic Guardrail API, and custom code guardrails that define `apply_guardrail`. **Where this does *not* apply:** Guardrails that run only via direct hooks on the raw request (e.g. Lakera v2, Aporia, DynamoAI, Javelin, Lasso, Pangea, Model Armor, Azure Content Safety hooks, Guardrails AI, AIM, tool permission, MCP security). It also does not apply to other routes until those endpoints use the same translation layer (e.g. Responses API, embeddings, speech). ### Load Balancing Guardrails Need to distribute guardrail requests across multiple accounts or regions? See [Guardrail Load Balancing](./guardrail_load_balancing.md) for details on: - Load balancing across multiple AWS Bedrock accounts (useful for rate limit management) - Weighted distribution across guardrail instances - Multi-region guardrail deployments ## 2. Start LiteLLM Gateway ```shell litellm --config config.yaml --detailed_debug ``` ## 3. Test request **[Langchain, OpenAI SDK Usage Examples](../proxy/user_keys#request-format)** Expect this to fail since since `ishaan@berri.ai` in the request is PII ```shell curl -i http://localhost:4000/v1/chat/completions \\ -H "Content-Type: application/json" \\ -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \\ -d '{ "model": "gpt-3.5-turbo", "messages": [ {"role": "user", "content": "hi my email is ishaan@berri.ai"} ], "guardrails": ["aporia-pre-guard", "aporia-post-guard"] }' ``` Expected response on failure ```shell { "error": { "message": { "error": "Violated guardrail policy", "aporia_ai_response": { "action": "block", "revised_prompt": null, "revised_response": "Aporia detected and blocked PII", "explain_log": null } }, "type": "None", "param": "None", "code": "400" } } ``` ```shell curl -i http://localhost:4000/v1/chat/completions \\ -H "Content-Type: application/json" \\ -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \\ -d '{ "model": "gpt-3.5-turbo", "messages": [ {"role": "user", "content": "hi what is the weather"} ], "guardrails": ["aporia-pre-guard", "aporia-post-guard"] }' ``` ## **Default On Guardrails** Set `default_on: true` in your guardrail config to run the guardrail on every request. This is useful if you want to run a guardrail on every request without the user having to specify it. **Note:** These will run even if user specifies a different guardrail or empty guardrails array. ```yaml guardrails: - guardrail_name: "aporia-pre-guard" litellm_params: guardrail: aporia mode: "pre_call" default_on: true ``` **Test Request** In this request, the guardrail `aporia-pre-guard` will run on every request because `default_on: true` is set. ```shell curl -i http://localhost:4000/v1/chat/completions \\ -H "Content-Type: application/json" \\ -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \\ -d '{ "model": "gpt-3.5-turbo", "messages": [ {"role": "user", "content": "hi my email is ishaan@berri.ai"} ] }' ``` **Expected response** Your response headers will include `x-litellm-applied-guardrails` with the guardrail applied ``` x-litellm-applied-guardrails: aporia-pre-guard ``` ### Guardrail Policies Need more control? Use [Guardrail Policies](./guardrail_policies.md) to: - Group guardrails into reusable policies - Enable/disable guardrails for specific teams, keys, or models - Inherit from existing policies and override specific guardrails ## **Using Guardrails Client Side** ### Test yourself **(OSS)** Pass `guardrails` to your request body to test it ```shell curl -i http://localhost:4000/v1/chat/completions \\ -H "Content-Type: application/json" \\ -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \\ -d '{ "model": "gpt-3.5-turbo", "messages": [ {"role": "user", "content": "hi my email is ishaan@berri.ai"} ], "guardrails": ["aporia-pre-guard", "aporia-post-guard"] }' ``` ### Expose to your users **(Enterprise)** Follow this simple workflow to implement and tune guardrails: ### 1. View Available Guardrails First, check what guardrails are available and their parameters: Call `/guardrails/list` to view available guardrails and the guardrail info (supported parameters, description, etc) ```shell curl -X GET 'http://0.0.0.0:4000/guardrails/list' ``` Expected response ```json { "guardrails": [ { "guardrail_name": "aporia-post-guard", "guardrail_info": { "params": [ { "name": "toxicity_score", "type": "float", "description": "Score between 0-1 indicating content toxicity level" }, { "name": "pii_detection", "type": "boolean" } ] } } ] } ``` This config will return the `/guardrails/list` response above. The `guardrail_info` field is optional and you can add any fields under info for consumers of your guardrail ```yaml - guardrail_name: "aporia-post-guard" litellm_params: guardrail: aporia # supported values: "aporia", "lakera" mode: "post_call" api_key: os.environ/APORIA_API_KEY_2 api_base: os.environ/APORIA_API_BASE_2 guardrail_info: # Optional field, info is returned on GET /guardrails/list # you can enter any fields under info for consumers of your guardrail params: - name: "toxicity_score" type: "float" description: "Score between 0-1 indicating content toxicity level" - name: "pii_detection" type: "boolean" ``` ### 2. Apply Guardrails Add selected guardrails to your chat completion request: ```shell curl -i http://localhost:4000/v1/chat/completions \\ -H "Content-Type: application/json" \\ -d '{ "model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "your message"}], "guardrails": ["aporia-pre-guard", "aporia-post-guard"] }' ``` ### 3. Test with Mock LLM completions Send `mock_response` to test guardrails without making an LLM call. More info on `mock_response` [here](../../completion/mock_requests) ```shell curl -i http://localhost:4000/v1/chat/completions \\ -H "Content-Type: application/json" \\ -H "Authorization: Bearer sk-npnwjPQciVRok5yNZgKmFQ" \\ -d '{ "model": "gpt-3.5-turbo", "messages": [ {"role": "user", "content": "hi my email is ishaan@berri.ai"} ], "mock_response": "This is a mock response", "guardrails": ["aporia-pre-guard", "aporia-post-guard"] }' ``` ### 4. ✨ Pass Dynamic Parameters to Guardrail :::info ✨ This is an Enterprise only feature [Get a free trial](https://www.litellm.ai/enterprise#trial) ::: Use this to pass additional parameters to the guardrail API call. e.g. things like success threshold. **[See `guardrails` spec for more details](#spec-guardrails-parameter)** Set `guardrails={"aporia-pre-guard": {"extra_body": {"success_threshold": 0.9}}}` to pass additional parameters to the guardrail In this example `success_threshold=0.9` is passed to the `aporia-pre-guard` guardrail request body ```python import openai client = openai.OpenAI( api_key="anything", base_url="http://0.0.0.0:4000" ) response = client.chat.completions.create( model="gpt-3.5-turbo", messages = [ { "role": "user", "content": "this is a test request, write a short poem" } ], extra_body={ "guardrails": { "aporia-pre-guard": { "extra_body": { "success_threshold": 0.9 } } } } ) print(response) ``` ```shell curl --location 'http://0.0.0.0:4000/chat/completions' \\ --header 'Content-Type: application/json' \\ --data '{ "model": "gpt-3.5-turbo", "messages": [ { "role": "user", "content": "what llm are you" } ], "guardrails": { "aporia-pre-guard": { "extra_body": { "success_threshold": 0.9 } } } }' ``` ## **Proxy Admin Controls** ### Monitoring Guardrails Monitor which guardrails were executed and whether they passed or failed. e.g. guardrail going rogue and failing requests we don't intend to fail ::: #### Setup 1. Connect LiteLLM to a [supported logging provider](../logging) 2. Make a request with a `guardrails` parameter 3. Check your logging provider for the guardrail trace #### Traced Guardrail Success #### Traced Guardrail Failure ### ✨ Control Guardrails per API Key :::info ✨ This is an Enterprise only feature [Get a free trial](https://www.litellm.ai/enterprise#trial) ::: Use this to control what guardrails run per API Key. In this tutorial we only want the following guardrails to run for 1 API Key - `guardrails`: ["aporia-pre-guard", "aporia-post-guard"] **Step 1** Create Key with guardrail settings ```shell curl -X POST 'http://0.0.0.0:4000/key/generate' \\ -H 'Authorization: Bearer sk-1234' \\ -H 'Content-Type: application/json' \\ -d '{ "guardrails": ["aporia-pre-guard", "aporia-post-guard"] }' ``` ```shell curl --location 'http://0.0.0.0:4000/key/update' \\ --header 'Authorization: Bearer sk-1234' \\ --header 'Content-Type: application/json' \\ --data '{ "key": "sk-jNm1Zar7XfNdZXp49Z1kSQ", "guardrails": ["aporia-pre-guard", "aporia-post-guard"] }' ``` **Step 2** Test it with new key ```shell curl --location 'http://0.0.0.0:4000/chat/completions' \\ --header 'Authorization: Bearer sk-jNm1Zar7XfNdZXp49Z1kSQ' \\ --he --- Source: [Claudary](https://claudary.paisolsolutions.com/skills/quick-start-1-2) · https://claudary.paisolsolutions.com