PII, PHI Masking - Presidio

Overview

Deployment options

For this guardrail you need a deployed Presidio Analyzer and Presido Anonymizer containers.

Quick Start

1. Create a PII, PHI Masking Guardrail

On the LiteLLM UI, navigate to Guardrails. Click "Add Guardrail". On this dropdown select "Presidio PII" and enter your presidio analyzer and anonymizer endpoints.

<br/> <br/>

1.2 Configure Entity Types

Now select the entity types you want to mask. See the supported actions here

1.3 Set Default Language (Optional)

You can also configure a default language for PII analysis using the presidio_language field in the UI. This sets the default language that will be used for all requests unless overridden by a per-request language setting.

Supported language codes include:

• en - English (default)
• es - Spanish
• de - German

If not specified, English (en) will be used as the default language.

Define your guardrails under the guardrails section

model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: openai/gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: "presidio-pii"
    litellm_params:
      guardrail: presidio  # supported values: "aporia", "bedrock", "lakera", "presidio"
      mode: "pre_call"
      presidio_language: "en"  # optional: set default language for PII analysis

Set the following env vars

Supported values for mode

• pre_call Run before LLM call, on input
• post_call Run after LLM call, on input & output
• logging_only Run after LLM call, only apply PII Masking before logging to Langfuse, etc. Not on the actual llm api request / response.

2. Start LiteLLM Gateway

litellm --config config.yaml --detailed_debug

3. Test it!

3.1 LiteLLM UI

On the litellm UI, navigate to the 'Test Keys' page, select the guardrail you created and send the following messaged filled with PII data.

My credit card is 4111-1111-1111-1111 and my email is test@example.com.

<br/>

3.2 Test in code

In order to apply a guardrail for a request send guardrails=["presidio-pii"] in the request body.

Langchain, OpenAI SDK Usage Examples

Expect this to mask Jane Doe since it's PII

curl http://localhost:4000/chat/completions \\
  -H "Content-Type: application/json" \\
  -H "Authorization: Bearer sk-1234" \\
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "Hello my name is Jane Doe"}
    ],
    "guardrails": ["presidio-pii"],
  }'

Expected response on failure

{
 "id": "chatcmpl-A3qSC39K7imjGbZ8xCDacGJZBoTJQ",
 "choices": [
   {
     "finish_reason": "stop",
     "index": 0,
     "message": {
       "content": "Hello, ! How can I assist you today?",
       "role": "assistant",
       "tool_calls": null,
       "function_call": null
     }
   }
 ],
 "created": 1725479980,
 "model": "gpt-3.5-turbo-2024-07-18",
 "object": "chat.completion",
 "system_fingerprint": "fp_5bd87c427a",
 "usage": {
   "completion_tokens": 13,
   "prompt_tokens": 14,
   "total_tokens": 27
 },
 "service_tier": null
}

curl http://localhost:4000/chat/completions \\
  -H "Content-Type: application/json" \\
  -H "Authorization: Bearer sk-1234" \\
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "Hello good morning"}
    ],
    "guardrails": ["presidio-pii"],
  }'

Tracing Guardrail requests

Once your guardrail is live in production, you will also be able to trace your guardrail on LiteLLM Logs, Langfuse, Arize Phoenix, etc, all LiteLLM logging integrations.

LiteLLM UI

On the LiteLLM logs page you can see that the PII content was masked for this specific request. And you can see detailed tracing for the guardrail. This allows you to monitor entity types masked with their corresponding confidence score and the duration of the guardrail execution.

Langfuse

When connecting Litellm to Langfuse, you can see the guardrail information on the Langfuse Trace.

Entity Types, Detection Confidence Score Threshold, and Scope Configuration

• Entity Types
  • You can configure specific entity types for PII detection and decide how to handle each entity type (mask or block).
• Detection Confidence Score Threshold
  • You can also provide an optional confidence score threshold at which detections will be passed to the anonymizer. Entities without an entry in presidio_score_thresholds keep all detections (no minimum score).
• Scope

  • Use the optional presidio_filter_scope to choose where checks run:

    • input: only user → model content is scanned
    • output: only model → user content is scanned
    • both (default): scan both directions

    What about output_parse_pii?
    This flag only un-masks tokens back to the originals after the model call; it does not run Presidio detection on outputs. Use presidio_filter_scope: output (or both) when you want Presidio to actively scan and mask the model’s response before it reaches the user.

    When to pick input vs output:

    • input: Protect upstream providers; strip PII before it leaves your boundary.
    • output: Catch PII the model might generate or leak back to users.
    • both: End-to-end protection in both directions.

Configure Entity Types, Detection Confidence Score Threshold, and Scope in config.yaml

Define your guardrails with specific entity type configuration:

model_list:
  - model_name: gpt-3.5-turbo
    litellm_params:
      model: openai/gpt-3.5-turbo
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: "presidio-mask-guard"
    litellm_params:
      guardrail: presidio
      mode: "pre_mcp_call"  # Use this mode for MCP requests
      presidio_filter_scope: both  # input | output | both, optional
      presidio_score_thresholds: # Optional
        ALL: 0.7            # Default confidence threshold applied to all entities
        CREDIT_CARD: 0.8    # Override for credit cards
        EMAIL_ADDRESS: 0.6  # Override for emails
      pii_entities_config:
        CREDIT_CARD: "MASK"  # Will mask credit card numbers
        EMAIL_ADDRESS: "MASK"  # Will mask email addresses
        
  - guardrail_name: "presidio-block-guard"
    litellm_params:
      guardrail: presidio
      mode: "pre_call"  # Use this mode for regular LLM requests
      presidio_filter_scope: both  # input | output | both, optional
      presidio_score_thresholds: # Optional
        CREDIT_CARD: 0.8  # Only keep credit card detections scoring 0.8+
      pii_entities_config:
        CREDIT_CARD: "BLOCK"  # Will block requests containing credit card numbers

Confidence threshold behavior:

• No presidio_score_thresholds: keep all detections (no thresholds applied)
• presidio_score_thresholds.ALL: apply this confidence threshold to every detection
• presidio_score_thresholds.: apply only to that entity
• If both ALL and an entity override exist, ALL applies globally and the entity override takes precedence for that entity

Supported Entity Types

LiteLLM Supports all Presidio entity types. See the complete list of presidio entity types here.

Supported Actions

For each entity type, you can specify one of the following actions:

• MASK: Replace the entity with a placeholder (e.g., ``)
• BLOCK: Block the request entirely if this entity type is detected

Test request with Entity Type Configuration

When using the masking configuration, entities will be replaced with placeholders:

curl http://localhost:4000/chat/completions \\
  -H "Content-Type: application/json" \\
  -H "Authorization: Bearer sk-1234" \\
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "My credit card is 4111-1111-1111-1111 and my email is test@example.com"}
    ],
    "guardrails": ["presidio-mask-guard"]
  }'

Example response with masked entities:

{
  "id": "chatcmpl-123abc",
  "choices": [
    {
      "message": {
        "content": "I can see you provided a <CREDIT_CARD> and an <EMAIL_ADDRESS>. For security reasons, I recommend not sharing this sensitive information.",
        "role": "assistant"
      },
      "index": 0,
      "finish_reason": "stop"
    }
  ],
  // ... other response fields
}

When using the blocking configuration, requests containing the configured entity types will be blocked completely with an exception:

curl http://localhost:4000/chat/completions \\
  -H "Content-Type: application/json" \\
  -H "Authorization: Bearer sk-1234" \\
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "My credit card is 4111-1111-1111-1111"}
    ],
    "guardrails": ["presidio-block-guard"]
  }'

When running this request, the proxy will raise a BlockedPiiEntityError exception.

{
  "error": {
    "message": "Blocked PII entity detected: CREDIT_CARD by Guardrail: presidio-block-guard."
  }
}

The exception includes the entity type that was blocked (CREDIT_CARD in this case) and the guardrail name that caused the blocking.

Advanced

Supported Modes

The Presidio guardrail supports the following modes:

• pre_call: Run before LLM call, on input
• post_call: Run after LLM call, on input & output
• logging_only: Run after LLM call, only apply PII Masking before logging to Langfuse, etc. Not on the actual llm api request / response
• pre_mcp_call: Run before MCP call, on input. Use this mode when you want to apply PII masking/blocking for MCP requests

MCP Usage Example

Here's how to use Presidio guardrails with MCP:

guardrails:
  - guardrail_name: "presidio-mcp-guard"
    litellm_params:
      guardrail: presidio
      mode: "pre_mcp_call"
      presidio_filter_scope: both  # input | output | both
      presidio_score_thresholds:
        CREDIT_CARD: 0.8  # Only keep credit card detections scoring 0.8+
        EMAIL_ADDRESS: 0.6  # Only keep email detections scoring 0.6+
      pii_entities_config:
        CREDIT_CARD: "MASK"  # Will mask credit card numbers
        EMAIL_ADDRESS: "BLOCK"  # Will block email addresses
        PHONE_NUMBER: "MASK"  # Will mask phone numbers
        MEDICAL_LICENSE: "BLOCK"  # Will block medical license numbers
      default_on: true

Test the MCP guardrail with a request:

curl http://localhost:4000/chat/completions \\
  -H "Content-Type: application/json" \\
  -H "Authorization: Bearer sk-1234" \\
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "My credit card is 4111-1111-1111-1111 and my medical license is ABC123"}
    ],
    "guardrails": ["presidio-mcp-guard"]
  }'

The request will be processed as follows:

1. Credit card number will be masked (e.g., replaced with <CREDIT_CARD>)
2. If a medical license is detected, the request will be blocked with a BlockedPiiEntityError

Set language per request

The Presidio API supports passing the language param. Here is how to set the language per request

curl http://localhost:4000/chat/completions \\
  -H "Content-Type: application/json" \\
  -H "Authorization: Bearer sk-1234" \\
  -d '{
    "model": "gpt-3.5-turbo",
    "messages": [
      {"role": "user", "content": "is this credit card number 9283833 correct?"}
    ],
    "guardrails": ["presidio-pre-guard"],
    "guardrail_config": {"language": "es"}
  }'

client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

# request sent to model set on litellm proxy, `litellm --model`
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages = [
        {