Noma Security

Use Noma Security to protect your LLM applications with comprehensive AI content moderation and safety guardrails.

:::warning Deprecated: guardrail: noma (Legacy) guardrail: noma is deprecated and users should migrate to guardrail: noma_v2. The legacy guardrail: noma API will no longer be supported after March 31, 2026.

For easier migration of existing integrations, keep guardrail: noma and set use_v2: true. With use_v2: true, requests route to noma_v2; monitor_mode and block_failures still apply, while anonymize_input is ignored. :::

Noma v2 guardrails (Recommended)

Quick Start

guardrails:
  - guardrail_name: "noma-v2-guard"
    litellm_params:
      guardrail: noma_v2
      mode: "pre_call"
      api_key: os.environ/NOMA_API_KEY
      api_base: os.environ/NOMA_API_BASE

If you want to migrate gradually without changing guardrail names yet:

guardrails:
  - guardrail_name: "noma-guard"
    litellm_params:
      guardrail: noma
      use_v2: true
      mode: "pre_call"
      api_key: os.environ/NOMA_API_KEY
      api_base: os.environ/NOMA_API_BASE

Supported Params

• guardrail: Use noma_v2 (recommended), or noma with use_v2: true for migration
• mode: pre_call, post_call, during_call, pre_mcp_call, during_mcp_call
• api_key: Noma API key (required for Noma SaaS, optional for self-managed deployments)
• api_base: Noma API base URL (defaults to https://api.noma.security/)
• application_id: Application identifier. If omitted, v2 checks dynamic extra_body.application_id, then configured/env application_id; otherwise it is omitted.
• monitor_mode: If true, runs in monitor-only mode without blocking (defaults to false)
• block_failures: If true, fail-closed on guardrail technical failures (defaults to true)
• use_v2: Migration toggle when guardrail: noma is used

Environment Variables

Multiple Guardrails

Apply different v2 configurations for input and output:

guardrails:
  - guardrail_name: "noma-v2-input"
    litellm_params:
      guardrail: noma_v2
      mode: "pre_call"
      api_key: os.environ/NOMA_API_KEY

  - guardrail_name: "noma-v2-output"
    litellm_params:
      guardrail: noma_v2
      mode: "post_call"
      api_key: os.environ/NOMA_API_KEY

Pass Additional Parameters

This is supported in v2 via extra_body.
Currently, noma_v2 consumes dynamic application_id.

curl 'http://0.0.0.0:4000/v1/chat/completions' \\
  -H 'Content-Type: application/json' \\
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "Hello, how are you?"
      }
    ],
    "guardrails": {
      "noma-v2-guard": {
        "extra_body": {
          "application_id": "my-specific-app-id"
        }
      }
    }
  }'

Noma guardrails (Legacy)

Quick Start

1. Define Guardrails on your LiteLLM config.yaml

Define your guardrails under the guardrails section:

model_list:
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY

guardrails:
  - guardrail_name: "noma-guard"
    litellm_params:
      guardrail: noma
      mode: "during_call"
      api_key: os.environ/NOMA_API_KEY
      api_base: os.environ/NOMA_API_BASE
  - guardrail_name: "noma-pre-guard"
    litellm_params:
      guardrail: noma
      mode: "pre_call"
      api_key: os.environ/NOMA_API_KEY
      api_base: os.environ/NOMA_API_BASE

Supported values for mode

• pre_call Run before LLM call, on input
• post_call Run after LLM call, on input & output
• during_call Run during LLM call, on input. Same as pre_call but runs in parallel with the LLM call. Response not returned until guardrail check completes
• pre_mcp_call: Scan MCP tool call inputs before execution
• during_mcp_call: Monitor MCP tool calls in real-time

2. Start LiteLLM Gateway

litellm --config config.yaml --detailed_debug

3. Test request

Expect this to fail since the request contains harmful content:

curl -i http://0.0.0.0:4000/v1/chat/completions \\
  -H "Content-Type: application/json" \\
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {"role": "user", "content": "Tell me how to hack into someone's email account"}
    ]
  }'

Expected response on failure:

{
  "error": {
    "message": "{\
      \\"error\\": \\"Request blocked by Noma guardrail\\",\
      \\"details\\": {\
        \\"prompt\\": {\
          \\"harmfulContent\\": {\
            \\"result\\": true,\
            \\"confidence\\": 0.95\
          }\
        }\
      }\
    }",
    "type": "None",
    "param": "None",
    "code": "400"
  }
}

curl -i http://0.0.0.0:4000/v1/chat/completions \\
  -H "Content-Type: application/json" \\
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {"role": "user", "content": "What is the capital of France?"}
    ]
  }'

Expected response:

{
  "id": "chatcmpl-123",
  "object": "chat.completion",
  "created": 1677652288,
  "model": "gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 12,
    "total_tokens": 21
  }
}

Supported Params

guardrails:
  - guardrail_name: "noma-guard"
    litellm_params:
      guardrail: noma
      mode: "pre_call"
      api_key: os.environ/NOMA_API_KEY
      api_base: os.environ/NOMA_API_BASE
      ### OPTIONAL ###
      # application_id: "my-app"
      # monitor_mode: false
      # block_failures: true
      # anonymize_input: false

Required Parameters

• api_key: Your Noma Security API key (set as os.environ/NOMA_API_KEY in YAML config)

Optional Parameters

• api_base: Noma API base URL (defaults to https://api.noma.security/)
• application_id: Your application identifier (defaults to "litellm")
• monitor_mode: If true, logs violations without blocking (defaults to false)
• block_failures: If true, blocks requests when guardrail API failures occur (defaults to true)
• anonymize_input: If true, replaces sensitive content with anonymized version (defaults to false)

Environment Variables

You can set these environment variables instead of hardcoding values in your config:

Advanced Configuration

Monitor Mode

Use monitor mode to test your guardrails without blocking requests:

guardrails:
  - guardrail_name: "noma-monitor"
    litellm_params:
      guardrail: noma
      mode: "pre_call"
      api_key: os.environ/NOMA_API_KEY
      monitor_mode: true  # Log violations but don't block

Handling API Failures

Control behavior when the Noma API is unavailable:

guardrails:
  - guardrail_name: "noma-failopen"
    litellm_params:
      guardrail: noma
      mode: "pre_call"
      api_key: os.environ/NOMA_API_KEY
      block_failures: false  # Allow requests to proceed if guardrail API fails

Content Anonymization

Enable anonymization to replace sensitive content instead of blocking:

guardrails:
  - guardrail_name: "noma-anonymize"
    litellm_params:
      guardrail: noma
      mode: "pre_call"
      api_key: os.environ/NOMA_API_KEY
      anonymize_input: true  # Replace sensitive data with anonymized version

Multiple Guardrails

Apply different configurations for input and output:

guardrails:
  - guardrail_name: "noma-strict-input"
    litellm_params:
      guardrail: noma
      mode: "pre_call"
      api_key: os.environ/NOMA_API_KEY
      block_failures: true

  - guardrail_name: "noma-monitor-output"
    litellm_params:
      guardrail: noma
      mode: "post_call"
      api_key: os.environ/NOMA_API_KEY
      monitor_mode: true

✨ Pass Additional Parameters

Use extra_body to pass additional parameters to the Noma Security API call, such as dynamically setting the application ID for specific requests.

client = openai.OpenAI(
    api_key="your-api-key",
    base_url="http://0.0.0.0:4000"
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello, how are you?"}],
    extra_body={
        "guardrails": {
            "noma-guard": {
                "extra_body": {
                    "application_id": "my-specific-app-id"
                }
            }
        }
    }
)

curl 'http://0.0.0.0:4000/v1/chat/completions' \\
    -H 'Content-Type: application/json' \\
    -d '{
    "model": "gpt-4o-mini",
    "messages": [
        {
            "role": "user",
            "content": "Hello, how are you?"
        }
    ],
    "guardrails": {
        "noma-guard": {
            "extra_body": {
                "application_id": "my-specific-app-id"
            }
        }
    }
}'

This allows you to override the default application_id parameter for specific requests, which is useful for tracking usage across different applications or components.

Response Details

When content is blocked, Noma provides detailed information about the violations as JSON inside the message field, with the following structure:

{
  "error": "Request blocked by Noma guardrail",
  "details": {
    "prompt": {
      "harmfulContent": {
        "result": true,
        "confidence": 0.95
      },
      "sensitiveData": {
        "email": {
          "result": true,
          "entities": ["user@example.com"]
        }
      },
      "bannedTopics": {
        "violence": {
          "result": true,
          "confidence": 0.88
        }
      }
    }
  }
}