Lexical Search - Indexing

This guide covers how to configure MongoDB Atlas Search indexes. Use this reference to build index definitions with proper field types, analyzers, mappings, and optimization settings.

Table of Contents

• Atlas Search Index Definition
• Analyzer Selection
• Field Types
• Dynamic vs Explicit Mappings
• Stored Source
• Synonyms

---

Atlas Search Index Definition

Syntax

{
  "analyzer": "<analyzer-for-index>",
  "searchAnalyzer": "<analyzer-for-query>",
  "mappings": {
    "dynamic": <boolean> | {
      "typeSet": "<typeSet-name>"
    },
    "fields": {
      <field-definition>
    }
  },
  "numPartitions": <integer>,
  "analyzers": [ <custom-analyzer> ],
  "storedSource": <boolean> | {
    <stored-source-definition>
  },
  "synonyms": [
    {
      <synonym-mapping-definition>
    }
  ],
  "typeSets": [
    {
      "types": [
        {<field-types-definition>}
      ]
    }
  ]
}

Options

Basic Definition

Most indexes only need the mappings configuration:

{
  "mappings": {
    "dynamic": <boolean> | { <typeSet-definition> },
    "fields": { <field-definition> }
  }
}

---

Analyzer Selection

The analyzer determines how text is processed for indexing and searching.

Default behavior: Most queries don't specify an analyzer and use MongoDB's default standard analyzer, which:

• Divides text into terms based on word boundaries (language-neutral)
• Converts terms to lowercase and removes punctuation
• Recognizes email addresses, acronyms, CJK characters, alphanumerics, and more

Common built-in analyzers:

---

Index Analyzer (Applied at Index Time)

Specify per-field or top-level for all fields:

// Field-level (add to mappings.fields):
{
  "title": {
    "type": "string",
    "analyzer": "lucene.standard"  // Or omit to use default
  },
  "category": {
    "type": "token"   // Exact matching — use token, not string with lucene.keyword
  }
}

// Top-level (applies to all fields unless overridden):
{
  "analyzer": "lucene.standard",
  "mappings": {
    "fields": {
      "title": { "type": "string" }  // Uses top-level analyzer
    }
  }
}

---

Search Analyzer (Applied at Query Time)

Apply different analysis to queries than to indexed content:

// Top-level searchAnalyzer:
{
  "searchAnalyzer": "lucene.simple",  // Query-time analyzer
  "mappings": {
    "fields": {
      "description": {
        "type": "string",
        "analyzer": "lucene.standard"  // Index-time analyzer
      }
    }
  }
}

Use case: Index with standard analysis, but search with simpler/synonym-aware analyzer.

If omitted, uses the index analyzer. If both omitted, defaults to lucene.standard.

---

Multi Analyzer (Alternate Analyzers for Same Field)

Index the same field with multiple analyzers:

// Field configuration (add to mappings.fields):
{
  "title": {
    "type": "string",
    "analyzer": "lucene.standard",  // Default analyzer
    "multi": {
      "keywordAnalyzer": {
        "type": "string",
        "analyzer": "lucene.keyword"  // Alternate analyzer
      }
    }
  }
}

Use case: Support both fuzzy matching and exact matching on the same field.

To query using the alternate analyzer, specify the path as fieldName.alternateAnalyzerName (e.g., title.keywordAnalyzer).

---

Custom Analyzers

Define custom tokenization and filtering:

// Index definition with custom analyzer:
{
  "analyzers": [
    {
      "name": "customAnalyzer",
      "tokenizer": {
        "type": "standard"
      },
      "charFilters": [],
      "tokenFilters": [
        { "type": "lowercase" },
        { "type": "stop", "tokens": ["the", "a", "an"] }
      ]
    }
  ],
  "mappings": {
    "fields": {
      "content": {
        "type": "string",
        "analyzer": "customAnalyzer"  // Reference custom analyzer
      }
    }
  }
}

Use case: Need specific tokenization or filtering not provided by built-in analyzers.

---

Normalizers (Token Type Only)

Normalizers produce a single token (used with token field type):

// Field configuration (add to mappings.fields):
{
  "username": {
    "type": "token",
    "normalizer": "lowercase"  // Options: "lowercase", "none"
  }
}

Normalizers:

• lowercase: Transforms to lowercase, creates single token
• none: No transformation, creates single token

Use case: Exact matching with case normalization for token fields.

---

Decision Guide

• lucene.standard (or omit): Default for most text fields
• lucene.keyword: Categories, tags
• Language-specific: When you know the content language
• searchAnalyzer: Different analysis for queries vs indexed content (e.g., synonyms)
• multi: Support multiple search patterns on same field
• Custom: Need specific tokenization/filtering logic
• Normalizers: Token fields requiring case normalization

---

Field Types

Dynamic mapping includes: boolean, date, number, objectId, string, uuid Must configure explicitly: autocomplete, token, geo, embeddedDocuments, vector

Quick Reference

Field definition structure:

{
  "mappings": {
    "fields": {
      "<field-name>": {
        "type": "<field-type>",
        // type-specific options here
      }
    }
  }
}

Multiple types on same field:

"<field-name>": [
  { "type": "string" },
  { "type": "token", "normalizer": "lowercase" }
]

Arrays: MongoDB Search automatically flattens arrays during indexing. Specify only the element type, not that it's an array.

---

Dynamic vs Explicit Mappings

Choose based on user's needs:

Use dynamic (true) when:

• User is prototyping or exploring data with unknown schema
• Need to get started quickly without defining all fields
• All or most fields need to be searchable
• Accept larger index size and slower performance for convenience

Use explicit (dynamic: false) (recommended for production) when:

• User has completed early stages of prototyping and knows exactly which fields to search
• Performance and index size are priorities
• Schema is stable and well-defined
• Only a subset of fields need to be searchable

Use typeSets (recommended for production) when:

• User wants automatic indexing but with control over which types
• Document schema is dynamic and new fields need to be indexed automatically without an index rebuild
• Want different indexing strategies for different nested documents
• Balance between convenience and performance is important

---

Dynamic mappings automatically index all fields:

{
  "mappings": {
    "dynamic": true  // Index everything
  }
}

• Pros: Quick setup, works immediately
• Cons: Larger index, slower queries, wastes resources on unused fields

Explicit mappings define exactly what to index:

{
  "mappings": {
    "dynamic": false,  // Only index specified fields
    "fields": {
      "title": { "type": "string" },
      "genre": { "type": "token" }
    }
  }
}

• Pros: Smaller index, faster queries, precise control
• Cons: Requires knowing your schema

Configurable dynamic with typeSets (recommended middle ground):

{
  "mappings": {
    "dynamic": {
      "typeSet": "customTypes"
    },
    "fields": {
      "metadata": {
        "type": "document",
        "dynamic": {
          "typeSet": "metadataTypes"
        }
      }
    }
  },
  "typeSets": [
    {
      "name": "customTypes",
      "types": [
        { "type": "string" },
        { "type": "number" }
      ]
    },
    {
      "name": "metadataTypes",
      "types": [
        {
          "type": "string",
          "analyzer": "lucene.standard"
        }
      ]
    }
  ]
}

• Pros: Automatically indexes specified field types, more control than full dynamic, can configure different typeSets for sub-documents
• Cons: Still indexes all fields of specified types

Recommendation: Use static mappings or a dynamic typeSet (within a specific path, not at the root document level) in production for optimized index size and performance.

---

Stored Source

Store frequently accessed fields directly in the search index (mongot) to avoid full document lookups from the database. This dramatically improves query performance, especially when filtering or sorting.

Requirements:

• Available on clusters running MongoDB 7.0+
• Stored fields must still be indexed separately to query them
• Retrieve stored fields at query time using returnStoredSource: true (see lexical-search-querying.md)

Syntax:

{
  "storedSource": true | false | {
    "include" | "exclude": ["<field-name>", ...]
  }
}

Options:

true - Store all fields in documents. Not supported if index contains vector type field. Can significantly impact performance.

false - Don't store any fields (default behavior).

{ "include": [...] } - Store only specified fields. MongoDB Search a