OpenTelemetry Data in Elasticsearch

OTEL data ingested via the managed OTLP endpoint lands in standard index patterns. This reference covers querying OTEL logs, traces, and metrics.

Index Patterns

These indices are created automatically when data arrives via the OTLP endpoint.

Field Structure

OTEL data maps to Elastic fields with a consistent structure:

@timestamp                          # Event timestamp
resource.attributes.service.name    # Service that emitted the signal
resource.attributes.*               # Resource-level attributes (deployment, host, etc.)
attributes.*                        # Signal-specific attributes

Log Fields

body                                # Log message / event name
severityNumber                      # Numeric severity (1-24)
severityText                        # INFO, WARN, ERROR, etc.
attributes.*                        # Custom attributes set on the log
resource.attributes.service.name    # Originating service
traceId                             # Trace correlation (if emitted within a span)
spanId                              # Span correlation (if emitted within a span)

Trace Fields (APM)

In Elastic APM, OTEL spans become:

• Transaction = root span (no parent) — the entry point
• Span = child span (has a parent) — operations within a transaction

trace.id                            # Links all spans in a distributed trace
span.id                             # This span's unique ID
parent.id                           # Parent span ID (empty for root/transaction)
span.name                           # Operation name
span.duration.us                    # Duration in microseconds
span.type                           # Span type (custom, db, http, etc.)
service.name                        # Service that emitted the span

Querying OTEL Logs

ES|QL

# All logs from a service
curl -s -X POST "$ES_URL/_query" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": "FROM logs-generic.otel-default | WHERE resource.attributes.service.name == \\"my-app\\" | SORT @timestamp DESC | LIMIT 50"
  }' | jq .

# Events by category
curl -s -X POST "$ES_URL/_query" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": "FROM logs-generic.otel-default | WHERE attributes.event.category == \\"user.interaction\\" | STATS count = COUNT(*) BY attributes.event.action | SORT count DESC"
  }' | jq .

# Errors in the last hour
curl -s -X POST "$ES_URL/_query" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": "FROM logs-generic.otel-default | WHERE severityText == \\"ERROR\\" AND @timestamp > NOW() - 1 hour | SORT @timestamp DESC | LIMIT 20"
  }' | jq .

Query DSL

# Filter logs by custom attribute
curl -s "$ES_URL/logs-generic.otel-default/_search" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": {
      "bool": {
        "filter": [
          {"term": {"resource.attributes.service.name": "my-app"}},
          {"range": {"@timestamp": {"gte": "now-1h"}}}
        ]
      }
    },
    "sort": [{"@timestamp": {"order": "desc"}}],
    "size": 20
  }' | jq '.hits.hits[]._source'

# Full-text search on log body
curl -s "$ES_URL/logs-generic.otel-default/_search" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": {
      "bool": {
        "must": [{"match": {"body": "error timeout"}}],
        "filter": [{"range": {"@timestamp": {"gte": "now-24h"}}}]
      }
    },
    "size": 20
  }' | jq '.hits.hits[]._source'

Querying OTEL Traces

ES|QL

# Slowest transactions for a service
curl -s -X POST "$ES_URL/_query" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": "FROM traces-generic.otel-default | WHERE service.name == \\"my-app\\" AND parent.id IS NULL | SORT span.duration.us DESC | LIMIT 10"
  }' | jq .

# All spans in a trace (distributed trace view)
curl -s -X POST "$ES_URL/_query" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": "FROM traces-generic.otel-default | WHERE trace.id == \\"abc123def456\\" | SORT @timestamp ASC"
  }' | jq .

# Error rate by service
curl -s -X POST "$ES_URL/_query" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": "FROM traces-generic.otel-default | WHERE parent.id IS NULL | STATS total = COUNT(*), errors = COUNT_IF(event.outcome == \\"failure\\") BY service.name | EVAL error_rate = errors / total | SORT error_rate DESC"
  }' | jq .

Query DSL

# Find transactions (root spans) for a service
curl -s "$ES_URL/traces-generic.otel-default/_search" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": {
      "bool": {
        "must": [{"term": {"service.name": "my-app"}}],
        "must_not": [{"exists": {"field": "parent.id"}}],
        "filter": [{"range": {"@timestamp": {"gte": "now-1h"}}}]
      }
    },
    "sort": [{"span.duration.us": {"order": "desc"}}],
    "size": 10
  }' | jq '.hits.hits[] | {name: ._source.span.name, duration_ms: (._source.span.duration.us / 1000), trace_id: ._source.trace.id}'

Trace-Log Correlation

Logs emitted within an active trace context automatically include traceId and spanId. Use these to correlate:

# Step 1: Find a slow trace
curl -s -X POST "$ES_URL/_query" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": "FROM traces-generic.otel-default | WHERE service.name == \\"my-app\\" AND span.duration.us > 5000000 | KEEP trace.id, span.name, span.duration.us | LIMIT 5"
  }' | jq .

# Step 2: Get all logs for that trace
curl -s -X POST "$ES_URL/_query" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": "FROM logs-generic.otel-default | WHERE traceId == \\"abc123def456\\" | SORT @timestamp ASC"
  }' | jq .

# Step 3: Get all spans for the same trace
curl -s -X POST "$ES_URL/_query" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": "FROM traces-generic.otel-default | WHERE trace.id == \\"abc123def456\\" | SORT @timestamp ASC | KEEP span.name, span.duration.us, parent.id"
  }' | jq .

Aggregations on OTEL Data

# Log volume by service over time
curl -s "$ES_URL/logs-generic.otel-default/_search?size=0" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": {"range": {"@timestamp": {"gte": "now-24h"}}},
    "aggs": {
      "over_time": {
        "date_histogram": {"field": "@timestamp", "fixed_interval": "1h"},
        "aggs": {
          "by_service": {
            "terms": {"field": "resource.attributes.service.name", "size": 10}
          }
        }
      }
    }
  }' | jq '.aggregations'

# Average span duration by operation
curl -s "$ES_URL/traces-generic.otel-default/_search?size=0" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "query": {
      "bool": {
        "filter": [
          {"term": {"service.name": "my-app"}},
          {"range": {"@timestamp": {"gte": "now-1h"}}}
        ]
      }
    },
    "aggs": {
      "by_operation": {
        "terms": {"field": "span.name", "size": 20},
        "aggs": {
          "avg_duration_ms": {"avg": {"field": "span.duration.us"}},
          "p99_duration": {"percentiles": {"field": "span.duration.us", "percents": [99]}}
        }
      }
    }
  }' | jq '.aggregations'

# Unique users per service (custom attribute)
curl -s "$ES_URL/logs-generic.otel-default/_search?size=0" \\
  -H "Authorization: ApiKey $ES_API_KEY" \\
  -H "Content-Type: application/json" \\
  -d '{
    "aggs": {
      "by_service": {
        "terms": {"field": "resource.attributes.service.name", "size": 10},
        "aggs": {
          "unique_users": {"cardinality": {"field": "attributes.user.id"}}
        }
      }
    }
  }' | jq '.aggregations'

KQL Patterns for Discover

Common KQL queries for the Kibana Discover tab:

# All events from a service
resource.attributes.service.name: "my-app"

# Error logs
severityText: "ERROR"

# Custom attribute filtering
attributes.user.id: "david"

# Exists check (has frustration data)
attributes.frustration.type: *

# Combine filters
resource.attributes.service.name: "my-app" AND severityText: "ERROR" AND @timestamp >= now-1h

OTLP Endpoint Setup

Elastic Cloud provides a managed OTLP endpoint (no APM Server needed):

# Endpoint (from Kibana: Add data → Applications → OpenTelemetry)
OTEL_EXPORTER_OTLP_ENDPOINT="https://your-otlp-endpoint.elastic.cloud:443"

# Authentication
OTEL_EXPORTER_OTLP_HEADERS="Authorization=ApiKey YOUR_BASE64_KEY"

# Service identity
OTEL_RESOURCE_ATTRIBUTES="service.name=my-app,deployment.environment=production"

The OTLP endpoint accepts:

• POST /v1/logs — log signals
• POST /v1/traces — trace signals
• POST /v1/metrics — metric signals

Tips

• OTEL attributes are nested — query attributes.my.field, not my.field. Check mappings if unsure.
• resource.attributes vs attributes — resource attributes describe the source (service, host), signal attributes describe the event.
• Trace correlation — traceId on logs links to trace.id on traces. Note the different field names.
• Transactions vs spans — root spans (no parent.id) become Elastic APM transactions. Filter with parent.id IS NULL or NOT exists parent.id.
• Duration is in microseconds — span.duration.us. Divide by 1000 for milliseconds.
• Data streams — OTEL indices are data streams. Use _data_stream/logs-generic.otel-default to check health.
• Field type conflicts — if the same attribute has different types across services, Elasticsearch may reject documents. Use consistent types.