Sizing & Parallelism Reference

Tier Hardware Specs

Memory rule: 20% is reserved for overhead. User state (window accumulation, sort buffers) must stay below 80% of tier RAM. Exceeding this causes OOM failure.

How Parallelism Works

Every stage in a pipeline runs with default parallelism: 1. This base level is included in your tier at no additional cost.

When you need higher throughput for specific stages, increase their parallelism beyond 1. Only values > 1 count toward your tier's maximum.

Stages that commonly benefit from parallelism:

• $merge — concurrent writes to Atlas
• $lookup — concurrent reads for enrichment
• $https — concurrent API calls

Parallelism Calculation

Formula: Total Parallelism = sum of (parallelism - 1) for all stages where parallelism > 1

Tier Selection Algorithm

If Total Parallelism = 0:   → SP2  (max 1)
If Total Parallelism = 1:   → SP5  (max 2)
If Total Parallelism ≤ 8:   → SP10 (max 8)
If Total Parallelism ≤ 16:  → SP30 (max 16)
If Total Parallelism ≤ 64:  → SP50 (max 64)

Worked Examples

Simple pipeline (all parallelism = 1):

$source:    parallelism = 1  (does not count)
$match:     parallelism = 1  (does not count)
$merge:     parallelism = 1  (does not count)

Total = 0 → SP2

Medium pipeline:

$source:    parallelism = 1  (does not count)
$match:     parallelism = 1  (does not count)
$lookup:    parallelism = 4  (counts as 3)
$merge:     parallelism = 4  (counts as 3)

Total = 3 + 3 = 6 → SP10 (max 8)

Complex pipeline:

$source:    parallelism = 1  (does not count)
$https:     parallelism = 6  (counts as 5)
$merge:     parallelism = 8  (counts as 7)

Total = 5 + 7 = 12 → SP30 (max 16)

API Error for Parallelism Exceeded

If you specify a tier too small for the pipeline's parallelism, the API returns:

"Operator parallelism requested exceeds limit for this tier.
(Requested: X, Limit: Y). Minimum tier for this workload: SPxx or larger."

Solution: Use atlas-streams-manage → stop-processor, then start-processor with a higher tier value.

Complexity-Based Tier Selection

When parallelism is all default (1), choose tier based on pipeline complexity:

Complexity Scoring Heuristic

For automated tier recommendation, score the pipeline:

Score → Tier mapping:

• 0-10: SP2
• 11-20: SP5
• 21-40: SP10
• 41-60: SP30
• 61+: SP50

Always take the higher of complexity-driven vs parallelism-driven tier recommendations.

Billing

Charges are per-hour, calculated per-second, only while the processor is running.

• start-processor begins billing
• stop-processor stops billing
• Stopped processors retain state for 45 days at no charge

What's included in the tier price:

• Compute (vCPU and RAM)
• State storage
• Base parallelism (parallelism = 1 for all stages)

Additional costs (separate from tier):

• Data transfer egress (varies by cloud provider and transfer type: intra-region, inter-region, internet)
• VPC Peering (AWS and GCP)
• Private Link connectivity

For current pricing: https://www.mongodb.com/docs/atlas/billing/stream-processing-costs/

Sizing Workflow with MCP Tools

Phase 1: Pre-deployment estimate

1. Score the pipeline using the complexity heuristic above
2. Calculate parallelism needs using the formula
3. Take the higher recommendation
4. Start with that tier (or one tier lower for cost savings during testing)

Phase 2: Validation

1. Deploy the processor: atlas-streams-build → resource: "processor" with autoStart: true
2. Let it run for a representative period
3. Check stats: atlas-streams-discover → diagnose-processor
4. Review memoryUsageBytes:
   • Below 50% of tier RAM → over-provisioned, consider downsizing
   • 50-70% → good fit
   • 70-80% → at limit, monitor closely
   • Above 80% → under-provisioned, upgrade before it OOMs

Phase 3: Optimization

1. Stop processor: atlas-streams-manage → stop-processor
2. Restart with adjusted tier: atlas-streams-manage → start-processor with tier override
3. Monitor for another period
4. Repeat until right-sized

Cost Optimization: Time-of-Day Strategy

For workloads with predictable traffic patterns, adjust tiers by time of day:

To change tiers: stop-processor → start-processor with new tier value. Note: resumeFromCheckpoint: true (default) preserves state across tier changes.