Agent Runtime Architecture

This document describes the internal architecture of the @n8n/agents agent runtime — the execution engine that drives a single agent turn from input to final response.

---

Overview

AgentRuntime (src/runtime/agent-runtime.ts) is the core execution engine for a single agent turn. It uses the Vercel AI SDK directly (generateText / streamText) and is responsible for:

• Building the LLM message context (memory history, semantic recall, working memory in the system prompt, user input)
• Stripping orphaned tool-call/tool-result pairs before LLM calls (stripOrphanedToolMessages)
• Running the agentic tool-call loop (default 20 iterations, MAX_LOOP_ITERATIONS)
• Configurable tool-call concurrency — tools in one LLM turn run in batches of toolCallConcurrency (default 1; Infinity runs all executable calls in parallel)
• Suspending and resuming runs for Human-in-the-Loop (HITL) and for tools that return a branded suspend result (suspendSchema / resumeSchema)
• Persisting new messages to a memory store at the end of each completed turn, optionally saving embeddings for semantic recall
• Extracting and persisting working memory from assistant output when configured
• Optional structured output (Output.object + Zod), thinking / reasoning provider options, title generation, and telemetry (AI SDK experimental_telemetry)
• Token usage and cost (catalog pricing via getModelCost / computeCost)
• Emitting lifecycle events via AgentEventBus
• Tracking run state (idle → running → success / failed / suspended / cancelled)

There are two parallel execution paths — non-streaming (generate) and streaming (stream) — that mirror each other in structure.

graph TD
    A[User Input] --> B[normalizeInput]
    B --> C[buildMessageList]
    C --> D{generate or stream?}
    D -->|generate| E[runGenerateLoop]
    D -->|stream| F[startStreamLoop → runStreamLoop]
    E --> G[saveToMemory]
    F --> G
    G --> H[Return Result]

---

Public API — BuiltAgent

Agent implements BuiltAgent, which exposes the full public surface:

ExecutionOptions includes abortSignal?: AbortSignal, forwarded into AgentEventBus.resetAbort() so callers can cancel via an external signal as well as agent.abort().

---

Event system

AgentEventBus

AgentEventBus (src/runtime/event-bus.ts) is the internal publish/subscribe channel shared between Agent (registers handlers via on()) and AgentRuntime (emits events during the loop). A single bus instance is created when the SDK wires the runtime and passed in via AgentRuntimeConfig.

flowchart LR
    UserCode -->|"agent.on(event, handler)"| AgentEventBus
    AgentEventBus -->|"passed via config"| AgentRuntime
    AgentRuntime -->|"bus.emit(data)"| AgentEventBus
    AgentEventBus -->|"calls handlers synchronously"| UserCode

Handlers have the signature (data: AgentEventData) => void — there is no separate “controls” object; cancellation is done with agent.abort() on the same bus that holds the AbortController.

AgentMiddleware in types/runtime/event.ts is a small alias type (on mirrors the agent) for future middleware-style composition.

Event types

---

abort()

agent.abort() synchronously aborts the internal AbortController. The resulting signal is passed to generateText / streamText as abortSignal so in-flight HTTP cancels promptly. The loop also checks bus.isAborted at batch boundaries.

AgentEventBus.resetAbort(externalSignal?) runs at the start of each run: it replaces the controller and, if ExecutionOptions.abortSignal is set, forwards that signal’s abort to the internal controller.

Abort behaviour

State becomes cancelled. resetAbort() supplies a fresh controller per run so the same Agent instance can run again.

---

getState()

agent.getState() returns a shallow copy of SerializableAgentState. Before the first generate() / stream(), the Agent builder returns a minimal idle state with an empty messageList (messages, historyIds, inputIds, responseIds all empty).

State machine

stateDiagram-v2
    [*] --> idle: constructed
    idle --> running: generate() / stream() / resume()
    running --> success: loop completes normally
    running --> failed: unhandled error
    running --> suspended: tool suspends (HITL or suspend/resume)
    running --> cancelled: abort() / external signal
    suspended --> running: resume() / approve() / deny() loads checkpoint

AgentRunState values

SerializableAgentState

Important fields (see types/sdk/agent.ts):

interface SerializableAgentState {
  persistence?: AgentPersistenceOptions; // threadId + resourceId when using memory
  status: AgentRunState;
  messageList: SerializedMessageList;
  resumeData?: AgentResumeData;
  pendingToolCalls: Record<string, PendingToolCall>;
  finishReason?: FinishReason;
  usage?: TokenUsage;
  executionOptions?: PersistedExecutionOptions; // maxIterations only — persisted on suspend
}

PendingToolCall distinguishes tools already suspended (suspended: true, suspendPayload, resumeSchema) from calls not yet executed (suspended: false) when a batch stops at the first suspension.

---

asTool()

agent.asTool(description) wraps the agent as a BuiltTool. The handler calls agent.generate(input, { telemetry: ctx.parentTelemetry }), collects assistant text, and returns { result: string }. When the sub-run produces usage, results are wrapped so the parent runtime can merge SubAgentUsage and totalCost into the parent GenerateResult / stream finish chunk.

---

Message types

Custom messages are stripped for the model via filterLlmMessages() before toAiMessages().

messages.ts provides toAiMessages, fromAiMessages, and consumers rely on filterLlmMessages from sdk/message.ts.

Tool results vs model: optional BuiltTool.toModelOutput maps the stored / event result before building the tool-result the LLM sees; toMessage still uses the raw result for custom DB messages.

---

AgentMessageList

AgentMessageList (src/runtime/message-list.ts) is the central structure for one turn. It keeps a single append-only array and three Sets for provenance: history, input, response.

Sources

Key methods

forLlm(baseInstructions, instructionProviderOptions?)
  → [system + working memory block, ...toAiMessages(filterLlm(stripOrphaned(all)))]
turnDelta()      → input ∪ response messages (memory persistence)
responseDelta()  → response only (user-facing / GenerateResult.messages)
serialize()      → { messages, historyIds, inputIds, responseIds }
deserialize()    → restores all three sets via stable message ids

Serialization

Serialized state stores message id arrays per set (historyIds, inputIds, responseIds), not a single historyCount. After a round-trip, history / input / response classification is fully restored — required for correct turnDelta() after suspend/resume.

stripOrphanedToolMessages runs on loaded history and inside forLlm() so incomplete tool pairs do not reach the model.

Ordering note: The in-memory list is append-only; LLM context follows array order. Persisted threads, however, are loaded with ORDER BY createdAt (and a seq tiebreaker in SQL backends). Every message therefore needs a unique, monotonically increasing createdAt while it flows through AgentMessageList so reloads and before-filtered fetches match the turn’s true sequence. See Monotonic createdAt.

---

Agentic loop

Both runGenerateLoop and runStreamLoop follow the same high-level pattern: emit TurnStart, call the model with list.forLlm(...), append assistant / tool traffic via addResponse, process tool calls through iterateToolCallsConcurrent (batched by toolCallConcurrency), handle suspension / persistence, repeat until finish or max iterations.

Tool execution and concurrency

• Executable tool calls (non–provider-executed) are processed in windows of size this.concurrency (toolCallConcurrency ?? 1).
• Each window uses Promise.allSettled so all tools in the batch settle; a suspension in the batch stops subsequent batches and records remaining calls in pending without suspendPayload.
• HITL and suspend/resume flows share the same pending-map machinery; processToolCall validates JSON Schema or Zod input schemas (Ajv / Zod) before invoking the handler.

Loop invariants

1. Single list — addResponse accumulates assistant, tool, and custom messages for the turn.
2. System prompt — rebuilt each call via forLlm; working memory content is injected there, not as separate list rows.
3. Suspension preserves pending calls — remaining calls in the batch and later calls are recorded for resume.
4. Max iterations — default 20 (MAX_LOOP_ITERATIONS).
5. Abort — checked between batches; signal passed into AI SDK calls.

Non-streaming vs streaming

---

HITL and suspend/resume

HITL (approval): tools can require approval (requiresApproval / needsApprovalFn). The runtime treats approval outcomes like resume data: approve() / deny() delegate to resume() with { approved: true | false }.

Programmatic suspend: tools can return a branded suspend object; the runtime requires resumeSchema (Zod → JSON Schema for clients) and validates suspendPayload when suspendSchema is set.

sequenceDiagram
    participant Caller
    participant AgentRuntime
    participant CheckpointStore
    participant LLM

    Caller->>AgentRuntime: generate/stream(input)
    AgentRuntime->>LLM: generateText/streamText
    LLM-->>AgentRuntime: tool calls
    Note over AgentRuntime: Suspension: persist pendingToolCalls + messageList
    AgentRuntime->>CheckpointStore: suspend(runId, state)
    AgentRuntime-->>Caller: pendingSuspend / tool-call-suspended chunks

    Caller->>AgentRuntime: resume/approve/deny(method, …)
    AgentRuntime->>CheckpointStore: resume(runId) — load only
    AgentRuntime->>AgentRuntime: processToolCall / iteratePendingToolCallsConcurrent
    AgentRuntime->>LLM: Continue loop if needed
    AgentRuntime->>CheckpointStore: complete(runId) when finished

With concurrency > 1, multiple tools may suspend in the same turn; the stream can emit multiple tool-call-suspended chunks, and GenerateResult can carry pendingSuspend with multiple entries.

---

RunStateManager

RunStateManager (src/runtime/run-state.ts) persists suspended runs through a CheckpointStore:

• Default: in-memory MemoryCheckpointStore when checkpointStorage is 'memory' or omitted.
• Custom: pass a CheckpointStore implementation for durability.

suspend(runId, state) writes the state. resume(runId) loads the state and returns it with status: 'running'; it does not delete the key. complete(runId) deletes the checkpoint when the run finishes without remaining suspensions.

Known limitations

In-memory checkpoints grow until complete() runs. Production stores should implement TTL or eviction as needed.

---

Memory persistence

At end of turn, saveToMemory() uses `list.tu