Browser — technical details

This document covers the command reference and internals of gstack's headless browser.

Command reference

All selector arguments accept CSS selectors, @e refs after snapshot, or @c refs after snapshot -C. 50+ commands total plus cookie import.

How it works

gstack's browser is a compiled CLI binary that talks to a persistent local Chromium daemon over HTTP. The CLI is a thin client — it reads a state file, sends a command, and prints the response to stdout. The server does the real work via Playwright.

┌─────────────────────────────────────────────────────────────────┐
│  Claude Code                                                    │
│                                                                 │
│  "browse goto https://staging.myapp.com"                        │
│       │                                                         │
│       ▼                                                         │
│  ┌──────────┐    HTTP POST     ┌──────────────┐                 │
│  │ browse   │ ──────────────── │ Bun HTTP     │                 │
│  │ CLI      │  localhost:rand  │ server       │                 │
│  │          │  Bearer token    │              │                 │
│  │ compiled │ ◄──────────────  │  Playwright  │──── Chromium    │
│  │ binary   │  plain text      │  API calls   │    (headless)   │
│  └──────────┘                  └──────────────┘                 │
│   ~1ms startup                  persistent daemon               │
│                                 auto-starts on first call       │
│                                 auto-stops after 30 min idle    │
└─────────────────────────────────────────────────────────────────┘

Lifecycle

1. First call: CLI checks .gstack/browse.json (in the project root) for a running server. None found — it spawns bun run browse/src/server.ts in the background. The server launches headless Chromium via Playwright, picks a random port (10000-60000), generates a bearer token, writes the state file, and starts accepting HTTP requests. This takes ~3 seconds.

2. Subsequent calls: CLI reads the state file, sends an HTTP POST with the bearer token, prints the response. ~100-200ms round trip.

3. Idle shutdown: After 30 minutes with no commands, the server shuts down and cleans up the state file. Next call restarts it automatically.

4. Crash recovery: If Chromium crashes, the server exits immediately (no self-healing — don't hide failure). The CLI detects the dead server on the next call and starts a fresh one.

Key components

browse/
├── src/
│   ├── cli.ts              # Thin client — reads state file, sends HTTP, prints response
│   ├── server.ts           # Bun.serve HTTP server — routes commands to Playwright
│   ├── browser-manager.ts  # Chromium lifecycle — launch, tabs, ref map, crash handling
│   ├── snapshot.ts         # Accessibility tree → @ref assignment → Locator map + diff/annotate/-C
│   ├── read-commands.ts    # Non-mutating commands (text, html, links, js, css, is, dialog, etc.)
│   ├── write-commands.ts   # Mutating commands (click, fill, select, upload, dialog-accept, etc.)
│   ├── meta-commands.ts    # Server management, chain, diff, snapshot routing
│   ├── cookie-import-browser.ts  # Decrypt + import cookies from real Chromium browsers
│   ├── cookie-picker-routes.ts   # HTTP routes for interactive cookie picker UI
│   ├── cookie-picker-ui.ts       # Self-contained HTML/CSS/JS for cookie picker
│   ├── activity.ts         # Activity streaming (SSE) for Chrome extension
│   └── buffers.ts          # CircularBuffer + console/network/dialog capture
├── test/                   # Integration tests + HTML fixtures
└── dist/
    └── browse              # Compiled binary (~58MB, Bun --compile)

The snapshot system

The browser's key innovation is ref-based element selection, built on Playwright's accessibility tree API:

1. page.locator(scope).ariaSnapshot() returns a YAML-like accessibility tree
2. The snapshot parser assigns refs (@e1, @e2, ...) to each element
3. For each ref, it builds a Playwright Locator (using getByRole + nth-child)
4. The ref-to-Locator map is stored on BrowserManager
5. Later commands like click @e3 look up the Locator and call locator.click()

No DOM mutation. No injected scripts. Just Playwright's native accessibility API.

Ref staleness detection: SPAs can mutate the DOM without navigation (React router, tab switches, modals). When this happens, refs collected from a previous snapshot may point to elements that no longer exist. To handle this, resolveRef() runs an async count() check before using any ref — if the element count is 0, it throws immediately with a message telling the agent to re-run snapshot. This fails fast (~5ms) instead of waiting for Playwright's 30-second action timeout.

Extended snapshot features:

• --diff (-D): Stores each snapshot as a baseline. On the next -D call, returns a unified diff showing what changed. Use this to verify that an action (click, fill, etc.) actually worked.
• --annotate (-a): Injects temporary overlay divs at each ref's bounding box, takes a screenshot with ref labels visible, then removes the overlays. Use -o <path> to control the output path.
• --cursor-interactive (-C): Scans for non-ARIA interactive elements (divs with cursor:pointer, onclick, tabindex>=0) using page.evaluate. Assigns @c1, @c2... refs with deterministic nth-child CSS selectors. These are elements the ARIA tree misses but users can still click.

Screenshot modes

The screenshot command supports five modes:

Element crop accepts CSS selectors (.class, #id, [attr]) or @e/@c refs from snapshot. Auto-detection for positional: @e/@c prefix = ref, ./#/[ prefix = CSS selector, -- prefix = flag, everything else = output path. Tag selectors like button aren't caught by the positional heuristic — use the --selector flag form.

The --base64 flag returns data:image/png;base64,... instead of writing to disk — composes with --selector, --clip, and --viewport.

Mutual exclusion: --clip + selector (flag or positional), --viewport + --clip, and --selector + positional selector all throw. Unknown flags (e.g. --bogus) also throw.

Retina screenshots — viewport --scale

viewport --scale <n> sets Playwright's deviceScaleFactor (context-level option, 1-3 gstack policy cap). A 2x scale doubles the pixel density of screenshots:

$B viewport 480x600 --scale 2
$B load-html /tmp/card.html
$B screenshot /tmp/card.png --selector .card
# .card element at 400x200 CSS pixels → card.png is 800x400 pixels

viewport --scale N alone (no WxH) keeps the current viewport size and only changes the scale. Scale changes trigger a browser context recreation (Playwright requirement), which invalidates @e/@c refs — rerun snapshot after. HTML loaded via load-html survives the recreation via in-memory replay (see below). Rejected in headed mode since scale is controlled by the real browser window.

Loading local HTML — goto file:// vs load-html

Two ways to render HTML that isn't on a web server:

Both are scoped to files under cwd or $TMPDIR via the same safe-dirs policy as the eval command. file:// URLs preserve query strings and fragments (SPA routes work). load-html has an extension allowlist (.html/.htm/.xhtml/.svg) and a magic-byte sniff to reject binary files mis-renamed as HTML, plus a 50 MB size cap (override via GSTACK_BROWSE_MAX_HTML_BYTES).

load-html content survives later viewport --scale calls via in-memory replay (TabSession tracks the loaded HTML + waitUntil). The replay is purely in-memory — HTML is never persisted to disk via state save to avoid leaking secrets or customer data.

Aliases: setcontent, set-content, and setContent all route to load-html via the server's alias canonicalization (happens before scope checks, so a read-scoped token still can't use the alias to run a write command).

Batch endpoint

POST /batch sends multiple commands in a single HTTP request. This eliminates per-command round-trip latency — critical for remote agents where each HTTP call costs 2-5s (e.g., Render → ngrok → laptop).

POST /batch
Authorization: Bearer <token>

{
  "commands": [
    {"command": "text", "tabId": 1},
    {"command": "text", "tabId": 2},
    {"command": "snapshot", "args": ["-i"], "tabId": 3},
    {"command": "click", "args": ["@e5"], "tabId": 4}
  ]
}

Response:

{
  "results": [
    {"index": 0, "status": 200, "result": "...page text...", "command": "text", "tabId": 1},
    {"index": 1, "status": 200, "result": "...page text...", "command": "text", "tabId": 2},
    {"index": 2, "status": 200, "result": "...snapshot...", "command": "snapshot", "tabId": 3},
    {"index": 3, "status": 403, "result": "{\\"error\\":\\"Element not found\\"}", "command": "click", "tabId": 4}
  ],
  "duration": 2340,
  "total": 4,
  "succeeded": 3,
  "failed": 1
}

Design decisions:

• Each command routes through handleCommandInternal — full security pipeline (scope checks, domain validation, tab ownership, content wrapping) enforced per command
• Per-command error isolation: one failure doesn't abort the batch
• Max 50 commands per batch
• Nested batches rejected
• Rate limiting: 1 batch = 1 request against the per-agent limit (individual commands skip rate check)
• Ref scoping is already per-tab — no changes needed

Usage pattern (agent crawling 20 pages):

# Step 1: Open 20 tabs (via individual newtab commands or batch)
# Step 2: Read all 20 pages at once
POST /batch → [{"command": "text", "tabId": 5}, {"command": "text", "tabId": 6}, ...]
# → 20 page contents in ~2-3 seconds total vs ~40-100 seconds serial

Authentication

Each server session generates a random UUID as a bearer token. The token is written to the state file (.gstack/browse.json) with chmod 600. Every HTTP request that mutates browser state must include Authorization: Bearer <token>. This prevents other processes on the machine from controlling the browser.

Dual-listener mode (v1.6.0.0+). When pair-agent activates an ngrok tunnel, the daemon binds a second HTTP socket that serves only /connect, /command (scoped tokens + a 17-command browser-driving allowlist), and /sidebar-chat. The tunnel listener is the only port ngrok forwards; /health, /cookie-picker, /inspector/*, and /welcome stay local-only. Root tokens sent over the tunnel return 403. See ARCHITECTURE.md for the full endpoint table.

SSE endpoints (/activity/stream, /inspector/events) accept the Bearer token OR the HttpOnly gstack_sse session cookie (30-minute stream-scope cookie minted by POST /sse-session). The ?token= query-param auth is no longer supported.

Console, network, and dialog capture

The server hooks into Playwright's page.on('console'), page.on('response'), and page.on('dialog') events. All entries are kept in O(1) circular buffers (50,000 capacity each) and flushed to disk asynchronously via Bun.write():

• Console: .gstack/browse-console.log
• Network: .gstack/browse-network.log
• Dialog: .gstack/browse-dialog.log

The console, network, and dialog commands read from the in-memory buffers, not disk.

Real browser mode (connect)

Instead of headless Chromium, connect launches your real Chrome as a headed window controlled by Playwright. You see everything Claude does in real time.

$B connect              # launch real Chrome, headed
$B goto https://app.com # navigates in the visible window
$B snapshot -i          # refs from the real page
$B click @e3            # clicks in the real window
$B focus                # bring Chrome window to foreground (macOS)
$B status               # shows Mode: cdp
$B disconnect           # back to headless mode

The window has a subtle green shimmer line at the top edge and a floating "gstack" pill in the bottom-right corner so you always know which Chrome window is being controlled.

How it works: Playwright's channel: 'chrome' launches your system Chrome binary via a native pipe protocol — not CDP WebSocket. All existing browse commands work unchanged because they