Submit
All skills
Skillintermediate

Design: Design Shotgun — Browser-to-Agent Feedback Loop

Generated on 2026-03-27 Branch: garrytan/agent-design-tools Status: LIVING DOCUMENT — update as bugs are found and fixed

Claude Code Knowledge Pack7/10/2026

Overview

Design: Design Shotgun — Browser-to-Agent Feedback Loop

Generated on 2026-03-27 Branch: garrytan/agent-design-tools Status: LIVING DOCUMENT — update as bugs are found and fixed

What This Feature Does

Design Shotgun generates multiple AI design mockups, opens them side-by-side in the user's real browser as a comparison board, and collects structured feedback (pick a favorite, rate alternatives, leave notes, request regeneration). The feedback flows back to the coding agent, which acts on it: either proceeding with the approved variant or generating new variants and reloading the board.

The user never leaves their browser tab. The agent never asks redundant questions. The board is the feedback mechanism.

The Core Problem: Two Worlds That Must Talk

  ┌─────────────────────┐          ┌──────────────────────┐
  │   USER'S BROWSER    │          │   CODING AGENT       │
  │   (real Chrome)     │          │   (Claude Code /     │
  │                     │          │    Conductor)         │
  │  Comparison board   │          │                      │
  │  with buttons:      │   ???    │  Needs to know:      │
  │  - Submit           │ ──────── │  - What was picked   │
  │  - Regenerate       │          │  - Star ratings      │
  │  - More like this   │          │  - Comments          │
  │  - Remix            │          │  - Regen requested?  │
  └─────────────────────┘          └──────────────────────┘

The "???" is the hard part. The user clicks a button in Chrome. The agent running in a terminal needs to know about it. These are two completely separate processes with no shared memory, no shared event bus, no WebSocket connection.

Architecture: How the Linkage Works

  USER'S BROWSER                    $D serve (Bun HTTP)              AGENT
  ═══════════════                   ═══════════════════              ═════
       │                                   │                           │
       │  GET /                            │                           │
       │ ◄─────── serves board HTML ──────►│                           │
       │    (with __GSTACK_SERVER_URL      │                           │
       │     injected into <head>)         │                           │
       │                                   │                           │
       │  [user rates, picks, comments]    │                           │
       │                                   │                           │
       │  POST /api/feedback               │                           │
       │ ─────── {preferred:"A",...} ─────►│                           │
       │                                   │                           │
       │  ◄── {received:true} ────────────│                           │
       │                                   │── writes feedback.json ──►│
       │  [inputs disabled,                │   (or feedback-pending    │
       │   "Return to agent" shown]        │    .json for regen)       │
       │                                   │                           │
       │                                   │                  [agent polls
       │                                   │                   every 5s,
       │                                   │                   reads file]

The Three Files

FileWritten whenMeansAgent action
feedback.jsonUser clicks SubmitFinal selection, doneRead it, proceed
feedback-pending.jsonUser clicks Regenerate/More Like ThisWants new optionsRead it, delete it, generate new variants, reload board
feedback.json (round 2+)User clicks Submit after regenerationFinal selection after iterationRead it, proceed

The State Machine

  $D serve starts
       │
       ▼
  ┌──────────┐
  │ SERVING  │◄──────────────────────────────────────┐
  │          │                                        │
  │ Board is │  POST /api/feedback                    │
  │ live,    │  {regenerated: true}                   │
  │ waiting  │──────────────────►┌──────────────┐     │
  │          │                   │ REGENERATING │     │
  │          │                   │              │     │
  └────┬─────┘                   │ Agent has    │     │
       │                         │ 10 min to    │     │
       │  POST /api/feedback     │ POST new     │     │
       │  {regenerated: false}   │ board HTML   │     │
       │                         └──────┬───────┘     │
       ▼                                │             │
  ┌──────────┐                POST /api/reload        │
  │  DONE    │                {html: "/new/board"}    │
  │          │                          │             │
  │ exit 0   │                          ▼             │
  └──────────┘                   ┌──────────────┐     │
                                 │  RELOADING   │─────┘
                                 │              │
                                 │ Board auto-  │
                                 │ refreshes    │
                                 │ (same tab)   │
                                 └──────────────┘

Port Discovery

The agent backgrounds $D serve and reads stderr for the port:

SERVE_STARTED: port=54321 html=/path/to/board.html
SERVE_BROWSER_OPENED: url=http://127.0.0.1:54321

The agent parses port=XXXXX from stderr. This port is needed later to POST /api/reload when the user requests regeneration. If the agent loses the port number, it cannot reload the board.

Why 127.0.0.1, Not localhost

localhost can resolve to IPv6 ::1 on some systems while Bun.serve() listens on IPv4 only. More importantly, localhost sends all dev cookies for every domain the developer has been working on. On a machine with many active sessions, this blows past Bun's default header size limit (HTTP 431 error). 127.0.0.1 avoids both issues.

Every Edge Case and Pitfall

1. The Zombie Form Problem

What: User submits feedback, the POST succeeds, the server exits. But the HTML page is still open in Chrome. It looks interactive. The user might edit their feedback and click Submit again. Nothing happens because the server is gone.

Fix: After successful POST, the board JS:

  • Disables ALL inputs (buttons, radios, textareas, star ratings)
  • Hides the Regenerate bar entirely
  • Replaces the Submit button with: "Feedback received! Return to your coding agent."
  • Shows: "Want to make more changes? Run /design-shotgun again."
  • The page becomes a read-only record of what was submitted

Implemented in: compare.ts:showPostSubmitState() (line 484)

2. The Dead Server Problem

What: The server times out (10 min default) or crashes while the user still has the board open. User clicks Submit. The fetch() fails silently.

Fix: The postFeedback() function has a .catch() handler. On network failure:

  • Shows red error banner: "Connection lost"
  • Displays the collected feedback JSON in a copyable <pre> block
  • User can copy-paste it directly into their coding agent

Implemented in: compare.ts:showPostFailure() (line 546)

3. The Stale Regeneration Spinner

What: User clicks Regenerate. Board shows spinner and polls /api/progress every 2 seconds. Agent crashes or takes too long to generate new variants. The spinner spins forever.

Fix: Progress polling has a hard 5-minute timeout (150 polls x 2s interval). After 5 minutes:

  • Spinner replaced with: "Something went wrong."
  • Shows: "Run /design-shotgun again in your coding agent."
  • Polling stops. Page becomes informational.

Implemented in: compare.ts:startProgressPolling() (line 511)

4. The file:// URL Problem (THE ORIGINAL BUG)

What: The skill template originally used $B goto file:///path/to/board.html. But browse/src/url-validation.ts:71 blocks file:// URLs for security. The fallback open file://... opens the user's macOS browser, but $B eval polls Playwright's headless browser (different process, never loaded the page). Agent polls empty DOM forever.

Fix: $D serve serves over HTTP. Never use file:// for the board. The --serve flag on $D compare combines board generation and HTTP serving in one command.

Evidence: See .context/attachments/image-v2.png — a real user hit this exact bug. The agent correctly diagnosed: (1) $B goto rejects file:// URLs, (2) no polling loop even with the browse daemon.

5. The Double-Click Race

What: User clicks Submit twice rapidly. Two POST requests arrive at the server. First one sets state to "done" and schedules exit(0) in 100ms. Second one arrives during that 100ms window.

Current state: NOT fully guarded. The handleFeedback() function doesn't check if state is already "done" before processing. The second POST would succeed and write a second feedback.json (harmless, same data). The exit still fires after 100ms.

Risk: Low. The board disables all inputs on the first successful POST response, so a second click would need to arrive within ~1ms. And both writes would contain the same feedback data.

Potential fix: Add if (state === 'done') return Response.json({error: 'already submitted'}, {status: 409}) at the top of handleFeedback().

6. The Port Coordination Problem

What: Agent backgrounds $D serve and parses port=54321 from stderr. Agent needs this port later to POST /api/reload during regeneration. If the agent loses context (conversation compresses, context window fills up), it may not remember the port.

Current state: The port is printed to stderr once. The agent must remember it. There is no port file written to disk.

Potential fix: Write a serve.pid or serve.port file next to the board HTML on startup. Agent can read it anytime:

cat "$_DESIGN_DIR/serve.port"  # → 54321

7. The Feedback File Cleanup Problem

What: feedback-pending.json from a regeneration round is left on disk. If the agent crashes before reading it, the next $D serve session finds a stale file.

Current state: The polling loop in the resolver template says to delete feedback-pending.json after reading it. But this depends on the agent following instructions perfectly. Stale files could confuse a new session.

Potential fix: $D serve could check for and delete stale feedback files on startup. Or: name files with timestamps (feedback-pending-1711555200.json).

8. Sequential Generate Rule

What: The underlying OpenAI GPT Image API rate-limits concurrent image generation requests. When 3 $D generate calls run in parallel, 1 succeeds and 2 get aborted.

Fix: The skill template must explicitly say: "Generate mockups ONE AT A TIME. Do not parallelize $D generate calls." This is a prompt-level instruction, not a code-level lock. The design binary does not enforce sequential execution.

Risk: Agents are trained to parallelize independent work. Without an explicit instruction, they will try to run 3 generates simultaneously. This wastes API calls and money.

9. The AskUserQuestion Redundancy

What: After the user submits feedback via the board (with preferred variant, ratings, comments all in the JSON), the agent asks them again: "Which variant do you prefer?" This is annoying. The whole point of the board is to avoid this.

Fix: The skill template must say: "Do NOT use AskUserQuestion to ask the user's preference. Read feedback.json, it contains their selection. Only AskUserQuestion to confirm you understood correctly, not to re-ask."

10. The CORS Problem

What: If the board HTML references external resources (fonts, images from CDN), the browser sends requests with Origin: http://127.0.0.1:PORT. Most CDNs allow this, but some might block it.

Current state: The server does not set CORS headers. The board HTML is self-contained (images base64-encoded, styles inline), so this hasn't been an issue in practice.

Risk: Low for current design. Would matter if the board loaded external resources.

11. The Large Payload Problem

What: No size limit on POST bodies to /api/feedback. If the board somehow sends a multi-MB payload, req.json() will parse it all into memory.

Current state: In practice, feedback JSON is ~500 bytes to ~2KB. The risk is theoretical, not practical. The board JS constructs a fixed-shape JSON object.

12. The fs.writeFileSync Error

What: feedback.json write in serve.ts:138 uses fs.writeFileSync() with no try/catch. If the disk is full or the directory is read-only, this throws and crashes the server. The user sees a spinner forever (server is dead, but board doesn't know).

Risk: Low in practice (the board HTML was just written to the same directory, proving it's writable). But a try/catch with a 500 response would be cleaner.

The Complete Flow (Step by Step)

Happy Path: User Picks on First Try

1. Agent runs: $D compare --images "A.png,B.png,C.png" --output board.html --serve &
2. $D serve starts Bun.serve() on random port (e.g. 54321)
3. $D serve opens http://127.0.0.1:54321 in user's browser
4. $D serve prints to stderr: SERVE_STARTED: port=54321 html=/path/board.html
5. $D serve writes board HTML with injected __GSTACK_SERVER_URL
6. User sees comparison board with 3 variants side by side
7. User picks Option B, rates A: 3/5, B: 5/5, C: 2/5
8. User writes "B has better spacing, go with that" in overall feedback
9. User clicks Submit
10. Board JS POSTs to http://127.0.0.1:54321/api/feedback
    Body: {"preferred":"B","ratings":{"A":3,"B":5,"C":2},"overall":"B has better spacing","regenerated":false}
11. Server writes feedback.json to disk (next to board.html)
12. Server prints feedback JSON to stdout
13. Server responds {received:true, action:"submitted"}
14. Board disables all inputs, shows "Return to your coding agent"
15. Server exits with code 0 after 100ms
16. Agent's polling loop finds feedback.json
17. Agent reads it, summarizes to user, proceeds

Regeneration Path: User Wants Different Options

1-6.  Same as above
7.  User clicks "Totally different" chiclet
8.  User clicks Regenerate
9.  Board JS POSTs to /api/feedback
    Body: {"regenerated":true,"regenerateAction":"different","preferred":"","ratings":{},...}
10. Server writes feedback-pending.json to disk
11. Server state → "regenerating"
12. Server responds {received:true, action:"regenerate"}
13. Board shows spinner: "Generating new designs..."
14. Board starts polling GET /api/progress every 2s

    Meanwhile, in the agent:
15. Agent's polling loop finds feedback-pending.json
16. Agent reads it, deletes it
17. Agent runs: $D variants --brief "totally different direction" --count 3
    (ONE AT A TIME, not parallel)
18. Agent runs: $D compare --images "new-A.png,new-B.png,new-C.png" --output board-v2.html
19. Agent POSTs: curl -X POST http://127.0.0.1:54321/api/reload -d '{"html":"/path/board-v2.html"}'
20. Server swaps htmlContent to new board
21. Server st