feat(resolve-pr-feedback): Add feedback clustering to detect systemic issues

Overview

Add a gated cluster analysis phase to the resolve-pr-feedback skill that detects when concentrated, thematically similar feedback signals a systemic issue rather than isolated bugs. The analysis is gated — it only runs when feedback patterns warrant it (same-file concentration, high volume, or verify-loop re-entry), keeping the common case (2-3 unrelated comments) at zero extra cost. When clusters are detected, dispatch a single investigation-aware agent per cluster that reads the broader area before fixing, rather than N individual fixers playing whack-a-mole. Verify-loop re-entry (new feedback after a fix round) automatically triggers the gate, so cross-cycle patterns are caught without a separate detection mechanism.

Problem Frame

The resolve-pr-feedback skill currently processes feedback items individually. The only grouping is same-file conflict avoidance (grouping threads that reference the same file into one agent dispatch). There is no semantic analysis of whether multiple feedback items collectively point to a deeper structural issue.

This leads to a whack-a-mole pattern:

1. Review bots post 4 comments about missing error handling across different functions in auth.ts
2. The skill fixes each one individually — adds a try/catch here, a null check there
3. The review bot re-runs and finds 3 more error handling gaps the individual fixes didn't cover
4. The cycle repeats because the underlying issue (the error handling strategy in that module) was never examined

The insight: individual comments don't say "this whole approach is wrong," but when you see 2+ comments about the same category of concern in the same area of code, the inference is that the approach in that area needs rethinking — not just N individual patches.

Requirements Trace

• R1. Detect thematic+spatial clusters in feedback before dispatching fix agents
• R2. When clusters are detected, investigate the broader area before making targeted fixes
• R3. Treat verify-loop re-entry (new feedback after a fix round) as a signal to investigate more broadly via the cluster analysis gate
• R4. Preserve existing behavior for non-clustered feedback (isolated items still get individual agents)
• R5. Keep the skill prompt-driven (no code changes — this is all SKILL.md and agent markdown)
• R6. Gate cluster analysis on signal strength — don't run it unconditionally on every pass, only when feedback patterns warrant the cost

Scope Boundaries

• No changes to the GraphQL scripts (fetch, reply, resolve)
• No changes to targeted mode (single-thread URL) — clustering only applies in full mode
• No new agents — extend the existing pr-comment-resolver agent with cluster context handling
• No changes to the verdict taxonomy (fixed, fixed-differently, replied, not-addressing, needs-human)
• Clustering is a signal for the orchestrator, not a new data structure or API

Context & Research

Relevant Code and Patterns

• plugins/compound-engineering/skills/resolve-pr-feedback/SKILL.md — the orchestrator skill, 285 lines
• plugins/compound-engineering/agents/workflow/ce-pr-comment-resolver.agent.md — the worker agent, 134 lines
• Current same-file grouping at SKILL.md lines 107-113 — conflict avoidance pattern to extend
• The ce:review skill's confidence-gated merge/dedup pipeline — precedent for pre-dispatch analysis
• The todo-resolve skill uses the same pr-comment-resolver agent and batching pattern

Institutional Learnings

• Whack-a-mole state machines (docs/solutions/skill-design/git-workflow-skills-need-explicit-state-machines-2026-03-27.md): Skills handling multiple dimensions of state need explicit re-verification after every mutating action. Directly applicable — after fixing a cluster, re-verify the whole area, not just the individual threads.
• Cluster before filter (docs/solutions/skill-design/claude-permissions-optimizer-classification-fix.md): Pipeline ordering is an architectural invariant. Group/cluster related items before deciding how to address them, otherwise individually below-threshold items that are part of a meaningful pattern get discarded.
• Status-gated resolution (docs/solutions/workflow/todo-status-lifecycle.md): Quality gates belong upstream in triage, not at the resolve boundary. The cluster analysis step is exactly this — a quality gate before dispatch.
• Pass paths not content (docs/solutions/skill-design/pass-paths-not-content-to-subagents-2026-03-26.md): When dispatching cluster-aware agents, pass thread IDs and file paths, not full comment bodies.

Key Technical Decisions

• Cluster analysis lives in the orchestrator (SKILL.md), not the agent: The orchestrator sees all feedback and can detect cross-thread patterns. Individual agents only see their assigned threads. The orchestrator synthesizes the cluster brief; the agent receives it as context alongside the thread details.

• Extend existing grouping rather than replacing it: The current same-file grouping (SKILL.md lines 107-113) already groups threads that reference the same file. Cluster analysis is a semantic layer on top of this — it groups by theme + proximity, and the same-file grouping becomes a special case of spatial proximity.

• Single agent per cluster, not a new "investigator" agent: The pr-comment-resolver agent already reads code, evaluates validity, and fixes. For clusters, it receives additional context (the cluster brief and all related threads) and follows an extended workflow: read the broader area first, assess root cause, then decide between holistic fix and individual fixes. This avoids a new agent and keeps the existing parallel dispatch architecture.

• Cross-cycle detection is a gate signal, not a separate mechanism: When the Verify step finds new feedback after a fix round, that re-entry automatically triggers the cluster analysis gate. No separate concern-category matching or structural comparison needed — the cluster analysis step handles thematic grouping with the just-fixed file context. This avoids the fragility of comparing LLM-generated category labels across inference passes.

• Cluster threshold: 2+ items with shared theme AND proximity: A single comment is never a cluster. Two items sharing both thematic similarity and spatial proximity form the minimum cluster. The threshold is deliberately low because the cost of investigating more broadly is small (agent time is cheap) and the cost of missing a systemic issue is high (another review loop).

• Cluster analysis is gated, not always-on: Running cluster analysis on every pass adds latency and token cost for the common case (2-3 unrelated comments). Instead, cluster analysis only fires when the feedback already shows concentration signals. The gate uses cheap, structural checks that are byproducts of triage — not new LLM inference. Gate signals: (a) volume threshold (4+ new items total — enough that patterns are plausible), or (b) verify-loop re-entry (new feedback appeared after a fix round — the strongest signal). Same-file concentration is deliberately excluded as a gate signal because it's the most common feedback pattern and is already handled by existing same-file grouping; it would cause the gate to fire on the majority of runs. If no gate signal fires, skip cluster analysis entirely and proceed directly to plan/dispatch as today.

• Verify-loop re-entry is a gate signal, not a separate comparison mechanism: Cross-cycle detection does not need its own concern-category matching or structural comparison. The fact that new feedback appeared after a fix round IS the whack-a-mole signal. Any verify-loop re-entry automatically triggers the cluster analysis gate. The cluster analysis step itself handles the thematic grouping — it doesn't need a separate mechanism to tell it "this is cross-cycle." On re-entry, the cluster analysis step receives which files were just fixed as additional context, so it can assess whether new feedback relates to just-fixed areas.

Open Questions

Resolved During Planning

• Should clusters replace or supplement individual dispatch? Supplement. Non-clustered items still get individual agents. A cluster dispatches one agent that handles all its threads together. Both can happen in the same run.
• Should the agent decide holistic vs. individual, or the orchestrator? The agent. The orchestrator detects the cluster and synthesizes the brief, but the agent reads the code and is better positioned to judge whether individual fixes suffice or a broader change is needed.
• How does the cluster brief get passed? In a <cluster-brief> XML block in the agent prompt — structurally delimited for unambiguous activation. The brief contains: theme label, affected directory/area, file paths, thread IDs, and a one-sentence hypothesis. No full comment bodies — the agent reads threads itself. This prevents accidental cluster mode activation (e.g., todo-resolve passing text that coincidentally mentions "cluster") and follows the pass-paths-not-content principle.

Deferred to Implementation

• Exact wording of the cluster analysis prompt: The heuristics are defined but the prompt phrasing that gets the LLM orchestrator to reliably detect clusters will need iteration.
• Whether the "holistic fix" mode needs examples in the agent: The agent may need 1-2 examples of cluster-aware evaluation in its <examples> section. Testing will show if the current examples plus the new workflow instructions are sufficient.

High-Level Technical Design

This illustrates the intended approach and is directional guidance for review, not implementation specification. The implementing agent should treat it as context, not code to reproduce.

Current flow:
  Fetch -> Triage -> Plan -> Dispatch(per-thread) -> Commit -> Reply -> Verify -> Summary

New flow:
  Fetch -> Triage -> [Gate Check] -> Plan -> Dispatch -> Commit -> Reply -> Verify -> Summary
                         |                     |                              |
                    Gate fires?            If clusters:                  New feedback?
                    /        \\             1 agent/cluster               /          \\
                 YES          NO           If isolated:              YES            NO
                  |            |            1 agent/thread        (re-entry         done
           Cluster Analysis    |            (same as today)     triggers gate)
                  |            |
           Synthesize briefs   |
                  \\           /
                   v         v
                 Plan step (unified)

Cluster analysis gate:

The gate uses cheap structural checks — byproducts of triage, not new LLM inference. Cluster analysis only runs when at least one gate signal fires:

Same-file concentration is deliberately NOT a gate signal. Multiple items on the same file is the most common feedback pattern and is already handled by existing same-file grouping for conflict avoidance. Running cluster analysis every time 2+ items hit the same file would add overhead to the majority of runs for little benefit. Same-file concentration is valuable inside the analysis (once the gate has fired for another reason) as a spatial proximity signal, but shouldn't open the gate itself.

If no gate signal fires (the common case: 1-3 items across different files), skip cluster analysis entirely and proceed to plan/dispatch with zero clustering overhead. If the first pass misses a cluster due to low volume, verify-loop re-entry catches it on the second pass.

Cluster detection decision matrix:

Spatial proximity is a hard requirement for clustering. Thematic similarity without proximity is better handled by cross-cycle escalation (Unit 4), which catches the case where the same theme keeps producing new issues across the codebase.

Spatial proximity means: same file, or files in the same directory subtree (e.g., src/auth/login.ts and src/auth/middleware.ts are proximate; src/auth/login.ts and src/database/pool.ts are not).

Cluster brief structure:

The cluster brief is passed to agents in a <cluster-brief> XML block for unambiguous activation. Contents are constrained to avoid inflating agent context:

<cluster-brief>
  <theme>Missing input validation</theme>
  <area>src/auth/</area>
  <files>src/auth/login.ts, src/auth/register.ts, src/auth/middleware.ts</files>
  <threads>PRRT_abc123, PRRT_def456, PRRT_ghi789</threads>
  <hypothesis>Individual validation gaps suggest the module lacks a consistent validation strategy</hypothesis>
</cluster-brief>

No full comment bodies in the brief. The agent reads threads via their IDs.

Cross-cycle escalation:

Verify re-fetch finds new threads
  -> Any new feedback after a fix round = verify-loop re-entry
  -> Re-entry automatically triggers the cluster analysis gate
  -> Cluster analysis receives additional context: files just fixed in previous cycle
  -> Cap at 2 fix-verify iterations before surfacing to user

No separate concern-category matching for cross-cycle detection. The re-entry itself is the signal. The cluster analysis step (which only runs because the gate fired) handles the thematic grouping and determines whether new feedback relates to just-fixed areas.

Implementation Units

• Unit 1: Add gated cluster analysis step to SKILL.md

Goal: Insert a gated step between Triage (Step 2) and Plan (Step 3) that checks whether feedback patterns warrant cluster analysis, and only runs the analysis when they do. The common case (2-3 unrelated comments) skips this step entirely.

Requirements: R1, R4, R6

Dependencies: None

Files:

• Modify: plugins/compound-engineering/skills/resolve-pr-feedback/SKILL.md

Approach:

• Add new "Step 2.5: Cluster Analysis (Gated)" after the triage step
• Gate check first: Before any thematic analysis, check two structural signals: (a) volume — 4+ new items total, (b) verify-loop re-entry — this is the 2nd+ pass thr