Replace document-review with Persona-Based Review Pipeline

Overview

Replace the single-voice document-review skill with a multi-persona review pipeline that dispatches specialized reviewer agents in parallel. Two always-on personas (coherence, feasibility) run on every review. Four conditional personas (product-lens, design-lens, security-lens, scope-guardian) activate based on document content analysis. Quality issues are auto-fixed; strategic questions are presented to the user.

Problem Frame

The current document-review applies five generic criteria (Clarity, Completeness, Specificity, Appropriate Level, YAGNI) through a single evaluator voice. This misses role-specific concerns: a security engineer, product leader, and design reviewer each see different problems in the same plan. The ce:review skill already demonstrates that multi-persona review produces richer, more actionable feedback for code. The same architecture applies to plan/requirements review. (see origin: docs/brainstorms/2026-03-23-plan-review-personas-requirements.md)

Requirements Trace

• R1. Replace document-review with persona pipeline dispatching specialized agents in parallel
• R2. 2 always-on personas: coherence, feasibility
• R3. 4 conditional personas: product-lens, design-lens, security-lens, scope-guardian
• R4. Auto-detect conditional persona relevance from document content
• R5. Hybrid action model: auto-fix quality issues, present strategic questions
• R6. Structured findings with confidence, dedup, synthesized report
• R7. Backward compatibility with all 4 callers (brainstorm, plan, plan-beta, deepen-plan-beta)
• R8. Pipeline-compatible for future automated workflows

Scope Boundaries

• Not adding new callers or pipeline integrations
• Not changing deepen-plan-beta behavior
• Not adding user configuration for persona selection
• Not inventing new review frameworks -- incorporating established review patterns into respective personas
• Not modifying any of the 4 existing caller skills

Context & Research

Relevant Code and Patterns

• plugins/compound-engineering/skills/ce-review/SKILL.md -- Multi-agent orchestration reference: parallel dispatch via Task tool, always-on + conditional agents, P1/P2/P3 severity, finding synthesis with dedup
• plugins/compound-engineering/skills/document-review/SKILL.md -- Current single-voice skill to replace. Key contract: "Review complete" terminal signal
• plugins/compound-engineering/agents/review/ce-*.agent.md -- 15 existing review agents. Frontmatter schema: name, description, model: inherit. Body: examples block, role definition, analysis protocol, output format
• plugins/compound-engineering/AGENTS.md -- Agent naming: fully-qualified compound-engineering:<category>:<agent-name>. Agent placement: agents/<category>/<name>.md

Caller Integration Points

All 4 callers use the same contract:

• ce-brainstorm/SKILL.md line 301: "Load the document-review skill and apply it to the requirements document"
• ce-plan/SKILL.md line 592: "Load document-review skill"
• ce-plan-beta/SKILL.md line 611: "Load the document-review skill with the plan path"
• deepen-plan-beta/SKILL.md line 402: "Load the document-review skill with the plan path"

All expect "Review complete" as the terminal signal. No callers check for specific output format. No caller changes needed.

Institutional Learnings

• Subagent design (docs/solutions/skill-design/compound-refresh-skill-improvements.md): Each persona agent needs explicit context (file path, scope, output format) -- don't rely on inherited context. Use native file tools, not shell commands. Avoid hardcoded tool names; use capability-first language with platform examples.
• Parallel dispatch safety: Persona reviewers are read-only (analyze the document, don't modify it). Parallel dispatch is safe. This differs from compound-refresh which used sequential subagents because they modified files.
• Contradictory findings: With 6 independent reviewers, findings will conflict (scope-guardian wants to cut; coherence wants to keep for narrative flow). Synthesis needs conflict-resolution rules, not just dedup.
• Classification pipeline ordering (docs/solutions/skill-design/claude-permissions-optimizer-classification-fix.md): Pipeline ordering matters: filter -> normalize -> group -> threshold -> re-classify -> output. Post-grouping safety checks catch misclassified findings. Single source of truth for classification logic.
• Beta skills framework (docs/solutions/skill-design/beta-skills-framework.md): Since we're replacing document-review entirely (not running side-by-side), the beta framework doesn't apply here.

Research Insights: iterative-engineering plan-review

The iterative-engineering plugin (v1.16.1) implements a mature plan-review skill with persona agents. Key architectural patterns to adopt:

Structured output contract: All personas return findings in a consistent JSON-like structure with: title (<=10 words), priority (HIGH/MEDIUM/LOW), section, line, why_it_matters (impact not symptom), confidence (0.0-1.0), evidence (quoted text, minimum 1), and optional suggestion. This consistency enables reliable synthesis.

Fingerprint-based dedup: normalize(section) + line_bucket(line, +/-5) + normalize(title). When fingerprints match: keep highest priority, highest confidence, union evidence, note all reviewers. This is more precise than judgment-based dedup.

Residual concerns: Findings below the confidence threshold (0.50) are stored separately as residual concerns. During synthesis, residual concerns are promoted to findings if they overlap with findings from other reviewers or describe concrete blocking risks. This catches issues that one persona sees dimly but another confirms.

Per-persona confidence calibration: Each persona defines its own confidence bands -- what HIGH (0.80+), MODERATE (0.60-0.79), and LOW mean for that persona's domain. This prevents apples-to-oranges confidence comparisons.

Explicit suppress conditions: Each persona lists what it should NOT flag (e.g., coherence suppresses style preferences and missing content; feasibility suppresses implementation style choices). This prevents noise and keeps personas focused.

Subagent prompt template: A shared template wraps each persona's identity + output schema + review context. This ensures consistent behavior across all personas without repeating boilerplate in each agent file.

Established Review Patterns

Three proven review approaches provide the behavioral foundation for specific personas:

Premise challenge pattern (-> product-lens persona):

• Nuclear scope challenge with 3 questions: (1) Is this the right problem? Could a different framing yield a simpler/more impactful solution? (2) What is the actual user/business outcome? Is the plan the most direct path? (3) What happens if we do nothing? Real pain or hypothetical?
• Implementation alternatives: Produce 2-3 approaches with effort (S/M/L/XL), risk (Low/Med/High), pros/cons
• Search-before-building: Layer 1 (conventional), Layer 2 (search results), Layer 3 (first principles)

Dimensional rating pattern (-> design-lens persona):

• 0-10 rating loop: Rate dimension -> explain gap ("4 because X; 10 would have Y") -> suggest fix -> re-rate -> repeat
• 7 evaluation passes: Information architecture, interaction state coverage, user journey/emotional arc, AI slop risk, design system alignment, responsive/a11y, unresolved design decisions
• AI slop blacklist: 10 recognizable AI-generated patterns to avoid (3-column feature grids, purple gradients, icons in colored circles, uniform border-radius, etc.)

Existing-code audit pattern (-> scope-guardian + feasibility personas):

• "What already exists?" check: (1) What existing code partially/fully solves each sub-problem? (2) What is minimum set of changes for stated goal? (3) Complexity check (>8 files or >2 new classes = smell). (4) Search check per architectural pattern. (5) TODOS cross-reference
• Completeness principle: With AI, completeness cost is 10-100x cheaper. If shortcut saves human hours but only minutes with AI, recommend complete version
• Error & rescue map: For every method/codepath that can fail, name the exception class, trigger, handler, and user-visible outcome

Key Technical Decisions

• Agents, not inline prompts: Persona reviewers are implemented as agent files under agents/review/. This enables parallel dispatch via Task tool, follows established patterns, and keeps the SKILL.md focused on orchestration. (Resolves deferred question from origin)

• Structured output contract aligned with ce:review-beta (PR #348): Same normalization mechanism -- findings-schema.json, subagent-template.md, review-output-template.md as reference files. Same field names and enums where applicable (severity P0-P3, autofix_class, owner, confidence, evidence). Document-specific adaptations: section replaces file+line, deferred_questions replaces testing_gaps, drop pre_existing. Each persona defines its own confidence calibration and suppress conditions. (Resolves deferred question from origin -- output format)

• Content-based activation heuristics: The orchestrator skill checks the document for keyword and structural patterns to select conditional personas. Heuristics are defined in the skill, not in the agents -- this keeps selection logic centralized and agents focused on review. (Resolves deferred question from origin)

• Separate auto-fix pass after synthesis: Personas are read-only (produce findings only). After dedup and synthesis, the orchestrator applies auto-fixes for quality issues in a single pass, then presents strategic questions. This prevents conflicting edits from multiple agents. (Resolves deferred question from origin)

• No caller modifications needed: The "Review complete" contract is sufficient. All 4 callers reference document-review by skill name and check for the terminal signal. (Resolves deferred question from origin)

• Fingerprint-based dedup over judgment-based: Use normalize(section) + normalize(title) fingerprinting for deterministic dedup. More reliable than asking the model to "remove duplicates" at synthesis time. When fingerprints match: keep highest priority, highest confidence, union evidence, note all agreeing reviewers.

• Residual concerns with cross-persona promotion: Findings below 0.50 confidence are stored as residual concerns. During synthesis, promote to findings if corroborated by another persona or if they describe concrete blocking risks. This catches issues one persona sees dimly but another confirms.

Open Questions

Resolved During Planning

• Agent category: Place under agents/review/ alongside existing code review agents. Names are distinct (coherence-reviewer, feasibility-reviewer, etc.) and don't conflict with existing agents. Fully-qualified: compound-engineering:review:<name>.
• Parallel vs serial dispatch: Always parallel. We have 2-6 agents per run (under the auto-serial threshold of 5 from ce:review's pattern). Even at max (6), these are document reviewers with bounded scope.
• Review pattern integration: Premise challenge -> product-lens opener. Dimensional rating -> design-lens evaluation method. Existing-code audit -> scope-guardian opener. These are incorporated as agent behavior, not separate orchestration mechanisms.
• Output format: Align with ce:review-beta (PR #348) normalization pattern. Same mechanism: JSON schema reference file, shared subagent template, output template. Same enums (P0-P3 severity, autofix_class, owner). Document-specific field swaps: section replaces file+line, deferred_questions replaces testing_gaps, drop pre_existing.

Deferred to Implementation

• Exact keyword lists for conditional persona activation -- start with the obvious signals, refine based on real usage
• Whether the auto-fix pass should re-read the document after applying changes to verify consistency, or trust a single pass

High-Level Technical Design

This illustrates the intended approach and is directional guidance for review, not implementation specification. The implementing agent should treat it as context, not code to reproduce.

Document Review Pipeline Flow:

1. READ document
2. CLASSIFY document type (requirements doc vs plan)
3. ANALYZE content for conditional persona signals
   - product signals? -> activate product-lens
   - design/UI signals? -> activate design-lens
   - security/auth signals? -> activate security-lens
   - scope/priority signals? -> activate scope-guardian
4. ANNOUNCE review team with per-conditional justifications
5. DISPATCH agents in parallel via Task tool
   - Always: coherence-reviewer, feasibility-reviewer
   - Conditional: activated personas from step 3
   - Each receives: subagent-template.md populated with persona + schema + doc content
6. COLLECT findings from all agents (validate against findings-schema.json)
7. SYNTHESIZE
   a. Validate: check structure compliance against schema, drop malformed
   b. Confidence gate: suppress findings below 0.50
   c. Deduplicate: fingerprint matching, keep highest severity/confidence
   d. Promote residual concerns: corroborated or blocking -> promote to finding
   e. Resolve contradictions: conflicting personas -> combined finding, manual + human
   f. Route: safe_auto -> apply, everything else -> present
8. APPLY safe_auto fixes (edit document inline, single pass)
9. PRESENT remaining findings to user, grouped by severity
10. FORMAT output using review-output-template.md
11. OFFER next action: "Refine again" or "Review complete"

Finding structure (aligned with ce:review-beta PR #348):

Envelope (per persona):
  reviewer:            Persona name (e.g., "coherence", "product-lens")
  findings:            Array of finding objects
  residual_risks:      Risks noticed but not confirmed as findings
  deferred_questions:  Questions that should be resolved in a later workflow stage

Finding object:
  title:               Short issue title (<=10 words)
  severity:            P0 / P1 / P2 / P3  (same scale as ce:review-beta)
  section:             Document section where issue appears (replaces file+line)
  why_it_matters:      Impact statement (what goes wrong if not addressed)
  autofix_class:       safe_auto / gated_auto / manual / advisory
  owner:               review-fixer / downstream-resolver / human / release
  requires_verification: Whether fix needs re-review
  suggested_fix:       Optional concrete fix (null if not obvious)
  confidence:          0.0-1.0 (calibrated per persona)
  evidence:            Quoted tex