/judge - Meta-Judge + Single-Agent Work Evaluation

Evaluate completed work using a two-phase pipeline: meta-judge generates tailored evaluation criteria, then LLM-as-Judge applies them with context isolation and evidence-based scoring.

• Purpose - Assess quality of work produced earlier in conversation with isolated context
• Pattern - Context Extraction → Meta-Judge → Judge (with meta-judge spec) → Validation & Report
• Output - Evaluation report with weighted scores, evidence citations, and actionable improvements
• Efficiency - Single focused judge for fast evaluation

Quality Assurance

Enhanced verification with meta-judge rubric generation, Chain-of-Thought scoring, self-verification, and bias mitigation

Pattern: Meta-Judge → LLM-as-Judge with Context Isolation

This command implements a four-phase evaluation pipeline:

Phase 1: Context Extraction
         Review conversation history
         Identify work to evaluate
         Extract: Original task, output, files, constraints, artifact type
                     │
Phase 2: Meta-Judge (sadd:meta-judge)
         ┌─────────────────────────────────────────┐
         │ Receives extracted context + artifact    │
         │ type + evaluation focus                  │
         │                                         │
         │ Generates evaluation specification YAML: │
         │   - Tailored rubrics per artifact type   │
         │   - Checklists                           │
         │   - Scoring criteria and weights         │
         └─────────────────────────────────────────┘
                     │
Phase 3: Judge Sub-Agent (sadd:judge, Fresh Context)
         ┌─────────────────────────────────────────┐
         │ Receives ONLY extracted context          │
         │ + exact meta-judge specification YAML    │
         │ (prevents confirmation bias)             │
         │                                         │
         │ For each criterion from meta-judge spec: │
         │   1. Review evidence                    │
         │   2. Write justification                │
         │   3. Assign score (1-5)                 │
         │   4. Self-verify with questions         │
         │   5. Adjust if needed                   │
         └─────────────────────────────────────────┘
                     │
Phase 4: Validation & Report
         Verify scores in valid range (1-5)
         Check justification has evidence
         Confirm weighted total calculation
         Present verdict with recommendations

Usage

> Write new controller for the user model

# Evaluate completed work
/judge

# Evaluate with specific focus
/judge code quality and test coverage

# Evaluate security considerations
/judge security implications

# Evaluate requirements alignment
/judge requirements fulfillment

# Evaluate documentation completeness
/judge documentation

When to Use

Use single judge when:

• Quick quality check needed
• Work is straightforward with clear criteria
• Speed/cost matters more than multi-perspective analysis
• Evaluation is formative (guiding improvements), not summative
• Low-to-medium stakes decisions

Use judge-with-debate instead when:

• High-stakes decisions requiring rigorous evaluation
• Subjective criteria where perspectives differ legitimately
• Complex solutions with many evaluation dimensions
• You need defensible, consensus-based evaluation

Scoring Interpretation

Quality Enhancement Techniques

Theoretical Foundation

Based on:

• LLM-as-a-Judge (Zheng et al., 2023) - Structured evaluation rubrics with calibrated scoring
• Chain of Thought Prompting (Wei et al., 2022) - Reasoning before conclusion improves accuracy
• Constitutional AI (Bai et al., 2022) - Self-critique and verification loops
• Inference-Time Scaling of Verification (Wan et al., 2026) - Rubric-guided verification with test-time self-evolution and iterative feedback refinement