# Agent Review Protocol > **Trust but verify.** Every agent's work must be reviewed before acceptance. **Related:** [AGENT_GUARDRAILS.md](../AGENT_GUARDRAILS.md) | [ADVERSARIAL_TESTING.md](../standards/ADVERSARIAL_TESTING.md) | [CODE_REVIEW.md](./CODE_REVIEW.md) --- ## Overview This document establishes the **mandatory review protocol** for AI agent work. No agent's output should be accepted without verification by either another agent, a different LLM, or systematic automated checks. **Core Principle:** The agent that builds should NOT be the only agent that validates. --- ## WHY AGENT REVIEW IS MANDATORY ### The Hallucination Problem ``` PROBLEM: AI agents confidently produce incorrect output. SYMPTOMS: - Code that "looks right" but has subtle bugs - Tests that pass but don't actually test the right thing - Logic that works for happy path but fails edge cases - Confident assertions about code that doesn't exist SOLUTION: Independent verification breaks the hallucination loop. ``` ### The Context Contamination Problem ``` PROBLEM: Long sessions accumulate errors in context. SYMPTOMS: - Agent references earlier mistakes as "facts" - Corrections don't fully override original errors - Agent defends wrong approaches because they're in context - Quality degrades as session length increases SOLUTION: Fresh agent with clean context reviews the work. ``` --- ## REVIEW MODELS ### Model 1: Dual-Agent Review (Recommended) ``` ┌─────────────────────────────────────────────────────────────┐ │ DUAL-AGENT REVIEW │ ├─────────────────────────────────────────────────────────────┤ │ │ │ BUILDER AGENT REVIEWER AGENT │ │ ┌──────────────┐ ┌──────────────┐ │ │ │ │ │ │ │ │ │ Writes code │ ───Output───► │ Reviews code │ │ │ │ Writes tests│ │ Finds issues │ │ │ │ │ ◄───Feedback── │ │ │ │ └──────────────┘ └──────────────┘ │ │ │ │ RULES: │ │ - Different session (clean context) │ │ - Can be same model or different model │ │ - Reviewer has NO knowledge of builder's reasoning │ │ - Reviewer ONLY sees the output, not the process │ └─────────────────────────────────────────────────────────────┘ ``` ### Model 2: Cross-Model Review ``` ┌─────────────────────────────────────────────────────────────┐ │ CROSS-MODEL REVIEW │ ├─────────────────────────────────────────────────────────────┤ │ │ │ MODEL A (Builder) MODEL B (Reviewer) │ │ ┌──────────────┐ ┌──────────────┐ │ │ │ Claude │ ───Output───► │ GPT-4 │ │ │ │ (builds) │ │ (reviews) │ │ │ └──────────────┘ └──────────────┘ │ │ │ │ BENEFITS: │ │ - Different training data = different blind spots │ │ - Catches model-specific hallucinations │ │ - Diverse perspective on code quality │ │ │ │ GOOD COMBINATIONS: │ │ - Claude builds → GPT-4 reviews │ │ - GPT-4 builds → Claude reviews │ │ - Either builds → Gemini reviews │ └─────────────────────────────────────────────────────────────┘ ``` ### Model 3: Specialized Agent Review ``` ┌─────────────────────────────────────────────────────────────┐ │ SPECIALIZED AGENT REVIEW │ ├─────────────────────────────────────────────────────────────┤ │ │ │ BUILDER SECURITY QA AGENT ARCHITECT │ │ AGENT AGENT AGENT │ │ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐ │ │ │ │───────►│ │────►│ │───►│ │ │ │ │ Writes │ │Security│ │ Tests │ │Pattern │ │ │ │ code │ │ review │ │ review │ │ review │ │ │ └────────┘ └────────┘ └────────┘ └────────┘ │ │ │ │ SPECIALIST PERSONAS: │ │ - Security Agent: XSS, SQLi, auth vulnerabilities │ │ - QA Agent: Edge cases, error handling, test coverage │ │ - Architect Agent: Patterns, dependencies, scalability │ │ - Performance Agent: Complexity, memory, efficiency │ └─────────────────────────────────────────────────────────────┘ ``` ### Model 4: Automated + Agent Hybrid ``` ┌─────────────────────────────────────────────────────────────┐ │ HYBRID REVIEW PIPELINE │ ├─────────────────────────────────────────────────────────────┤ │ │ │ BUILDER AUTOMATED AGENT HUMAN │ │ AGENT CHECKS REVIEW REVIEW │ │ ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐│ │ │ │────►│ Lint │──────►│ │─────►│ ││ │ │ Writes │ │ Test │ │ Semantic│ │ Final ││ │ │ code │ │ Types │ │ review │ │ approval││ │ └────────┘ └────────┘ └────────┘ └────────┘│ │ │ │ AUTOMATED GATES (Must Pass): │ │ - Linting (ESLint, Pylint, etc.) │ │ - Type checking (TypeScript, MyPy, etc.) │ │ - Unit tests (all pass) │ │ - Security scan (no critical issues) │ │ │ │ AGENT REVIEW (After Automated): │ │ - Logic review │ │ - Architecture review │ │ - Edge case analysis │ │ │ │ HUMAN REVIEW (For Critical Code): │ │ - Security-sensitive code │ │ - Database migrations │ │ - Authentication changes │ └─────────────────────────────────────────────────────────────┘ ``` --- ## REVIEWER AGENT PROMPTS ### General Code Reviewer Prompt ```markdown ROLE: Senior Code Reviewer CONTEXT: You are reviewing code written by another AI agent. You have NO knowledge of the original requirements or the builder's reasoning. You only see the code output. TASK: Review the following code for: 1. CORRECTNESS - Does the logic appear correct? - Are there obvious bugs? - Are edge cases handled? 2. SECURITY - Any injection vulnerabilities? - Any hardcoded secrets? - Any authentication issues? 3. QUALITY - Is the code readable? - Are names descriptive? - Is complexity appropriate? 4. COMPLETENESS - Are error cases handled? - Is validation present? - Are tests comprehensive? CODE TO REVIEW: ``` [paste code here] ``` OUTPUT FORMAT: ## Review Summary - **Verdict:** APPROVE / REQUEST_CHANGES / REJECT - **Confidence:** HIGH / MEDIUM / LOW - **Critical Issues:** [count] - **Warnings:** [count] ## Critical Issues (Must Fix) 1. [Issue description, file, line] ## Warnings (Should Fix) 1. [Warning description, file, line] ## Suggestions (Optional) 1. [Suggestion] ## Questions for Builder 1. [Clarifying question] ``` ### Security-Focused Reviewer Prompt ```markdown ROLE: Security Engineer (Penetration Tester mindset) CONTEXT: Review code for security vulnerabilities. Assume the code will be attacked. Find every weakness. CHECKLIST: [ ] Input validation on all user inputs [ ] Output encoding for XSS prevention [ ] Parameterized queries for SQL [ ] Authentication on protected routes [ ] Authorization checks before actions [ ] No hardcoded secrets [ ] Secure session handling [ ] Rate limiting present [ ] Error messages don't leak info [ ] Dependencies have no known CVEs CODE TO REVIEW: ``` [paste code here] ``` REPORT VULNERABILITIES BY SEVERITY: - CRITICAL: Immediate exploit possible - HIGH: Significant security risk - MEDIUM: Should be fixed soon - LOW: Minor security concern ``` ### Test Quality Reviewer Prompt ```markdown ROLE: QA Engineer (Test Specialist) CONTEXT: Review tests written by another agent. Verify tests actually test what they claim to test. CHECKLIST: [ ] Tests cover happy path [ ] Tests cover error cases [ ] Tests cover edge cases [ ] Tests are independent (no shared state) [ ] Tests are deterministic (no random failures) [ ] Assertions are meaningful (not just "doesn't crash") [ ] Mocks are appropriate (not mocking the thing being tested) [ ] Test names describe what they test CODE TO REVIEW: ``` [paste test code here] ``` PRODUCTION CODE BEING TESTED: ``` [paste production code here] ``` IDENTIFY: 1. Missing test cases 2. Weak assertions 3. Tests that would pass even if code is broken 4. Tests that test implementation, not behavior ``` ### Architecture Reviewer Prompt ```markdown ROLE: Software Architect CONTEXT: Review code for architectural quality. Ensure it fits the project patterns and is maintainable. CHECKLIST: [ ] Follows established patterns (see PROJECT_CONTEXT.md) [ ] Appropriate separation of concerns [ ] No circular dependencies [ ] Appropriate abstraction level [ ] Scalable design [ ] Testable design [ ] Error handling strategy [ ] Logging strategy CODE TO REVIEW: ``` [paste code here] ``` PROJECT PATTERNS (from PROJECT_CONTEXT.md): ``` [paste relevant patterns] ``` EVALUATE: 1. Pattern compliance 2. Coupling assessment 3. Cohesion assessment 4. Suggestions for improvement ``` --- ## REVIEW WORKFLOW ### Standard Review Flow ``` ┌─────────────────────────────────────────────────────────────┐ │ REVIEW WORKFLOW │ ├─────────────────────────────────────────────────────────────┤ │ │ │ 1. BUILDER COMPLETES WORK │ │ └── Code written, tests pass, lint passes │ │ │ │ 2. AUTOMATED CHECKS (Blocking) │ │ ├── [ ] TypeScript/Lint passes │ │ ├── [ ] All tests pass │ │ ├── [ ] Security scan clean │ │ ├── [ ] Coverage threshold met │ │ ├── [ ] Regression check passes │ │ └── [ ] No failure registry patterns matched │ │ │ │ 3. PREPARE REVIEW PACKAGE │ │ ├── Git diff of changes │ │ ├── List of files modified │ │ ├── Test results summary │ │ ├── Builder's description of changes │ │ └── Regression check report │ │ │ │ 4. AGENT REVIEW (New session/different agent) │ │ ├── [ ] Logic review │ │ ├── [ ] Security review │ │ ├── [ ] Test quality review │ │ ├── [ ] Architecture review │ │ ├── [ ] Regression pattern review │ │ └── [ ] Previous fixes intact review │ │ │ │ 5. REVIEW OUTCOME │ │ ├── APPROVE → Proceed to merge │ │ ├── REQUEST_CHANGES → Return to builder │ │ └── REJECT → Escalate to human │ │ │ │ 6. IF REQUEST_CHANGES: │ │ ├── Builder addresses feedback │ │ ├── Re-run automated checks │ │ └── Re-submit for review (max 3 cycles) │ │ │ │ 7. FINAL APPROVAL │ │ └── Commit and merge │ └─────────────────────────────────────────────────────────────┘ ``` ### Regression Prevention Review Checklist ``` REGRESSION PREVENTION CHECKLIST (for Reviewer): Before Review: [ ] Run: python scripts/regression_check.py --staged [ ] Review any warnings or matches [ ] Check if files have known bug history During Review: [ ] Check if changes touch files with known bugs (see failure-registry.jsonl) [ ] Verify no regression patterns present in diff [ ] Verify no known bad patterns are being introduced [ ] Confirm previous fixes remain intact [ ] Check for null checks, error handling patterns from previous bugs For Bug Fixes: [ ] Confirm regression test exists [ ] Verify test naming: test__regression_ [ ] Verify test would fail with old code, pass with fix [ ] Check if prevention rule should be added/updated Post Review: [ ] If new bug found: Ensure builder will log it to registry [ ] If pattern emerges: Suggest prevention rule addition ``` ### Review Package Template ```markdown # Review Package: [Task Name] ## Summary - **Builder Agent:** [Agent/Model used] - **Session ID:** [If applicable] - **Task:** [Brief description] - **Files Changed:** [Count] ## Changes Overview [Brief description of what was done] ## Files Modified | File | Type | Lines Changed | |------|------|---------------| | src/components/Button.tsx | Modified | +45, -12 | | src/components/Button.test.tsx | Created | +78 | ## Git Diff ```diff [Paste git diff here] ``` ## Test Results - Total tests: X - Passed: X - Failed: 0 - Coverage: X% ## Builder Notes [Any context the builder wants to provide] ## Review Request Please review for: - [ ] Correctness - [ ] Security - [ ] Test quality - [ ] Architecture compliance ``` --- ## REVIEW DECISION MATRIX ### When to APPROVE ``` APPROVE if ALL of the following are true: [ ] No critical issues found [ ] No security vulnerabilities found [ ] Code is readable and maintainable [ ] Tests adequately cover the changes [ ] Architecture patterns are followed [ ] No obvious bugs or logic errors APPROVAL MESSAGE: "Reviewed and approved. No critical issues found. [Optional: Minor suggestions for future consideration]" ``` ### When to REQUEST_CHANGES ``` REQUEST_CHANGES if ANY of the following are true: [ ] Logic error that could cause incorrect behavior [ ] Missing error handling for likely scenarios [ ] Tests don't cover important cases [ ] Code violates project patterns [ ] Security concern (non-critical) [ ] Poor naming or readability issues REQUEST_CHANGES MESSAGE: "Changes requested. Please address the following before re-review: 1. [Specific issue and suggested fix] 2. [Specific issue and suggested fix]" ``` ### When to REJECT ``` REJECT if ANY of the following are true: [ ] Critical security vulnerability [ ] Obvious data loss risk [ ] Fundamental misunderstanding of requirements [ ] Code is unreviewable (too complex, no structure) [ ] Three review cycles without resolution REJECT MESSAGE: "Review rejected. This requires human intervention because: [Reason for rejection] Recommended action: [What should happen next]" ``` --- ## REVIEW CYCLE LIMITS ### Three Strikes Rule ``` REVIEW CYCLE LIMITS: MAX_REVIEW_CYCLES = 3 Cycle 1: Initial review └── If REQUEST_CHANGES → Builder fixes and resubmits Cycle 2: Second review └── If REQUEST_CHANGES → Builder fixes and resubmits Cycle 3: Final review └── If REQUEST_CHANGES → ESCALATE to human └── Do NOT allow further agent-only cycles RATIONALE: - Prevents infinite loops of fixes and reviews - Forces escalation when agent can't resolve issues - Protects against hallucination loops ``` ### Context Reset Between Cycles ``` CONTEXT RESET PROTOCOL: After each review cycle: 1. Start NEW reviewer session (fresh context) 2. Do NOT carry forward previous review comments 3. Reviewer should see ONLY: - Current code state - Test results - Original requirements 4. Prevents bias from previous review attempts ``` --- ## AUTOMATION INTEGRATION ### GitHub Actions Review Gate ```yaml # .github/workflows/agent-review.yml name: Agent Review Gate on: pull_request: types: [opened, synchronize] jobs: automated-checks: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Lint run: npm run lint - name: Type Check run: npm run typecheck - name: Test run: npm test - name: Security Scan run: npm audit --audit-level=high agent-review: needs: automated-checks runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Prepare Review Package run: | echo "## Review Package" > review-package.md echo "### Files Changed" >> review-package.md git diff --name-only ${{ github.event.pull_request.base.sha }} >> review-package.md echo "### Diff" >> review-package.md git diff ${{ github.event.pull_request.base.sha }} >> review-package.md - name: Request Agent Review # This would call your agent review API/service run: | # Example: Send to review agent endpoint curl -X POST ${{ secrets.AGENT_REVIEW_URL }} \ -H "Authorization: Bearer ${{ secrets.AGENT_REVIEW_TOKEN }}" \ -d @review-package.md - name: Check Review Result run: | # Check if review passed if [ "$REVIEW_RESULT" != "APPROVED" ]; then echo "Agent review did not approve changes" exit 1 fi ``` --- ## QUICK REFERENCE ``` +------------------------------------------------------------------+ | AGENT REVIEW PROTOCOL QUICK REFERENCE | +------------------------------------------------------------------+ | MODELS: | | 1. Dual-Agent: Builder → Reviewer (different session) | | 2. Cross-Model: Claude → GPT-4 (or vice versa) | | 3. Specialized: Builder → Security → QA → Architect | | 4. Hybrid: Automated → Agent → Human | +------------------------------------------------------------------+ | REVIEW FLOW: | | 1. Builder completes work | | 2. Automated checks pass (blocking) | | 3. Prepare review package (diff, tests, description) | | 4. Agent review (new session) | | 5. APPROVE / REQUEST_CHANGES / REJECT | +------------------------------------------------------------------+ | LIMITS: | | - MAX_REVIEW_CYCLES = 3 | | - After 3 cycles → Escalate to human | | - Fresh context for each review cycle | +------------------------------------------------------------------+ | VERDICTS: | | APPROVE: No critical issues, proceed to merge | | REQUEST_CHANGES: Fixable issues, return to builder | | REJECT: Critical issues, escalate to human | +------------------------------------------------------------------+ | KEY RULE: The agent that builds ≠ the agent that validates | +------------------------------------------------------------------+ ``` --- **Authored by:** TheArchitectit **Document Owner:** Project Maintainers **Last Updated:** 2026-01-21 **Line Count:** ~450