Testing Agents With Subagents

Operator Context

Hardcoded Behaviors (Always Apply)

• CLAUDE.md Compliance: Read and follow repository CLAUDE.md files before testing
• Over-Engineering Prevention: Only test what's directly needed. No elaborate test harnesses or infrastructure. Keep test cases focused and minimal.
• Verbatim Output Capture: Document exact agent outputs. NEVER summarize or paraphrase.
• Isolated Execution: Each test runs in a fresh subagent to avoid context pollution
• Evidence-Based Claims: Every claim about agent behavior MUST be backed by actual test execution
• No Self-Exemption: You cannot decide an agent doesn't need testing. Human partner must confirm exemptions.

Default Behaviors (ON unless disabled)

• Multi-Case Testing: Run at least 3 test cases per agent (success, failure, edge case)
• Output Schema Validation: Verify agent output matches expected structure and required sections
• Consistency Testing: Run same input 2+ times to verify deterministic behavior
• Regression Testing: After fixes, re-run ALL previous test cases before declaring green
• Temporary File Cleanup: Remove test files and artifacts at completion. Keep only files needed for documentation.
• Document Findings: Log all observations, hypotheses, and test results in structured format

Optional Behaviors (OFF unless enabled)

• A/B Testing: Compare agent variants using agent-comparison skill
• Performance Benchmarking: Measure response time and token usage
• Stress Testing: Test with large inputs, many iterations, concurrent requests
• Eval Harness Integration: Use

  evals/harness.py skill-test

  for YAML-based automated testing

What This Skill CAN Do

• Systematically validate agents through RED-GREEN-REFACTOR test cycles
• Dispatch subagents with controlled inputs and capture verbatim outputs
• Distinguish between output structure issues and behavioral correctness issues
• Verify fixes don't introduce regressions across the full test suite
• Test routing logic, skill invocation, and multi-agent workflows

What This Skill CANNOT Do

• Deploy agents without completing all three test phases
• Substitute reading agent prompts for executing actual test runs
• Make claims about agent behavior without evidence from subagent dispatch
• Evaluate agent quality structurally (use agent-evaluation instead)
• Skip the RED phase even when "the fix is obvious"

Instructions

Phase 0: PREPARE — Understand the Agent

# Read agent definition
cat agents/{agent-name}.md

# Read any referenced skills
cat skills/{skill-name}/SKILL.md

• What inputs does it accept?
• What output structure does it produce?
• What routing triggers should activate it?
• What error conditions does it handle?
• What skills does it invoke?

Phase 1: RED — Observe Current Behavior

## Test Plan: {agent-name}

**Agent Purpose:** {what the agent does}
**Agent File:** agents/{agent-name}.md
**Date:** {date}

**Test Cases:**

| ID | Input | Expected Output | Validates |
|----|-------|-----------------|-----------|
| T1 | {input} | {expected} | Happy path |
| T2 | {input} | {expected} | Error handling |
| T3 | {input} | {expected} | Edge case |

Task(
  prompt="""
  [Test input for the agent]

  Context: [Any required context]

  {Include the actual problem/request the agent should handle}
  """,
  subagent_type="{agent-name}"
)

## Test T1: Happy Path

**Input:**
{exact input provided}

**Expected Output:**
{what you expected}

**Actual Output:**
{verbatim output from agent — do not summarize}

**Result:** PASS / FAIL
**Failure Reason:** {if FAIL, exactly what was wrong}

• Which test categories fail (happy path, error, edge)?
• Are failures structural (missing sections) or behavioral (wrong answers)?
• Do failures correlate with input characteristics?

Phase 2: GREEN — Fix Agent Definition

## Fix Log

| Iteration | Change Made | Tests Passed | Tests Failed | Action |
|-----------|------------|-------------|-------------|--------|
| 1 | Added output schema | T1, T2 | T3 | Continue |
| 2 | Added error handling instruction | T1, T2, T3 | — | Green |

Phase 3: REFACTOR — Edge Cases and Robustness

• Same structure
• Same key findings (for analysis agents)
• Acceptable variation in phrasing only

## Test Report: {agent-name}

| Metric | Result |
|--------|--------|
| Test Cases Run | N |
| Passed | N |
| Failed | N |
| Pass Rate | N% |

## Verdict
READY FOR DEPLOYMENT / NEEDS FIXES / REQUIRES REVIEW

Examples

Example 1: Testing a New Reviewer Agent

1. Define 6 test cases: 2 real issues, 2 clean code, 1 edge case, 1 ambiguous (RED)
2. Dispatch subagent for each, capture verbatim outputs (RED)
3. Fix agent definition for any failures, re-run all tests (GREEN)
4. Add edge cases (empty input, malformed code), verify consistency (REFACTOR) Result: Agent passes all tests, report documents pass rate and verdict

Example 2: Testing After Agent Modification

1. Run existing test cases against modified agent (RED)
2. Compare outputs to previous baseline (RED)
3. Fix any regressions introduced by the modification (GREEN)
4. Test edge cases to verify robustness not degraded (REFACTOR) Result: Agent modification validated, no regressions confirmed

Example 3: Testing Routing Logic

1. Define test cases: "Review this Go code", "Fix this .go file", "Write a goroutine" (RED)
2. Dispatch each through router, verify correct agent handles it (RED)
3. Fix routing triggers if wrong agent selected (GREEN)
4. Test ambiguous inputs like "Review this code" with mixed-language context (REFACTOR) Result: Routing validated for all trigger phrases, ambiguous cases documented

Error Handling

Error: "Agent type not found"

1. Verify agent file exists:

   ls agents/{agent-name}.md

2. Check YAML frontmatter has correct

   name

   field
3. Restart Claude Code to pick up new agents

Error: "Inconsistent outputs across runs"

1. Document the inconsistency — this is a valid finding
2. Add more explicit instructions to agent definition
3. Re-test consistency after fix
4. Determine if variation is acceptable (phrasing) or problematic (structure/findings)

Error: "Subagent timeout"

1. Simplify test input to reduce processing
2. Check agent isn't in an infinite loop or excessive tool use
3. Increase timeout if agent legitimately needs more time

Error: "Agent asks questions instead of answering"

1. This may be correct behavior — agent properly requesting context
2. Update test input to provide the required context
3. Or update agent definition to handle ambiguity with defaults
4. Document whether questioning behavior is acceptable for this agent type

Anti-Patterns

Anti-Pattern 1: Testing Without Capturing Exact Output

Anti-Pattern 2: Testing Only Happy Path

Anti-Pattern 3: Skipping Re-test After Fixes

Anti-Pattern 4: Reading Prompts Instead of Running Agents

Anti-Pattern 5: Self-Exempting from Testing

References

• Anti-Rationalization — Prevents shortcut rationalizations
• Anti-Rationalization: Testing — Testing-specific rationalization blocks
• Verification Checklist — Pre-completion checks

Domain-Specific Anti-Rationalization

Integration

• agent-comparison

  : A/B test agent variants

• agent-evaluation

  : Structural quality checks

• test-driven-development

  : TDD principles applied to agents

Reference Files

• ${CLAUDE_SKILL_DIR}/references/testing-patterns.md

  : Dispatch patterns, test scenarios, eval harness integration