Pairwise Test Coverage
Combinatorial testing that covers all factor pairs in near-minimal test cases.
When to use: Multi-factor systems where exhaustive testing is impractical, state machines, retry/recovery logic, configuration matrices, compatibility testing, any code with 3+ interacting parameters.
When not to use: Single-factor tests (just test each value), two-factor systems (test all combinations directly), UI snapshot tests, type-only changes.
Core Philosophy
Exhaustive testing doesn't scale. If a system has 4 factors with 3 values each, that's 81 test cases. Pairwise testing covers all pair interactions in ~12 cases -- an 85% reduction with near-complete defect detection.
Rationalizations (Do Not Skip)
| Rationalization | Why It's Wrong | Required Action |
|---|
| "We'll test the important combinations" | Unexpected factor interactions go untested without systematic coverage | Generate the pairwise matrix |
| "81 test cases is fine" | 81 cases means 81 things to maintain and debug when they fail | Use pairwise to get 12 |
| "The test passes, so the code works" | Test must FAIL before the fix to prove it catches the bug | Validate detection first |
Quick Reference
| Technique | Use When | Key Pattern |
|---|
| Pairwise Matrix | Multiple factors with discrete values | table-driven |
| Property-Based | Value ranges, type safety invariants | fc.assert(fc.property(...))
|
| Model-Based | State machine transitions | Transition table tests |
| Fault Injection | Storage/network failures | Mock rejection scenarios |
| Contract Tests | Schema validation at boundaries | assertions |
Included Utilities
typescript
// Pairwise matrix generator (zero dependencies)
import { generatePairwiseMatrix, formatAsMarkdownTable } from './pairwise.ts';
// Pairwise test case helpers
import { createPairwiseTestCases, generateTestCaseName } from './test-fixtures.ts';
Performance and Limits
The greedy algorithm never enumerates the Cartesian product. Pair count grows as O(factors² × values²).
| Factors | Values | Cartesian | Pairwise Cases | Time |
|---|
| 3 | 3 | 27 | ~10 | <1ms |
| 8 | 4 | 65,536 | ~46 | ~2ms |
| 8 | 8 | 16,777,216 | ~100 | ~10ms |
Hard limits (throws if exceeded): max 20 factors, max 50 values per factor. Beyond these, pair count exceeds ~475K (C(20,2) × 50²) and in-memory generation becomes impractical.
Violation Rules
| Slug | Rule | Severity |
|---|
missing_pairwise_coverage
| Multi-factor code changes need pairwise tests | must-fail |
bug_detection_not_validated
| Tests must fail before fix, pass after | must-fail |
Definition of Done
Companion Skills
This skill provides combinatorial test matrix generation, not test design guidance. For broader methodology:
- Search on skills.sh for constraint handling, higher-strength covering arrays, and test oracle strategies
- Guard truth tables with 5+ boolean inputs use pairwise for coverage — use model-based-testing for state machine transition matrices and guard truth table generation
- Zod schemas with 5+ optional fields use pairwise for compound state coverage — use zod-contract-testing for schema boundary validation and compound state matrices
Details
See references for:
- workflow.md: Step-by-step implementation guide
- violations.md: Full violation rules with detection patterns
- examples.md: 6 testing technique examples (pairwise, property-based, model-based, fault injection, contract, observability)