testing-craft

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Testing Craft

测试技艺

Durable, language-agnostic testing principles, plus a router to the dialect of whatever language you are testing. A test earns its keep by failing when behavior breaks and staying green through refactors.
这套持久化、与语言无关的测试原则,同时提供了针对你所测试语言的适配指引。一个合格的测试应当在行为异常时失败,而在代码重构时保持通过状态。

How to use this skill

如何使用本技能

  1. Apply the universal core below. It holds in every language.
  2. Detect the language(s) in scope from the files being touched, then read
    languages/<lang>.md
    for the test runner, fixture tools, and property-test library in that dialect. Load only the language(s) in scope.
  3. Run the project's own test suite (plus its type-check and lint gates) before claiming done.
For general code quality while writing the code under test, use the
code-craft
skill; this one covers the tests.
  1. 遵循以下通用核心原则。这些原则适用于所有语言。
  2. 从待处理文件中识别目标语言,然后阅读
    languages/<lang>.md
    了解该语言对应的测试运行器、Fixture工具和属性化测试库。仅加载目标语言的相关内容。
  3. 在完成操作前,运行项目自身的测试套件(包括类型检查和代码检查环节)。
在编写被测代码时,若需通用代码质量指导,请使用
code-craft
技能;本技能专注于测试相关内容。

Universal core

通用核心原则

1. Test behavior at the boundary, not the wiring

1. 在边界处测试行为,而非内部实现

Pick the feature boundary first, then test inputs to outputs across it. Good boundaries are the ones a user or a caller actually cares about: config parsing, request handling, a planning/translation step, a state transition. Tests that poke internal helpers directly tend to pin the current implementation and break on every refactor while catching few real bugs.
Prefer the public API. Exhaustive tests of a private helper, fed inputs no caller ever passes, are change detectors in disguise; if the helper deserves that much testing, it deserves to be a public unit.
A
check(input) -> expected_output
helper with a table of cases beats a dozen tests that each call private functions. Data in, data out, assert on the data.
优先选择功能边界,然后测试跨边界的输入输出。优质的边界是用户或调用方真正关心的环节:配置解析、请求处理、规划/转换步骤、状态转换。直接调用内部辅助函数的测试往往会固化当前实现,每次重构都会失败,却很少能捕获真实bug。
优先使用公开API。对私有辅助函数进行 exhaustive 测试,传入调用方永远不会传递的输入,本质上就是伪装的变更检测器;如果某个辅助函数值得如此细致的测试,那它就应该成为一个公开单元。
使用带有案例表的
check(input) -> expected_output
辅助函数,胜过十几个各自调用私有函数的测试。数据输入、数据输出,对数据进行断言即可。

2. No change-detector tests

2. 禁止使用变更检测器测试

A change-detector test is a transformation of the same information as the code under test. The absurd form reads the production source and asserts each line; that is obviously a checksum, not a test. The common form is only superficially different: mock every collaborator and verify the exact sequence of internal calls.
// Production code:
def process(w: Work)
  firstPart.process(w)
  secondPart.process(w)

// Test code:
part1 = mock(FirstPart)
part2 = mock(SecondPart)
w = Work()
Processor(part1, part2).process(w)
verify_in_order
  was_called part1.process(w)
  was_called part2.process(w)
This test restates the implementation. A correct and an incorrect program are equally likely to pass it, and any change to the production code breaks it without verifying the behavior of either version.
The symptom: a refactor with no behavior change turns the suite red, and fixing it means mechanically applying one transformation across many tests (add a parameter, update a hundred call sites with an empty string). When a test failure is fixed by mirroring the production edit rather than by thinking about behavior, the test is a change detector.
Change detectors have negative value: they catch no defects and their maintenance cost slows every future change. Rewrite them to assert state, a return value, rendered output, or persisted data, or delete them. Deletion is a valid fix; a test that cannot fail for a real reason protects nothing. Two questions expose one: could this test fail while observable behavior stays correct? Could behavior break while this test passes? A yes to either means rewrite or delete.
变更检测器测试是对被测代码信息的重复转换。极端的形式是读取生产源码并逐行断言;这显然是校验和,而非测试。常见形式只是表面上有所不同:模拟所有协作对象并验证内部调用的精确顺序。
// Production code:
def process(w: Work)
  firstPart.process(w)
  secondPart.process(w)

// Test code:
part1 = mock(FirstPart)
part2 = mock(SecondPart)
w = Work()
Processor(part1, part2).process(w)
verify_in_order
  was_called part1.process(w)
  was_called part2.process(w)
这个测试只是重述了实现逻辑。正确和错误的程序都同样可能通过它,而且对生产代码的任何修改都会导致测试失败,却无法验证两个版本的行为是否正确。
典型症状:不改变行为的重构会导致测试套件变红,修复测试意味着要机械地在大量测试中执行同一转换(添加参数、在一百个调用点更新空字符串)。当测试失败的修复方式是镜像生产代码的修改,而非思考行为逻辑时,该测试就是变更检测器。
变更检测器具有负面价值:它们无法捕获缺陷,维护成本还会拖慢未来的所有变更。将它们重写为断言状态、返回值、渲染输出或持久化数据,或者直接删除。删除是有效的修复方式;无法因真实原因失败的测试毫无保护作用。两个问题可以暴露变更检测器:当可观察行为保持正确时,这个测试是否可能失败?当行为出现问题时,这个测试是否可能通过?只要有一个答案为是,就需要重写或删除该测试。

3. Keep the core IO-free

3. 核心逻辑无IO操作

Push IO (filesystem, network, clock, randomness) to the edges so the core logic is a pure function you can test by building values in memory. A function that takes the parsed config and returns a plan is trivial to test; one that reads the file, calls the network, and writes the result is not. Separate them.
Give the shell honest seams: inject long-lived collaborators through the constructor and pass per-call data as method arguments. A hidden singleton, a static call, or a
now()
buried inside the logic under test is a dependency the test cannot control.
将IO操作(文件系统、网络、时钟、随机数)移至边缘,使核心逻辑成为可通过内存构建值来测试的纯函数。接收已解析配置并返回规划的函数测试起来轻而易举;而读取文件、调用网络并写入结果的函数则不然。请将它们分离。
为代码提供清晰的依赖注入点:通过构造函数注入长期协作对象,将每次调用的数据作为方法参数传递。被测逻辑内部隐藏的单例、静态调用或
now()
函数,都是测试无法控制的依赖。

4. Choose the double: real, fake, stub, mock, in that order

4. 按顺序选择测试替身:真实对象、Fake、Stub、Mock

Mocks that record "method X was called with Y" test the implementation, not the behavior (see change detectors above), and they drift from reality. Work down this list and stop at the first entry that fits:
  1. The real thing, when it is fast, deterministic, and in-process: pure functions, real temp directories, throwaway git repos, ephemeral databases (
    sqlx::test
    , testcontainers), a real local HTTP server.
  2. A fake: a simplified working implementation of the contract (an in-memory backend, a hermetic server), maintained by whoever owns the real type. Keep it narrow. If a fake is shared, contract-test it against the real implementation so the two cannot drift.
  3. A stub: canned return values for queries. Never verify that a query or getter was called; a query has no side effect worth asserting.
  4. A mock, last: verify only state-changing calls whose occurrence is itself the contract (charge the card once, send exactly one email, tell the view to show X), and verify only the arguments that matter. If you are unsure whether the call is the contract, rewrite to a state or return assertion.
Two hard rules:
  • Do not mock types you do not own. A mock of a third-party API encodes your guess about its behavior and goes stale silently. Wrap the library in a thin interface you own, double the wrapper, and test the wrapper against the real library.
  • Needing to mock more than one or two collaborators is a seam smell. So is a long stub/verify chain. The fix is in the production code: extract a narrower port and fake that.
If you find yourself asserting on call counts and argument matchers, you are testing the wrong layer.
记录“方法X被传入Y调用”的Mock测试的是实现,而非行为(参见上文的变更检测器),而且它们会与实际情况逐渐脱节。按以下顺序选择,找到第一个符合需求的选项即可:
  1. 真实对象:当它速度快、确定性强且在进程内时适用:纯函数、真实临时目录、临时Git仓库、临时数据库(
    sqlx::test
    、testcontainers)、真实本地HTTP服务器。
  2. Fake:契约的简化可行实现(内存后端、封闭服务器),由真实类型的维护者维护。保持范围狭窄。如果Fake是共享的,请针对真实实现进行契约测试,避免两者脱节。
  3. Stub:为查询提供预设返回值。永远不要验证查询或 getter 是否被调用;查询没有值得断言的副作用。
  4. Mock:最后选择的选项:仅验证那些调用本身就是契约的状态变更操作(刷卡一次、发送恰好一封邮件、告知视图显示X),且仅验证关键参数。若不确定调用是否属于契约,请重写为状态或返回值断言。
两条硬性规则:
  • 不要模拟你不拥有的类型。第三方API的Mock编码了你对其行为的猜测,会悄然过时。将库包装在你拥有的薄接口中,对包装器使用测试替身,并针对真实库测试包装器。
  • 需要模拟超过一两个协作对象是代码耦合的信号。冗长的Stub/验证链也是如此。修复方法在于生产代码:提取更窄的端口并使用Fake替代。
如果你发现自己在断言调用次数和参数匹配器,说明你测试的层级不正确。

5. Pick the cheapest layer that still catches the risk

5. 选择能捕获风险的最低成本层级

Name the bug you are defending against first, then take the topmost row of this table that can catch it. Escalate a row only when the cheaper layer would miss the risk, and say what it would miss.
RiskLayer
Pure logic, parsing, validation, state transitionsFast unit test through the public API
Collaborators you own, in-processReal objects or an owner-maintained fake
Service or HTTP contractOwner-maintained fake or hermetic server, not a handwritten request mock
UI wiring (disabled, unbound, hidden controls)Drive the rendered control (click, type), not the handler function
Cross-system critical pathA tiny end-to-end set: one path per use case plus the key error classes, asserting system outcomes, not copy or layout
When two layers could both catch the risk, compare them on SMURF: Speed, Maintainability, Utilization (resource cost), Reliability, Fidelity. Prefer the faster, cheaper, more reliable layer.
On coverage: meters find untested gaps; they do not certify quality. Cover both sides of every branch (an
if
with no
else
still has an implicit else worth a case). Do not enumerate input combinatorics; extract each predicate and cover it independently.
Keep unit tests in-source and close to the code, keep integration tests few and high-value behind one modular surface, and use doctests or runnable examples where the language supports them cheaply; they keep docs honest.
首先明确你要防范的bug,然后选择下表中能捕获该风险的最顶层选项。只有当较低成本层级无法捕获风险时,才升级到更高层级,并说明较低层级会遗漏什么。
风险层级
纯逻辑、解析、验证、状态转换通过公开API进行快速单元测试
你拥有的进程内协作对象真实对象或维护者提供的Fake
服务或HTTP契约维护者提供的Fake或封闭服务器,而非手写请求Mock
UI绑定(禁用、未绑定、隐藏控件)驱动渲染后的控件(点击、输入),而非处理函数
跨系统关键路径小型端到端测试集:每个用例一条路径加上关键错误类别,断言系统结果,而非文案或布局
当两个层级都能捕获风险时,通过SMURF进行比较:速度(Speed)、可维护性(Maintainability)、资源消耗(Utilization)、可靠性(Reliability)、保真度(Fidelity)。优先选择更快、成本更低、更可靠的层级。
关于覆盖率:覆盖率工具能发现未测试的缺口,但无法保证测试质量。覆盖每个分支的两侧(没有
else
的
if
仍有隐含的else分支值得测试)。不要枚举输入组合;提取每个谓词并独立覆盖。
将单元测试放在源码中,靠近被测代码;将集成测试控制在少量高价值的模块化接口后;在语言支持的情况下使用文档测试或可运行示例;它们能保证文档的准确性。

6. Author tests DAMP

6. 编写DAMP风格的测试

Tests have no tests, so a reader must be able to verify one by eye. That means DAMP (descriptive and meaningful phrases) beats DRY inside a test body: prefer a little repetition over indirection that hides cause and effect.
  • One behavior per test. Name the test for the scenario and the outcome it pins:
    parses_empty_config_as_default
    , not
    test_config_2
    . The name should read as a sentence about the system.
  • Arrange / act / assert in one block. Setup soup in a distant shared fixture is how wrong expected values sneak in. Multiple asserts are fine if they describe one outcome.
  • Helpers and builders may hide irrelevant construction; every value the scenario depends on stays visible in the test body. Do not assert against a default a helper set silently.
  • No logic in tests: literal inputs, literal expected outputs. A loop or conditional that computes the expectation can share a bug with production. If a helper genuinely needs logic, unit-test the helper.
  • Pick distinct, non-default values for each input.
    0
    ,
    ""
    , and the first enum variant match zero-initialized memory and let a broken implementation pass by accident.
  • Assert narrowly: the fields under test, not a whole-object snapshot whose irrelevant churn breaks the test. One full-equality check on the common happy object is enough.
  • Make failures actionable: precise matchers that print expected vs. actual (
    contains_entry
    , an
    is_ok
    that shows the error) over a bare
    assert(result.ok())
    . Compare floats with a tolerance, never exact equality.
  • Cover the unhappy paths: invalid input, empty input, boundary values, the error branches. Bugs live there.
测试本身没有测试,因此读者必须能够通过肉眼验证测试的正确性。这意味着在测试体内,DAMP(描述性有意义的短语)优于DRY:宁愿有一点重复,也不要使用隐藏因果关系的间接方式。
  • 每个测试对应一个行为。测试名称应描述场景和它验证的结果:
    parses_empty_config_as_default
    ,而非
    test_config_2
    。名称应读起来像一句关于系统的陈述。
  • 按Arrange/Act/Assert的结构组织测试块。在遥远的共享Fixture中进行零散的设置,会导致错误的预期值悄然混入。如果多个断言描述的是同一个结果,那么多个断言是可行的。
  • 辅助函数和构建器可以隐藏无关的构造;场景依赖的每个值都应在测试体内可见。不要对辅助函数静默设置的默认值进行断言。
  • 测试中不要包含逻辑:使用字面量输入和字面量预期输出。计算预期值的循环或条件可能与生产代码共享同一个bug。如果辅助函数确实需要逻辑,请对辅助函数进行单元测试。
  • 为每个输入选择不同的非默认值。
    0
    、
    ""
    和第一个枚举变体与零初始化内存匹配,会让有问题的实现意外通过测试。
  • 窄范围断言:仅断言被测字段,而非整个对象的快照——无关的变动会导致测试失败。对常见正常对象进行一次全相等检查即可。
  • 让失败信息可操作:使用能打印预期值与实际值的精确匹配器(
    contains_entry
    、显示错误信息的
    is_ok
    ),而非简单的
    assert(result.ok())
    。比较浮点数时使用容差,不要使用精确相等。
  • 覆盖异常路径:无效输入、空输入、边界值、错误分支。bug往往藏在这些地方。

7. Determinism

7. 确定性

Flaky tests are worse than no tests because they train people to ignore red.
  • No sleeps for synchronization. Do not
    sleep(100ms)
    and hope the work finished. Expose a join handle, a completion channel, a callback, or an observable side effect, and wait on that with a timeout.
  • Control time and randomness. Inject the clock and tick it; seed the RNG so a test is reproducible.
  • Isolate state. Each test gets its own temp dir / database / fixture; no shared mutable global that leaks between tests and orderings.
  • Force rare failures through the double. Timeouts, connection errors, and torn writes must be producible on demand; live infrastructure cannot do that for you.
不稳定的测试比没有测试更糟糕,因为它们会让人们习惯忽略测试失败。
  • 不要用睡眠来同步。不要使用
    sleep(100ms)
    并期望工作已完成。暴露join句柄、完成通道、回调或可观察的副作用,并通过超时等待这些信号。
  • 控制时间和随机性。注入时钟并手动推进时间;为随机数生成器设置种子,使测试可复现。
  • 隔离状态。每个测试都拥有自己的临时目录/数据库/Fixture;不要在测试之间共享可变全局状态,避免测试顺序影响结果。
  • 通过测试替身强制触发罕见失败。超时、连接错误和中断写入必须能够按需复现;实时基础设施无法做到这一点。

8. Property-based tests

8. 属性化测试

When the input space is large or has invariants (round-trip encode/decode, sort is a permutation, parse then serialize equals input), use property tests (proptest, fast-check, gopter, hypothesis) to generate cases and shrink failures. They find the edge you did not think to write.
当输入空间很大或存在不变量(编码/解码往返、排序是排列、解析后序列化等于输入)时,使用属性化测试(proptest、fast-check、gopter、hypothesis)生成测试用例并缩小失败范围。它们能发现你没想到的边缘情况。

9. Prove the test can fail

9. 验证测试可失败性

A test proves nothing until you have seen it red for the right reason.
  • Writing a test: run it green, then break the production code (or invert the assertion), confirm it fails with a useful message, and restore.
  • Reviewing a test you cannot run: name the one concrete behavior bug this assertion would catch. If you cannot, treat it as a change detector and do not approve it.
  • Refactoring tests: put production in a known-broken state first, so an assertion dropped during cleanup shows up as an unexpected green; then restore. Never do green-to-green test cleanup without that check.
只有当你看到测试因正确的原因失败时,它才能证明自身的价值。
  • 编写测试时:先运行测试使其通过,然后修改生产代码(或反转断言),确认测试会输出有用的失败信息,再恢复代码。
  • 评审无法运行的测试时:明确该断言能捕获的一个具体行为bug。如果无法明确,将其视为变更检测器,不要批准合并。
  • 重构测试时:先将生产代码置于已知的错误状态,这样清理过程中遗漏的断言会表现为意外的通过状态;然后恢复代码。永远不要在没有此检查的情况下进行从通过到通过的测试清理。

Review checklist

评审检查清单

A test you would merge:
  • Would fail on a real behavior bug, not only on a rename or extract
  • Asserts a result or state, not a call script (unless the call is the contract)
  • Exercises the public API or a user-facing path, not a private helper
  • Keeps cause and effect visible; the name states the outcome
  • Is deterministic: clock, IO, and fixture isolation handled
  • Fails with a message that starts the fix without extra logging
可合并的测试需满足:
  • 会因真实行为bug失败,而非仅在重命名或提取操作时失败
  • 断言结果或状态,而非调用流程(除非调用本身就是契约)
  • 测试公开API或用户可见路径,而非私有辅助函数
  • 因果关系清晰可见;名称明确说明预期结果
  • 具有确定性:时钟、IO和Fixture隔离已处理
  • 失败信息能直接指导修复,无需额外日志

Language router

语言适配指引

Detect the language, then read its file for the runner, fixture idioms, async test support, and property-test library.
LanguageFileDetect by
Rust
languages/rust.md
*.rs
,
Cargo.toml
TypeScript / JavaScript
languages/typescript.md
*.ts
,
*.tsx
,
*.js
,
tsconfig.json
,
package.json
Go
languages/go.md
*.go
,
go.mod
Python
languages/python.md
*.py
,
pyproject.toml
,
requirements.txt
For a language not listed, apply the universal core directly and follow the project's existing test conventions; the principles transfer.
识别语言后,阅读对应文件了解测试运行器、Fixture惯例、异步测试支持和属性化测试库。
语言文件识别依据
Rust
languages/rust.md
*.rs
,
Cargo.toml
TypeScript / JavaScript
languages/typescript.md
*.ts
,
*.tsx
,
*.js
,
tsconfig.json
,
package.json
Go
languages/go.md
*.go
,
go.mod
Python
languages/python.md
*.py
,
pyproject.toml
,
requirements.txt
对于未列出的语言,直接应用通用核心原则并遵循项目现有的测试惯例;这些原则具有通用性。

Precedence

优先级

Project instructions and existing test conventions win over this skill. If a repo's
AGENTS.md
/
CLAUDE.md
or its established patterns conflict with a principle here, follow the repo and say so. This skill is the default, not an override.
项目说明和现有测试惯例优先于本技能。如果仓库的
AGENTS.md
/
CLAUDE.md
或既定模式与本技能的原则冲突,请遵循仓库的要求并说明原因。本技能是默认方案,而非强制覆盖规则。

Provenance

来源

Distilled from Google's Testing on the Toilet series (2007-2026): the change-detector section is from Alex Eagle's "Change-Detector Tests Considered Harmful" (2015), and the double taxonomy, layer table, SMURF, DAMP rules, and prove-it-can-fail loop follow the episodes indexed in
references/episodes.md
, by way of
shamashel/testing-on-the-toilet
's distillation. The rest was carved out of the code-craft skill's testing principle.
提炼自Google的Testing on the Toilet系列(2007-2026):变更检测器部分来自Alex Eagle的《Change-Detector Tests Considered Harmful》(2015),测试替身分类、层级表、SMURF、DAMP规则和验证可失败性流程遵循
references/episodes.md
中索引的内容,源自
shamashel/testing-on-the-toilet
的提炼。其余内容来自code-craft技能的测试原则。