tests-audit

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Tests Audit

测试套件审计

Find the tests that don't pin behavior and report what to do with each. This skill reports; it never edits the suite or the code under test. It does run the suite — order dependence and flakiness cannot be found any other way — so route any coverage output to a temp path rather than the repo. The suite command is project-specific, so it is deliberately outside

allowed-tools

: the mechanical scan is pre-approved, running the suite is not.

找出未锁定行为的测试，并报告每个测试的处理方式。本技能仅生成报告；绝不修改测试套件或被测代码。 它会运行测试套件——因为依赖顺序和不稳定问题无法通过其他方式发现——因此请将覆盖率输出路由到临时路径而非代码仓库。测试套件的运行命令因项目而异，因此被特意排除在

allowed-tools

之外：机械扫描已预先批准，但运行测试套件需要额外授权。

Core Principle

核心原则

A good test fails if and only if a promised behavior breaks.

Every bad test violates one direction of that biconditional:

Direction violated	Failure mode	Typical shapes
Fails when nothing broke	False alarm — erodes trust, blocks refactoring	implementation coupling, over-mocking, flaky timing, order dependence
Green when something broke	False confidence — coverage without protection	tautologies, weak asserts, snapshot rubber-stamps, self-fulfilling setups, over-claimed guarantees, trivia tests

A suite that cries wolf gets ignored; a suite that never cries protects nothing. Both cost maintenance and return nothing — fewer, sharper tests beat either. Deleting a bad test is a quality improvement, not a coverage regression.

Corollary: coverage percentage measures neither direction. 0% on a module is real information (a gap); high coverage proves nothing. A test whose only justification is a coverage number is a finding, not a defense — read which lines are uncovered instead. Uncovered defensive branches and exhaustiveness guards are correct; a coverage threshold is only worth raising for a gap in real behavior.

优质测试的唯一失败场景是承诺的行为被破坏。

每一个不良测试都会违反这个双向条件中的某一个方向：

违反的方向	失败模式	典型表现
无问题却失败	误报——削弱信任，阻碍重构	实现耦合、过度模拟、时序不稳定、依赖顺序
出问题却通过	虚假信心——有覆盖率却无防护	循环冗余断言、弱断言、快照橡皮图章式验证、自我实现的前置设置、过度夸大的保障、无关紧要的测试

频繁误报的测试套件会被忽略；从不报错的测试套件毫无防护作用。两者都需要维护成本却没有任何回报——更少、更精准的测试比这两种情况都要好。删除不良测试是质量提升，而非覆盖率倒退。

推论：覆盖率百分比无法衡量上述任何一个方向。模块覆盖率为0%是真实的信息（存在缺口）；高覆盖率则无法证明任何事情。仅以覆盖率为存在理由的测试是需要整改的对象，而非辩护的依据——应关注哪些代码行未被覆盖。未被覆盖的防御分支和穷尽性检查是合理的；仅当真实行为存在缺口时，才值得提高覆盖率阈值。

The Five-Question Gate

五问审核关卡

Every test must pass all five. Any "no" is a Tighten, Rewrite, or Delete verdict.

Contract — which one-sentence promise of the public API does it pin, and does the name claim no more than the assert delivers? No sentence → it tests implementation detail, a mock, or nothing. Name (or the CI job around it) over-claims → a false guarantee, worse than no test.
Refactor-proof — would it survive a behavior-preserving refactor of the internals? No → it's wrong by definition, regardless of what it has caught before. Non-negotiable.
Falsifiable — would it fail for a realistic bug? Flip a
```
<
```
to
```
<=
```
, break the formula, swap an argument. Sharpest form: name the dumbest implementation that still passes — if a constant, the identity function, or the test's own Arrange step satisfies the assert, it constrains nothing.
Diagnostic — does the name plus the failure diff identify the broken rule without a debugger?
Deterministic — same result every run, in any order: no real time, no real network, no shared mutable state, no sleeps.

每个测试必须通过全部五项审核。任何一项不通过都将给出优化、重写或删除的结论。

契约性——它锁定了公共API的哪一句承诺？测试名称是否没有超出断言所实现的内容？无法用一句话描述→它测试的是实现细节、模拟对象或无意义内容。测试名称（或其所在的CI任务）夸大其词→属于虚假保障，比没有测试更糟。
抗重构性——在不改变行为的前提下重构内部实现，测试是否仍能通过？不能→无论它之前捕获过什么问题，本质上都是错误的。这是不可协商的要求。
可证伪性——对于真实存在的bug，它是否会失败？比如将
```
<
```
改为
```
<=
```
、破坏公式、交换参数。最严格的形式：找出能通过测试的最简陋实现——如果常量、恒等函数或测试自身的Arrange步骤就能满足断言，那么它没有任何约束作用。
诊断性——测试名称加上失败差异信息，是否无需调试就能定位被破坏的规则？
确定性——无论运行多少次、顺序如何，结果都一致：不依赖真实时间、真实网络、共享可变状态或休眠操作。

Test Double Policy

Test Double 规范

The most-violated rule in real suites, so it gets its own section:

Double only unmanaged boundaries: network, clock, filesystem, randomness, other processes/services. Use real collaborators for everything you own.
Prefer hand-rolled fakes (in-memory repo) over interaction mocks — fakes verify state, mocks verify your assumptions about choreography.
Interaction asserts (
```
toHaveBeenCalledWith
```
, Mockito
```
verify
```
) are legitimate only when the outgoing call is the contract — the amount sent to a payment gateway, the params forwarded to an external API. Call counts only when the count is documented behavior ("retries exactly once"), never as a change detector.
Thin adapter caveat: when the module under test is a thin layer over a mocked service, asserting the response echoes the mock is a tautology. The real contract of an adapter is the translation: parameter mapping (an outgoing-call assert), response shaping, error mapping, authorization gating. Test those; if there's no translation, there's nothing to unit-test — cover it in an integration test or not at all.
Hard to test without heavy mocking? That's design feedback about the code under test, not a mocking problem — a hidden
```
new Date()
```
wants to be a parameter, buried IO wants to be injected, a god object wants splitting. Report it as a design finding on the module. Do not refactor production code to make a test cleaner; that is outside this skill.

这是实际测试套件中最常被违反的规则，因此单独列为一节：

仅对非受控边界使用替身：网络、时钟、文件系统、随机数、其他进程/服务。所有自主维护的协作对象都使用真实实例。
优先使用手动编写的伪对象（fake）（如内存仓库）而非交互模拟对象（mock）——伪对象验证状态，模拟对象验证你对协作流程的假设。
交互断言（
```
toHaveBeenCalledWith
```
、Mockito
```
verify
```
）仅在对外调用本身就是契约时才合法——比如发送给支付网关的金额、转发给外部API的参数。调用次数仅在次数是文档化行为时（如“恰好重试一次”）才验证，绝不能用作变更检测器。
轻量适配器注意事项：当被测模块是被模拟服务之上的轻量层时，断言响应与模拟对象一致属于循环冗余。适配器的真实契约是转换：参数映射（对外调用断言）、响应格式化、错误映射、授权控制。测试这些内容；如果没有转换逻辑，则无需进行单元测试——通过集成测试覆盖或不覆盖均可。
**不大量模拟就难以测试？**这是对被测代码的设计反馈，而非模拟问题——隐藏的
```
new Date()
```
应改为参数、内嵌的IO操作应改为依赖注入、上帝对象应拆分。将其作为模块的设计问题报告。不要为了让测试更简洁而重构生产代码；这超出了本技能的范围。

Workflow

工作流程

Scope: explicit argument → exactly that. Otherwise the project's test suite, sampled when it is too large to read whole — a mix of small and large files, pure-logic and mock-heavy, plus any e2e specs. Reviewing only the tests added in a diff is
```
review:changes-review
```
's job, not this skill's.
Mechanical scan for grep-able smells (weak asserts, sleeps,
```
.skip
```
/
```
@Disabled
```
,
```
.only
```
, loops in test bodies, mock round-trips, catch-only error tests) — commands in
```
references/audit-procedure.md
```
.
Dynamic checks: run the suite, then re-run it shuffled and repeated. Isolation, flakiness, and runtime don't grep — a green shuffled run is evidence no static audit can produce, and a red one is a High finding that names itself.

Per-test pass: run each test through the Five-Question Gate and the catalog in

references/anti-patterns.md

. Assign a verdict:

Verdict	When
Keep	Passes the gate
Tighten	Right contract, weak execution — imprecise asserts, bad name, sleep
Rewrite	Real contract worth pinning, but the test pins implementation or a mock
Delete	No contract sentence, duplicate coverage, trivia, rotting disabled test

Report using the template in
```
references/audit-procedure.md
```
. Name what the suite does well alongside the defects — an unnamed good pattern is one refactor from deletion. Close with an overall verdict and an ordered fix path.

Do not edit the suite, even when a fix is obvious — the report is the deliverable. For whoever applies it,

references/audit-procedure.md

§6 gives the fix-pass order and verification steps, and

references/writing-tests.md

gives the contract-first procedure for rewriting a flagged test. Reference both in the fix path so the report is actionable without this skill.

范围：指定参数→仅处理该范围。否则处理项目的整个测试套件，当套件过大无法全部审阅时进行抽样——混合小文件和大文件、纯逻辑测试和重度模拟测试，以及任何端到端（e2e）测试用例。仅审阅diff中新增的测试是
```
review:changes-review
```
的任务，而非本技能的职责。
机械扫描：查找可通过grep识别的不良模式（弱断言、休眠、
```
.skip
```
/
```
@Disabled
```
、
```
.only
```
、测试体中的循环、模拟往返、仅捕获错误的测试）——扫描命令见
```
references/audit-procedure.md
```
。
动态检查：运行测试套件，然后打乱顺序重复运行。隔离性、不稳定问题和运行时间无法通过grep发现——打乱顺序后仍能通过是静态审计无法提供的证据，而失败则是明确的高优先级问题。

逐测试审核：将每个测试通过五问审核关卡和

references/anti-patterns.md

中的分类进行评估，给出结论：

结论	适用场景
保留	通过所有审核关卡
优化	契约正确，但执行存在缺陷——断言不精确、名称不佳、使用休眠等
重写	存在值得锁定的真实契约，但测试锁定的是实现细节或模拟对象
删除	无明确契约描述、重复覆盖、无关紧要、长期禁用的测试

报告：使用
```
references/audit-procedure.md
```
中的模板。除了缺陷，还要指出套件的优点——未被明确提及的良好模式可能在重构时被删除。最后给出整体结论和有序的修复路径。

即使修复方案显而易见，也不要修改测试套件——报告是最终交付物。对于执行修复的人员，

references/audit-procedure.md

第6节给出了修复顺序和验证步骤，

references/writing-tests.md

给出了重写标记测试的契约优先流程。在修复路径中引用这两个文档，使报告无需依赖本技能即可执行。

Quick Reference

快速参考

Smell	Verdict → fix
Mock returns X, assert X comes back	Delete, or Rewrite against the translation the module performs
Name/CI claims a property (complexity, perf, security) the assert can't measure	Rewrite to what it does pin + rename, or Delete the claim
Arrange — or a grep of the SUT's own source — establishes what the Assert checks	Rewrite around the real producer: run it, inspect the artifact
All assertions live inside `catch`	Tighten: `toThrow` / a helper that fails when nothing throws
Property a constant or identity function would satisfy	Tighten to two-sided/metamorphic, or Delete
N feature tests all re-proving one mechanism	Not a finding — suite-level observation only
Expected value computed with SUT's formula	Tighten: replace with hand-computed constant
`toBeDefined` / `assertNotNull` / `not.toThrow` as the only assert	Tighten: assert the precise value or shape
Asserting internal call order/counts (undocumented)	Rewrite against observable output, or Delete
Mocking code you own (incl. whole UI libraries)	Rewrite with real collaborators; mock only the boundary
300-line snapshot nobody reads	Rewrite as explicit asserts on the parts that matter
`sleep` / `Thread.sleep` / real clock	Tighten: fake timers, injected clock, condition-based waits
`if` / `for` / `try-catch` in a test body; conditional mocks	Tighten: table test, or split into one test per branch
Loop over cases with one assert (which case failed?)	Tighten: `it.each` / `@ParameterizedTest`
Giant `beforeEach` / deep base-class setup (mystery guest)	Tighten: builders with defaults; DAMP over DRY
`it('fixes JIRA-4521')`	Tighten: rename to the rule the bug violated
Test depends on a previous test's state	Rewrite: each test arranges its own world
e2e: `waitFor...` with no assertion after it	Tighten: assert the outcome explicitly
Test asserts an acknowledged-wrong value ("should be 4, left as is")	Report separately: that's a bug, not a test defect
`.skip` / `@Disabled` / commented-out for months	Delete (git remembers) — Rewrite if its intent names an uncovered promise
Getters, framework wiring, generated code under test	Delete — cost > 0, information = 0

Full catalog with mechanisms, detection, and worked fixes:

references/anti-patterns.md

不良模式	结论→修复方案
模拟返回X，断言返回X	删除，或针对模块执行的转换逻辑重写
名称/CI声称断言无法衡量的属性（复杂度、性能、安全性）	重写为实际锁定的内容并改名，或删除该声称
Arrange步骤——或对被测系统（SUT）源码的grep——已经确定了断言的内容	围绕真实生成逻辑重写：运行它，检查输出结果
所有断言都在 `catch` 块内	优化：使用 `toThrow` / 无异常时会失败的辅助函数
断言可被常量或恒等函数满足	优化为双向/变形断言，或删除
N个功能测试重复验证同一机制	不属于单个测试问题——仅作为套件层面的观察结果
预期值通过被测系统（SUT）的公式计算得出	优化：替换为手动计算的常量
唯一断言是 `toBeDefined` / `assertNotNull` / `not.toThrow`	优化：断言精确值或结构
断言未文档化的内部调用顺序/次数	针对可观察输出重写，或删除
模拟自主维护的代码（包括整个UI库）	使用真实协作对象重写；仅模拟边界
无人阅读的300行快照	重写为对关键部分的显式断言
使用 `sleep` / `Thread.sleep` / 真实时钟	优化：使用伪造计时器、注入式时钟、基于条件的等待
测试体中包含 `if` / `for` / `try-catch` ；条件模拟	优化：使用表格测试，或拆分为每个分支对应一个测试
循环遍历用例但只有一个断言（无法定位哪个用例失败）	优化：使用 `it.each` / `@ParameterizedTest`
庞大的 `beforeEach` / 深层基类设置（神秘依赖）	优化：使用带默认值的构建器；优先DAMP（易读）而非DRY（复用）
`it('fixes JIRA-4521')`	优化：重命名为该bug违反的规则
测试依赖前一个测试的状态	重写：每个测试自行构建测试环境
端到端（e2e）： `waitFor...` 之后无断言	优化：显式断言结果
断言公认错误的值（如“应为4，暂保留现状”）	单独报告：这是bug，而非测试缺陷
`.skip` / `@Disabled` / 注释掉的测试已存在数月	删除（git会保留历史）——如果其意图对应未被覆盖的承诺，则重写
测试getter、框架配置、生成代码	删除——成本>0，信息价值=0

包含机制、检测方法和修复示例的完整分类：

references/anti-patterns.md

。

Common Mistakes

常见误区

Verdicting Delete on a test whose intent was real. Check question 1 of the gate first — a badly-written test for a real rule earns Rewrite, not Delete.
Reading repetition as duplication. Copy-pasted arrange blocks are a real Low finding (3.4), but test code optimizes for the reader of a failure, not for zero duplication. Never recommend DRY-ing tests into a shared-helper labyrinth as the fix.
Auditing e2e suites with unit-test rules. e2e tests legitimately chain steps and share a browser; hold them to determinism, explicit asserts, and independence — not to one-Act purity.
Reporting defects without calibration. "12 findings" reads the same for an excellent suite as for a rotten one. Name the strengths, rank the findings, and don't lead with redundant deterministic tests — they cost little, and a pass to lower a test count costs more review than it returns. Count measures effort; ask instead what each test is the only one to catch.

对意图合理的测试给出删除结论。首先检查审核关卡的第1项——针对真实规则的编写糟糕的测试应给出重写结论，而非删除。
将重复视为冗余。复制粘贴的Arrange块是真实的低优先级问题（3.4），但测试代码的优化目标是让查看失败信息的人易于理解，而非零重复。绝不要建议将测试重构为共享辅助函数的复杂结构作为修复方案。
用单元测试规则审核端到端（e2e）套件。端到端测试合理地串联步骤并共享浏览器；应要求它们具备确定性、显式断言和独立性——而非严格遵循单一Act原则。
报告缺陷时未进行校准。“12个问题”对于优秀套件和糟糕套件的含义完全不同。指出优点，对问题进行分级，不要以冗余的确定性测试作为重点——它们成本低，为了降低测试数量而通过审核所花费的评审成本高于其收益。数量衡量的是工作量；应关注每个测试唯一能捕获的问题。

References

参考文档

```
references/audit-procedure.md
```
— scan commands, dynamic checks, verdict rubric, report template, and the fix-pass order for whoever applies the report
```
references/anti-patterns.md
```
— full catalog: symptom, mechanism, detection, fix
```
references/writing-tests.md
```
— how to rewrite a flagged test: contract listing, naming, case selection, doubles, worked example, per-stack idioms (Vitest/TS, R3F, JUnit/Mockito, WebdriverIO)

```
references/audit-procedure.md
```
——扫描命令、动态检查、结论准则、报告模板，以及供执行修复人员使用的修复顺序
```
references/anti-patterns.md
```
——完整分类：症状、机制、检测方法、修复方案
```
references/writing-tests.md
```
——如何重写标记的测试：契约梳理、命名、用例选择、替身使用、示例、各栈惯用写法（Vitest/TS、R3F、JUnit/Mockito、WebdriverIO）