Back to Details

test-gap-analysis

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Test Gap Analysis via Pseudo-Mutation

基于伪变异的测试缺口分析

Analyze .NET production code by reasoning about hypothetical mutations and checking whether existing tests would catch them. This reveals blind spots where tests pass but would continue to pass even if the code were broken.
通过推理假设性变异并检查现有测试是否能捕获这些变异，来分析.NET生产代码。这会揭示测试通过但即使代码出现问题仍会通过的盲区。

Why Pseudo-Mutation Matters

伪变异的重要性

Code coverage tells you what code ran during tests. It does not tell you whether tests would fail if that code were wrong. A method can have 100% line coverage but zero tests that would catch a sign flip, an off-by-one error, or a removed null check.
Pseudo-mutation analysis asks: "If I changed this line, would any test fail?" When the answer is "no," you've found a test gap.
Coverage MetricWhat It MeasuresWhat It Misses
Line coverageWhich lines executedWhether assertions verify those lines' behavior
Branch coverageWhich branches takenWhether both branches produce different asserted outcomes
Mutation scoreWhether tests detect code changesNothing — this is the gold standard
This skill performs static pseudo-mutation — reasoning about mutations without actually running them — to approximate mutation testing at the speed of code review.
代码覆盖率告诉你测试期间运行了哪些代码，但无法告诉你如果这些代码出错，测试是否会失败。一个方法可能有100%的行覆盖率，但没有任何测试能捕获符号翻转、差一错误或移除的null检查。
伪变异分析提出问题："如果我修改这行代码，会有测试失败吗？" 当答案是“否”时，你就找到了一个测试缺口。
覆盖率指标测量内容遗漏内容
行覆盖率执行了哪些代码行断言是否验证了这些代码行的行为
分支覆盖率执行了哪些分支两个分支是否产生不同的断言结果
变异分数测试是否能检测到代码变更无——这是黄金标准
此技能执行静态伪变异——无需实际运行即可推理变异——以代码审查的速度近似变异测试。

When to Use

使用场景

  • User asks "would my tests catch a bug in this code?"
  • User wants to find weak or shallow tests
  • User wants to evaluate test effectiveness beyond coverage
  • User asks for mutation testing or mutation analysis
  • User asks "where are my tests blind?"
  • User wants to prioritize which tests to strengthen
  • 用户询问“我的测试能捕获这段代码中的bug吗？”
  • 用户想要寻找薄弱或浅层测试
  • 用户想要评估超出覆盖率的测试有效性
  • 用户询问变异测试或变异分析
  • 用户询问“我的测试盲区在哪里？”
  • 用户想要确定优先强化哪些测试

When Not to Use

不适用场景

  • User wants to write new tests from scratch (use 
    writing-mstest-tests
    )
  • User wants to detect test anti-patterns like flakiness or poor naming (use 
    test-anti-patterns
    )
  • User wants to measure assertion variety (use 
    assertion-quality
    )
  • User wants to run an actual mutation testing framework like Stryker (help them directly)
  • User only wants code coverage numbers (out of scope)
  • 用户想要从头编写新测试（使用
    writing-mstest-tests
    ）
  • 用户想要检测测试反模式，如不稳定或命名不佳（使用
    test-anti-patterns
    ）
  • 用户想要衡量断言多样性（使用
    assertion-quality
    ）
  • 用户想要运行实际的变异测试框架如Stryker（直接提供帮助）
  • 用户只想要代码覆盖率数值（超出范围）

Inputs

输入

InputRequiredDescription
Production codeYesThe source files to analyze for mutation points
Test codeYesThe test files that cover the production code
Focus areaNoA specific mutation category or code region to focus on
输入是否必填描述
生产代码是要分析变异点的源文件
测试代码是覆盖生产代码的测试文件
重点区域否要聚焦的特定变异类别或代码区域

Workflow

工作流程

Step 1: Gather production and test code

步骤1：收集生产代码和测试代码

Read both the production code and its corresponding test files. If the user points to a directory, identify production/test pairs by convention (e.g., 
Calculator.cs
tested by 
CalculatorTests.cs
).
Establish which production methods are exercised by which test methods — trace this through method calls in test code, setup, and helper methods.
读取生产代码及其对应的测试文件。如果用户指向一个目录，按惯例识别生产/测试文件对（例如，
Calculator.cs
由
CalculatorTests.cs
测试）。
确定哪些生产方法由哪些测试方法执行——通过测试代码、设置和辅助方法中的方法调用追踪这一点。

Step 2: Identify mutation points

步骤2：识别变异点

Scan the production code and annotate every location where a mutation could reveal a test gap. Use the mutation catalog below.
扫描生产代码并标注所有变异可能揭示测试缺口的位置。使用下面的变异目录。

Boundary Mutations

边界变异

OriginalMutationWhat it tests
<
<=
Off-by-one at upper bound
>
>=
Off-by-one at lower bound
<=
<
Boundary inclusion
>=
>
Boundary inclusion
== 0
== 1
or 
<= 0
Zero-boundary handling
i < length
i < length - 1
or 
i <= length
Loop boundary
index + 1
index
or 
index + 2
Index arithmetic
原始代码变异代码测试内容
<
<=
上边界的差一错误
>
>=
下边界的差一错误
<=
<
边界包含情况
>=
>
边界包含情况
== 0
== 1
或 
<= 0
零边界处理
i < length
i < length - 1
或 
i <= length
循环边界
index + 1
index
或 
index + 2
索引算术运算

Boolean and Logic Mutations

布尔与逻辑变异

OriginalMutationWhat it tests
&&
||
Condition independence
||
&&
Condition necessity
!condition
condition
Negation correctness
if (x)
if (!x)
Branch selection
true
(constant)
false
Hardcoded assumption
flag || other
other
Short-circuit first operand
原始代码变异代码测试内容
&&
`
``
!condition
condition
否定正确性
if (x)
if (!x)
分支选择
true
（常量）
false
硬编码假设
`flagother`

Return Value Mutations

返回值变异

OriginalMutationWhat it tests
return result
return null
Null handling downstream
return result
return default
Default value handling
return true
return false
Boolean return verification
return list
return new List<T>()
Empty collection handling
return count
return 0
or 
return count + 1
Numeric return verification
return string
return ""
or 
return null
String return verification
原始代码变异代码测试内容
return result
return null
下游null处理
return result
return default
默认值处理
return true
return false
布尔返回值验证
return list
return new List<T>()
空集合处理
return count
return 0
或 
return count + 1
数值返回值验证
return string
return ""
或 
return null
字符串返回值验证

Exception Removal Mutations

异常移除变异

OriginalMutationWhat it tests
throw new ArgumentNullException(...)
(remove entire throw)Guard clause verification
throw new InvalidOperationException(...)
(remove entire throw)State validation testing
if (x == null) throw ...
(remove entire guard)Null guard testing
if (!IsValid()) throw ...
(remove entire check)Validation testing
原始代码变异代码测试内容
throw new ArgumentNullException(...)
(移除整个throw语句)保护子句验证
throw new InvalidOperationException(...)
(移除整个throw语句)状态验证测试
if (x == null) throw ...
(移除整个保护子句)Null保护测试
if (!IsValid()) throw ...
(移除整个检查)验证测试

Arithmetic Mutations

算术变异

OriginalMutationWhat it tests
a + b
a - b
Addition correctness
a - b
a + b
Subtraction correctness
a * b
a / b
Multiplication correctness
a / b
a * b
Division correctness
a % b
a / b
Modulo correctness
x++
x--
Increment direction
-value
value
Sign flip
原始代码变异代码测试内容
a + b
a - b
加法正确性
a - b
a + b
减法正确性
a * b
a / b
乘法正确性
a / b
a * b
除法正确性
a % b
a / b
取模正确性
x++
x--
增量方向
-value
value
符号翻转

Null-Check Removal Mutations

Null检查移除变异

OriginalMutationWhat it tests
if (x == null) return ...
(remove null check)Null path coverage
if (x != null) { ... }
(always enter block)Null guard necessity
x ?? defaultValue
x
Null coalescing coverage
x?.Method()
x.Method()
Null-conditional coverage
x!
x
Null-forgiving operator necessity
原始代码变异代码测试内容
if (x == null) return ...
(移除null检查)Null路径覆盖
if (x != null) { ... }
(始终进入代码块)Null保护必要性
x ?? defaultValue
x
Null合并覆盖
x?.Method()
x.Method()
Null条件覆盖
x!
x
Null原谅运算符必要性

Step 3: Evaluate each mutation against tests

步骤3：针对测试评估每个变异

For each identified mutation point, reason about whether existing tests would detect the change:
  1. Find covering tests — Which test methods exercise the mutated line? Follow call chains through helpers and setup methods.
  2. Check assertion relevance — Do those tests assert something that would change if the mutation were applied? A test that calls the method but only asserts an unrelated property would NOT catch the mutation.
  3. Classify the mutation as:
VerdictMeaningAction
KilledAt least one test would fail if this mutation were appliedNo action needed — tests are effective here
SurvivedNo test would fail — the mutation would go undetectedThis is a test gap — recommend a test improvement
No coverageNo test exercises this code path at allWorse than survived — the code is untested
EquivalentThe mutation produces identical behavior (e.g., 
x * 1
→ 
x / 1
)		Skip — not a real mutation
对于每个识别出的变异点，推理现有测试是否能检测到变更：
  1. 找到覆盖测试——哪些测试方法执行了变异代码行？通过辅助方法和设置方法追踪调用链。
  2. 检查断言相关性——这些测试是否断言了应用变异后会改变的内容？调用方法但仅断言不相关属性的测试不会捕获变异。
  3. 将变异分类为：
结论含义操作
已杀死应用此变异后至少有一个测试会失败无需操作——此处测试有效
存活没有测试会失败——变异不会被检测到这是测试缺口——建议改进测试
无覆盖没有测试执行此代码路径比存活更糟——代码未被测试
等效变异产生相同行为（例如，
x * 1
→ 
x / 1
）		跳过——不是真正的变异

Step 4: Calibrate findings

步骤4：校准结果

Before reporting, apply these calibration rules:
  • Don't flag trivial code. Simple property getters (
    return _name;
    ), auto-properties, and boilerplate don't need mutation analysis. Focus on logic, conditions, calculations, and error handling.
  • Consider defensive depth. If a null guard has a survived mutation but the caller also checks for null, note the redundancy but rate it lower priority.
  • Equivalent mutations are not gaps. If changing 
    >=
    to 
    >
    doesn't alter behavior because the 
    ==
    case is impossible given the domain, mark it Equivalent and skip.
  • Private methods reached through public API are valid targets. Trace through the call chain — a private method called from a tested public method may still have survived mutations if the test doesn't assert the specific behavior affected.
  • Rate by risk, not count. A single survived mutation in payment calculation logic is more important than five survived mutations in logging code.
报告前应用以下校准规则：
  • 不要标记琐碎代码。简单的属性获取器（
    return _name;
    ）、自动属性和样板代码不需要变异分析。聚焦于逻辑、条件、计算和错误处理。
  • 考虑防御深度。如果一个null保护的变异存活，但调用者也检查了null，记录冗余性但降低优先级。
  • 等效变异不是缺口。如果将
    >=
    改为
    >
    不会改变行为（因为
    ==
    情况在领域中不可能出现），标记为等效并跳过。
  • 通过公共API访问的私有方法是有效目标。追踪调用链——从已测试的公共方法调用的私有方法如果测试未断言受影响的特定行为，仍可能存在存活变异。
  • 按风险而非数量评级。支付计算逻辑中的一个存活变异比日志代码中的五个存活变异更重要。

Step 5: Report findings

步骤5：报告结果

Present the analysis in this structure:
  1. Summary — Overall mutation score and key findings:
    | Metric              | Value    |
|---------------------|----------|
| Mutation points      | 42       |
| Killed               | 28 (67%) |
| Survived             | 10 (24%) |
| No coverage          | 2 (5%)   |
| Equivalent (skipped) | 2 (5%)   |
  2. Survived Mutations (Test Gaps) — For each survived mutation, report:
    • Location: File, method, line
    • Mutation category: Boundary / Boolean / Return value / Exception / Arithmetic / Null-check
    • Original code: The current code
    • Hypothetical mutation: What would change
    • Why it survives: Which tests cover this code and why their assertions miss it
    • Recommended fix: A concrete test assertion or new test case that would kill this mutation
    Group by priority: high-risk survived mutations first (business logic, calculations, security checks), lower-risk last (logging, formatting).
  3. No-Coverage Zones — Code paths that no test reaches at all. These are worse than survived mutations.
  4. Killed Mutations (Strengths) — Briefly note areas where tests are effective. Highlight well-tested methods and strong assertion patterns. Don't enumerate every killed mutation — summarize.
  5. Recommendations — Prioritized list:
    • Which survived mutations to address first (by risk)
    • Specific test methods to add or strengthen
    • Patterns the team can adopt to prevent future gaps (e.g., always test boundary values, always assert exception types)
按以下结构呈现分析：
  1. 摘要——总体变异分数和关键发现：
    | 指标              | 数值    |
|---------------------|----------|
| 变异点数量          | 42       |
| 已杀死变异          | 28 (67%) |
| 存活变异            | 10 (24%) |
| 无覆盖变异          | 2 (5%)   |
| 等效变异（已跳过）  | 2 (5%)   |
  2. 存活变异（测试缺口）——对于每个存活变异，报告：
    • 位置：文件、方法、行号
    • 变异类别：边界 / 布尔 / 返回值 / 异常 / 算术 / Null检查
    • 原始代码：当前代码
    • 假设变异：会发生的变更
    • 存活原因：哪些测试覆盖此代码，以及为什么它们的断言遗漏了变异
    • 建议修复：能杀死此变异的具体测试断言或新测试用例
    按优先级分组：高风险存活变异优先（业务逻辑、计算、安全检查），低风险最后（日志、格式化）。
  3. 无覆盖区域——完全没有测试触及的代码路径。这些比存活变异更糟。
  4. 已杀死变异（优势）——简要说明测试有效的区域。突出测试良好的方法和强断言模式。不要枚举每个已杀死变异——总结即可。
  5. 建议——优先级列表：
    • 优先解决哪些存活变异（按风险）
    • 要添加或强化的特定测试方法
    • 团队可采用的防止未来缺口的模式（例如，始终测试边界值、始终断言异常类型）

Validation

验证

  • Every mutation point was classified (Killed / Survived / No coverage / Equivalent)
  • Every survived mutation includes the original code, the hypothetical change, and why tests miss it
  • Every survived mutation includes a concrete recommended fix (a test assertion or test case)
  • Equivalent mutations are correctly identified and excluded from the score
  • Trivial code (simple getters, auto-properties) is excluded from analysis
  • Findings are prioritized by risk, not just listed in source order
  • Report includes strengths (killed mutations) alongside gaps
  • Mutation categories are correctly labeled
  • 每个变异点都已分类（已杀死 / 存活 / 无覆盖 / 等效）
  • 每个存活变异都包含原始代码、假设变更以及测试遗漏的原因
  • 每个存活变异都包含具体的建议修复（测试断言或测试用例）
  • 等效变异已正确识别并从分数中排除
  • 琐碎代码（简单获取器、自动属性）已排除在分析之外
  • 结果按风险优先级排序，而非仅按源代码顺序列出
  • 报告同时包含优势（已杀死变异）和缺口
  • 变异类别已正确标记

Common Pitfalls

常见陷阱

PitfallSolution
Analyzing trivial codeSkip auto-properties, simple getters, and boilerplate — focus on logic
Reporting equivalent mutations as gapsIf the mutation doesn't change behavior, it's not a gap — mark Equivalent
Ignoring call chainsA private helper called from a tested public method is reachable — trace the chain
Over-counting mutations in generated codeSkip auto-generated code, designer files, and migration files
Recommending a new test for every survived mutationMultiple survived mutations in the same method often share a single missing test — recommend one test that kills several
Ignoring production contextA survived mutation in 
ToString()
formatting is less important than one in 
CalculateTotal()
— prioritize by business risk
Claiming 100% kill rate is requiredSome mutations in low-risk code are acceptable to leave — acknowledge this in the report
Not considering integration with other skillsIf gaps are found, mention that 
writing-mstest-tests
can help write the missing tests, and 
test-anti-patterns
can audit existing test quality
陷阱解决方案
分析琐碎代码跳过自动属性、简单获取器和样板代码——聚焦于逻辑
将等效变异报告为缺口如果变异不改变行为，则不是缺口——标记为等效
忽略调用链从已测试公共方法调用的私有辅助方法是可访问的——追踪调用链
过度统计生成代码中的变异跳过自动生成的代码、设计器文件和迁移文件
为每个存活变异推荐新测试同一方法中的多个存活变异通常共享一个缺失的测试——推荐一个能杀死多个变异的测试
忽略生产上下文
ToString()
格式化中的存活变异不如
CalculateTotal()
中的重要——按业务风险优先级排序
声称需要100%杀死率低风险代码中的某些变异可以接受——在报告中承认这一点
未考虑与其他技能的集成如果发现缺口，提及
writing-mstest-tests
可帮助编写缺失的测试，
test-anti-patterns
可审核现有测试质量