benchmark-fp-fn-audit

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

React Doctor benchmark FP/FN audit

React Doctor基准测试假阳性/假阴性审计

Perform an evidence-backed audit of React Doctor diagnostics across a benchmark corpus. Read the complete rule documentation, quantify the distribution of failures, inspect every relevant trial artifact, and independently perform a second pass for additional false positives and false negatives.

针对基准语料库对React Doctor的诊断结果开展基于证据的审计。阅读完整的规则文档，量化故障分布情况，检查每一个相关的试验产物，并独立进行第二轮检查以发现更多假阳性和假阴性问题。

Corpus and required resources

语料库及所需资源

For this benchmark, use:

text

/home/aidenybai/Developer/react-bench-internal/jobs/ReactBench-v1.1-beta-rd-0.9.3-consolidated-final-20260801

Read the complete React Doctor rule documentation from:

text

https://www.react.doctor/docs/rules

Read these repository review materials when they exist:

text

/home/aidenybai/Developer/react-bench-internal/docs/aiden-review

Do not treat prior reviews or reports as ground truth. Use them as leads and reverify every claim against the current benchmark revision.

For every inspected trial, locate and read:

```
verifier/model.patch
```
```
verifier/rd-before.json
```
```
verifier/rd-after.json
```
```
verifier/rd.log
```
```
result.json
```
reward and test logs, discovered with
```
rg --files
```
the task README, prompt, or review material
the base and patched source files referenced by the patch

Inspect all trial directories, including primary and verified variants. Normalize duplicate task names and repeated patch hashes before computing task-level statistics.

本次基准测试使用：

text

/home/aidenybai/Developer/react-bench-internal/jobs/ReactBench-v1.1-beta-rd-0.9.3-consolidated-final-20260801

从以下地址阅读完整的React Doctor规则文档：

text

https://www.react.doctor/docs/rules

若存在以下仓库评审材料，请阅读：

text

/home/aidenybai/Developer/react-bench-internal/docs/aiden-review

请勿将之前的评审或报告视为绝对真理。仅将其作为线索，并针对当前基准测试版本重新验证每一项结论。

对于每一个被检查的试验，需定位并阅读：

```
verifier/model.patch
```
```
verifier/rd-before.json
```
```
verifier/rd-after.json
```
```
verifier/rd.log
```
```
result.json
```
通过
```
rg --files
```
查找的奖励和测试日志
任务README、提示信息或评审材料
补丁中引用的基础和已修补源代码文件

检查所有试验目录，包括主要和已验证的变体。在计算任务级统计数据之前，先标准化重复的任务名称和重复的补丁哈希值。

Evidence rules

证据规则

Classify findings conservatively:

Confirmed FP: a diagnostic is caused by changed code, the behavior is intentional or required, and the rule contract does not apply or its recommended fix would regress behavior.
Confirmed FN: a real defect exists in the base or final code, no applicable diagnostic is emitted, and an existing documented rule should cover it.
Taxonomy gap: a real defect is present but no existing rule contract reasonably claims to cover it.
Harness artifact: the result is caused by malformed output, stale baseline data, verifier behavior, generated files, or an untouched-file diagnostic.
Candidate: plausible but missing enough evidence for confirmation.

Do not call a target-task miss a false negative. Do not call a diagnostic a false positive merely because the model failed tests.

react_doctor=1

is a gate result, not proof that the code is correct.

Every confirmed finding must cite the exact trial, file, line or code span, rule documentation, diagnostic delta, task behavior, and test result. Separate strict rule misses from out-of-taxonomy gaps.

保守地对发现结果进行分类：

确认假阳性：诊断由修改后的代码导致，行为是有意或必要的，规则契约不适用，或其推荐的修复会导致行为退化。
确认假阴性：基础或最终代码中存在真实缺陷，但未发出适用的诊断，且现有已记录的规则应覆盖该缺陷。
分类缺口：存在真实缺陷，但现有规则契约均无法合理覆盖该情况。
测试框架产物：结果由格式错误的输出、过时的基线数据、验证器行为、生成文件或未修改文件的诊断导致。
候选案例：看似合理，但缺乏足够证据以确认。

请勿将目标任务遗漏称为假阴性。请勿仅因模型测试失败就将诊断称为假阳性。

react_doctor=1

是一个准入结果，并非代码正确的证明。

每一项已确认的发现必须引用确切的试验、文件、行或代码段、规则文档、诊断差异、任务行为和测试结果。区分严格的规则遗漏与分类外的缺口。

Workflow

工作流程

1. Inventory the corpus

1. 清点语料库

Use

rg --files

to enumerate trial artifacts. Record the benchmark revision, trial count, missing artifacts, duplicate variants, malformed reports, and available task reviews. Do not silently skip incomplete trials.

使用

rg --files

枚举试验产物。记录基准测试版本、试验数量、缺失的产物、重复变体、格式错误的报告以及可用的任务评审。请勿静默跳过不完整的试验。

2. Build diagnostic distributions

2. 构建诊断结果分布

Recompute distributions from raw

rd.log

rd-before.json

, and

rd-after.json

data. Rank by trial coverage before raw occurrence count. Report:

total trials and artifact completeness;
reward, test, and React Doctor gate tuple counts;
direct
```
NEW
```
or introduced-diagnostic failures;
health-mode
```
NEW
```
deltas;
health-mode
```
STILL
```
target misses;
baseline-to-head rule-count deltas;
rule, category, severity, task, repository, and file concentration;
changed-file versus untouched-file diagnostics;
malformed or truncated reports;
primary versus verified-task duplication;
successful versus unsuccessful patch comparisons.

Always recompute before relying on a prior summary. Large files can inflate occurrence counts, so show both occurrence count and affected-trial count.

从原始的

rd.log

、

rd-before.json

和

rd-after.json

数据重新计算分布情况。在原始出现次数之前，按试验覆盖率排序。报告内容包括：

总试验数和产物完整性；
奖励、测试和React Doctor准入元组计数；
直接
```
NEW
```
或引入的诊断故障；
健康模式
```
NEW
```
差异；
健康模式
```
STILL
```
目标遗漏；
基线到头部的规则计数差异；
规则、类别、严重性、任务、仓库和文件的集中度；
修改文件与未修改文件的诊断对比；
格式错误或截断的报告；
主要任务与已验证任务的重复情况；
成功与失败的补丁对比。

在依赖之前的摘要之前，务必重新计算。大文件可能会增加出现次数，因此需同时显示出现次数和受影响的试验数。

3. Inspect high-impact clusters

3. 检查高影响集群

Prioritize high-coverage rules and exact introduced-diagnostic failures, especially:

text

exhaustive-deps
no-giant-component
js-set-map-lookups
no-array-index-as-key
js-combine-iterations
effect-needs-cleanup
no-adjust-state-on-prop-change
button-has-type
prefer-module-scope-static-value
no-static-element-interactions
prefer-module-scope-pure-function
no-pass-data-to-parent
no-pass-live-state-to-parent
no-derived-state
no-ref-current-in-render
click-events-have-key-events
rerender-lazy-ref-init
only-export-components
rerender-lazy-state-init

For each high-impact cluster, inspect representative positives, representative negatives, the largest task concentration, and sibling trials with different outcomes.

优先处理高覆盖率规则和确切的引入诊断故障，尤其是：

text

exhaustive-deps
no-giant-component
js-set-map-lookups
no-array-index-as-key
js-combine-iterations
effect-needs-cleanup
no-adjust-state-on-prop-change
button-has-type
prefer-module-scope-static-value
no-static-element-interactions
prefer-module-scope-pure-function
no-pass-data-to-parent
no-pass-live-state-to-parent
no-derived-state
no-ref-current-in-render
click-events-have-key-events
rerender-lazy-ref-init
only-export-components
rerender-lazy-state-init

对于每个高影响集群，检查代表性阳性案例、代表性阴性案例、任务集中度最高的案例以及结果不同的同类试验。

4. Perform the independent second pass

4. 执行独立第二轮检查

Search every trial for additional candidates, not just the known clusters. Apply these heuristics:

Separate expensive lazy initialization from cheap empty
```
Map
```
,
```
Set
```
, and equivalent containers.
Distinguish render-derived state from async status, media errors, retries, resource lifetimes, subscriptions, and transient UI state.
For controlled components, distinguish required parent synchronization from effect loops or unnecessary data forwarding.
Follow dangerous HTML values through local aliases,
```
useMemo
```
, wrappers, and helper functions; preserve provenance for trusted serializers such as KaTeX.
Distinguish React-docs-blessed previous-prop comparisons from pure derived state and render-phase side effects.
Inspect timers, promises, animations, abort signals, subscriptions, and stale closures for races that generic dependency or cleanup warnings do not directly explain.
Check focus stealing, focus restoration, controlled selection, keyboard semantics, and ARIA transitions.
Check thresholds, aliasing, destructuring, nested callbacks, JSX wrappers, computed properties, and TypeScript syntax.
Verify that a diagnostic is on changed code and not a pre-existing issue, generated artifact, fixture, or untouched neighbor.
For every proposed FP, find a nearby true-positive counterexample. For every proposed FN, find a nearby case the detector catches.

Known leads to independently recheck, without presuming their classification:

text

fix-react-rdh-nteract-semiotic-a__5uDDmgx
write-react-glific-glific-fronte__22Gg4p2
fix-react-jumperexchange-jumper__2GBTh7Z
write-react-frankchen021-datasto__2N6xFwQ
fix-react-viclafouch-mui-tel-inp__hQXzJgB
fix-react-floating-ui-floating-u__27iKG4t
fix-react-formidablelabs-victory__2uwi2vz
fix-react-igordanchenko-yet-anot__2GZGZYT
fix-react-rdh-appflowy-io-appflo__vK9uEUy

搜索每一个试验以寻找更多候选案例，而不仅仅是已知集群。应用以下启发式规则：

区分昂贵的延迟初始化与廉价的空
```
Map
```
、
```
Set
```
及等效容器。
区分渲染派生状态与异步状态、媒体错误、重试、资源生命周期、订阅和临时UI状态。
对于受控组件，区分必要的父组件同步与副作用循环或不必要的数据转发。
追踪危险HTML值在本地别名、
```
useMemo
```
、包装器和辅助函数中的流向；保留KaTeX等可信序列化工具的来源信息。
区分React文档认可的先前属性比较与纯派生状态和渲染阶段副作用。
检查计时器、Promise、动画、中止信号、订阅和陈旧闭包中，通用依赖项或清理警告无法直接解释的竞争情况。
检查焦点窃取、焦点恢复、受控选择、键盘语义和ARIA过渡。
检查阈值、别名、解构、嵌套回调、JSX包装器、计算属性和TypeScript语法。
验证诊断是否针对修改后的代码，而非预先存在的问题、生成的产物、测试夹具或未修改的相邻文件。
对于每一个提出的假阳性案例，找到附近的真阳性反例。对于每一个提出的假阴性案例，找到检测器捕获的附近案例。

需独立重新检查的已知线索，无需预设其分类：

text

fix-react-rdh-nteract-semiotic-a__5uDDmgx
write-react-glific-glific-fronte__22Gg4p2
fix-react-jumperexchange-jumper__2GBTh7Z
write-react-frankchen021-datasto__2N6xFwQ
fix-react-viclafouch-mui-tel-inp__hQXzJgB
fix-react-floating-ui-floating-u__27iKG4t
fix-react-formidablelabs-victory__2uwi2vz
fix-react-igordanchenko-yet-anot__2GZGZYT
fix-react-rdh-appflowy-io-appflo__vK9uEUy

5. Produce audit artifacts

5. 生成审计产物

Write audit-only outputs under the benchmark directory:

text

audit/rd-0.9.3-second-pass.md
audit/rd-0.9.3-second-pass.jsonl
audit/rd-0.9.3-rule-distribution.tsv

Do not edit React Doctor source, benchmark source, task tests, or verifier data. The JSONL must contain one record per finding with:

json

{
  "id": "...",
  "classification": "confirmed_fp | confirmed_fn | taxonomy_gap | harness_artifact | candidate",
  "confidence": "high | medium | low",
  "rule": "...",
  "task": "...",
  "trial": "...",
  "file": "...",
  "lines": "...",
  "evidence": "...",
  "baselineDiagnostics": "...",
  "headDiagnostics": "...",
  "testStatus": "...",
  "rdStatus": "...",
  "rationale": "...",
  "recommendedAction": "..."
}

The Markdown report must contain: corpus/methodology, distributions, confirmed FPs, confirmed FNs, taxonomy gaps, harness artifacts, manual-adjudication candidates, prioritized rule fixes, and limitations.

Prioritize fixes using:

text

affected trial coverage × confidence × reproducibility

Never make a code change as part of this skill unless the user separately requests implementation of a confirmed rule fix.

在基准测试目录下编写仅用于审计的输出文件：

text

audit/rd-0.9.3-second-pass.md
audit/rd-0.9.3-second-pass.jsonl
audit/rd-0.9.3-rule-distribution.tsv

请勿编辑React Doctor源代码、基准测试源代码、任务测试或验证器数据。JSONL文件需为每个发现包含一条记录，格式如下：

json

{
  "id": "...",
  "classification": "confirmed_fp | confirmed_fn | taxonomy_gap | harness_artifact | candidate",
  "confidence": "high | medium | low",
  "rule": "...",
  "task": "...",
  "trial": "...",
  "file": "...",
  "lines": "...",
  "evidence": "...",
  "baselineDiagnostics": "...",
  "headDiagnostics": "...",
  "testStatus": "...",
  "rdStatus": "...",
  "rationale": "...",
  "recommendedAction": "..."
}

Markdown报告需包含：语料库/方法论、分布情况、已确认的假阳性、已确认的假阴性、分类缺口、测试框架产物、人工裁决候选案例、优先级规则修复以及局限性。

使用以下公式确定修复优先级：

text

受影响试验覆盖率 × 置信度 × 可复现性

除非用户单独要求实施已确认的规则修复，否则请勿在此技能中进行任何代码更改。