simulate-reviewers

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Simulate Reviewers

模拟审稿人

Run a paper through a simulated, venue-calibrated review panel before submission. A NeurIPS main-track reviewer and a SIGSPATIAL demo-track judge reject for different reasons at different thresholds — this skill reproduces that difference: persona-driven weakness hunting, rubric scoring on the venue's own scale, and a deterministic decision-risk readout that tells the authors what to fix while there is still time.

在提交前，让论文通过一个经过会议/期刊校准的模拟评审小组。NeurIPS主赛道审稿人和SIGSPATIAL演示赛道评委的拒稿原因及阈值各不相同——本Skill复现了这种差异：基于角色设定的弱点排查、采用目标会议/期刊自有标准的评分表打分，以及确定性的决策风险输出，让作者在还有时间的情况下明确需要修改的内容。

When to use

使用场景

"What would reviewers say about this paper?" / "simulate a review"
"Review this like a harsh NeurIPS reviewer" / "what will Reviewer 2 hate?"
"Is this good enough for KDD, or should I aim for the short track?"
"Find the weaknesses before the reviewers do" / "red-team my submission"
After
```
preflight-check
```
passes (format is clean) but before submitting — this skill judges content, preflight judges compliance.

「审稿人会怎么评价这篇论文？」/「模拟评审」
「像严苛的NeurIPS审稿人一样评审这篇论文」/「审稿人2会反感哪些内容？」
「这篇论文够格投KDD吗？还是我应该瞄准短论文赛道？」
「在审稿人发现之前找出论文弱点」/「red-team我的投稿」
在
```
preflight-check
```
（格式合规检查）通过后、提交前使用——本Skill评判论文内容，preflight-check则评判格式合规性。

Inputs

输入项

The paper: a
```
.tex
```
source tree, a PDF, or a draft in any readable form. Process it transiently — never copy paper text into the repo.
A venue profile:
```
venues/conferences/<venue>-<year>.yml
```
(schema in
```
venues/schema.yml
```
). No profile? Create one with
```
parse-cfp
```
first, or run against the nearest family default and say so.
The target track (page limits and reviewer expectations differ — ask).

论文文件：
```
.tex
```
源码目录、PDF文件，或任何可读格式的草稿。仅临时处理——绝不将论文文本复制到仓库中。
会议/期刊配置文件：
```
venues/conferences/<venue>-<year>.yml
```
（配置文件格式定义在
```
venues/schema.yml
```
中）。如果没有对应配置文件？先使用
```
parse-cfp
```
生成，或基于最相近的同体系默认配置运行，并告知用户这一点。
目标赛道（不同赛道的页数限制和审稿人预期不同——请确认）。

Process

流程

Build the calibrated review packet. Run:
```
python3 scripts/review_form.py venues/conferences/<venue>-<year>.yml \
    --track "<track>"
```
This is deterministic and offline. It merges the family profile and emits the panel (personas + harshness), the venue score scale with its borderline threshold, the rubric, the per-reviewer form skeleton, and a
```
scores.json
```
template. Add
```
--json
```
for machine-readable output. Exit codes: 0 ok, 2 missing/unparsable profile or unknown track.
Re-verify against the live CFP — mandatory. Profiles and the script's scale anchors are historical norms, never ground truth. Fetch the profile's
```
cfp_url
```
(and reviewer-guidelines page if linked) and confirm: review scale and form, blind level, rebuttal format, track expectations. If anything differs, update the profile YAML, note the discrepancy in the report, and prefer the live facts. Label every venue fact you state with a confidence tag and a clickable source:
```
verified-live
```
/
```
corroborated
```
/
```
inferred-from-family
```
/
```
needs-verification
```
. A scale number quoted to the user with no source is a bug, not a convenience.
Read the whole paper and build a claim inventory. List every claim of novelty ("first", "state-of-the-art", "outperforms"), every empirical claim, and where its supporting evidence lives. This inventory is what the personas attack. Method in references/weakness-hunting.md.
Write each review independently, in persona. One pass per reviewer from the packet, in order, without referring to the other reviews while writing (real reviews are independent; convergent complaints found independently are the strongest signal). Persona behavior, harshness calibration, and track modifiers are specified in references/reviewer-personas.md. Grounding rules — non-negotiable:
- Each review opens with genuine Strengths, then Weaknesses — like a real review form. State 2–4 specific strengths (what the paper does well: novelty, a strong experiment, clarity, a useful artifact), each grounded in a section/figure the same way weaknesses are. A review that is all cons is not a real review and misleads the author about what to protect while fixing. Do not invent strengths to pad — if the paper is weak, say so, but find what genuinely works.
- Every weakness cites a section/figure/line or quotes ≤1 sentence.
- Never invent prior work. If a persona suspects missing related work, find real candidates with
```
find-papers
```
  and verify them with
```
verify-citations
```
  — or phrase the concern conditionally ("if prior work on X exists, R4 will find it") with no fake reference.
- Misreadings are allowed only for the skimmer persona, and must be misreadings the actual text permits.
Score with the rubric, then the venue scale. Score the four core dimensions (novelty, soundness, reproducibility, clarity) 1–5 per reviewer using the anchors in references/rubrics.md, then map to the venue's overall scale + confidence from the packet. Harshness calibration: at harshness 5, an unaddressed soundness weakness caps the overall at borderline-reject; at harshness 2 (demo track), it becomes a question, not a cap.
Aggregate deterministically. Fill the
```
scores.json
```
template from step 1 and run:
```
python3 scripts/aggregate_scores.py scores.json
```
It computes the confidence-weighted mean, disagreement/champion/detractor flags, drag dimensions, and the decision-risk band (likely-reject / borderline-reject / borderline-accept / likely-accept) — including the "borderline without a champion resolves downward" rule.
```
--example
```
prints a valid input;
```
--json
```
for machine output; exit 2 on invalid input.
Write the meta-review and the fix list. As the AC/1AC/AE persona: synthesize the reviews, name the biggest shared concern, state whether a champion exists. Then convert every weakness into a prioritized fix list, each item tagged:
- ```
fix-now
```
  — addressable before submission (add ablation, soften claim, add reproducibility statement);
- ```
rebuttal-defensible
```
  — survivable in this venue's rebuttal format (check
```
review.rebuttal_format
```
  / limits in the profile);
- ```
structural
```
  — cannot be fixed this cycle; consider a different venue/track (hand off to
```
select-venue
```
  ). Borderline-reject predictors to check explicitly are listed in references/weakness-hunting.md.

构建校准后的评审数据包。运行以下命令：
```
python3 scripts/review_form.py venues/conferences/<venue>-<year>.yml \
    --track "<track>"
```
该操作是确定性的且离线运行。它会合并同体系配置文件，生成评审小组（角色设定+严苛程度）、带有临界阈值的会议/期刊评分标准、评分表、每位审稿人的评审表框架，以及
```
scores.json
```
模板。添加
```
--json
```
参数可输出机器可读格式。退出码说明：0表示运行正常，2表示配置文件缺失/无法解析或赛道未知。
根据最新CFP重新核实——强制要求。配置文件和脚本的评分标准锚点是历史惯例，绝非绝对事实。获取配置文件中的
```
cfp_url
```
（以及链接的审稿人指南页面）并确认：评审标准与表单格式、盲审级别、rebuttal（回复）格式、赛道预期。如果存在差异，更新配置文件YAML，在报告中注明差异，并以最新事实为准。对所有提及的会议/期刊相关事实标注置信度标签和可点击来源：
```
verified-live
```
（实时验证）/
```
corroborated
```
（多方佐证）/
```
inferred-from-family
```
（同体系推断）/
```
needs-verification
```
（待验证）。向用户引用评分标准数值但未标注来源属于错误，而非便捷操作。
通读论文并构建声明清单。列出所有创新性声明（如“首次”“state-of-the-art（最先进）”“性能优于”）、所有实证声明，以及支持这些声明的证据所在位置。该清单是角色设定的审稿人重点排查的对象。具体方法见references/weakness-hunting.md。
以角色身份独立撰写每份评审意见。按照数据包中的审稿人顺序，逐个撰写评审意见，撰写过程中不得参考其他评审意见（真实评审是独立的；不同审稿人独立发现的共性问题是最强烈的信号）。角色行为、严苛程度校准及赛道调整规则详见references/reviewer-personas.md。锚定规则——不可协商：
- 每份评审意见先列出真实的优点（Strengths），再列出缺点（Weaknesses）——与真实评审表单一致。列出2-4个具体优点（如论文在创新性、实验严谨性、表述清晰度、实用工具产出等方面的优势），每个优点都像缺点一样标注对应的章节/图表位置。全是缺点的评审意见不符合真实评审逻辑，会误导作者在修改时忽略需要保留的优势。不得编造优点凑数——如果论文质量不佳，如实说明，但要找出真正的可取之处。
- 每个缺点都必须引用对应的章节/图表/行数，或引用不超过1句话的原文。
- 绝不编造已有研究。如果角色设定的审稿人怀疑论文遗漏相关研究，使用
```
find-papers
```
  查找真实候选文献，并通过
```
verify-citations
```
  核实——或采用条件性表述（如“如果存在关于X的已有研究，R4会发现它”），且不得伪造引用。
- 仅允许“快速浏览型”审稿人角色出现误读，且误读必须是论文文本可能导致的合理误读。
先按评分表打分，再映射到会议/期刊标准。每位审稿人使用references/rubrics.md中的锚点，对四个核心维度（创新性、严谨性、可复现性、清晰度）打1-5分，然后映射到数据包中的会议/期刊整体评分标准及置信度。严苛程度校准：当严苛程度为5时，未解决的严谨性问题会将整体评分上限设为borderline-reject（临界拒稿）；当严苛程度为2（演示赛道）时，该问题仅作为疑问提出，不限制评分上限。
确定性汇总评分。填写步骤1生成的
```
scores.json
```
模板，然后运行以下命令：
```
python3 scripts/aggregate_scores.py scores.json
```
该脚本会计算置信度加权平均分、分歧/支持者/反对者标记、拉低评分的维度，以及决策风险区间（likely-reject（可能拒稿）/ borderline-reject（临界拒稿）/ borderline-accept（临界录用）/ likely-accept（可能录用））——包括“临界状态无支持者则倾向拒稿”规则。
```
--example
```
参数可打印有效输入示例；
```
--json
```
参数可输出机器可读格式；输入无效时退出码为2。
撰写元评审意见和修改清单。以AC/1AC/AE（领域主席/资深领域主席/副编辑）角色身份：汇总所有评审意见，指出最受关注的共性问题，说明是否存在支持者。然后将每个缺点转化为优先级排序的修改清单，每项标注：
- ```
fix-now
```
  （立即修改）——提交前可解决的问题（如添加消融实验、弱化声明、补充可复现性说明）；
- ```
rebuttal-defensible
```
  （可在回复中辩护）——在目标会议/期刊的回复格式下可应对的问题（查看配置文件中的
```
review.rebuttal_format
```
  及限制）；
- ```
structural
```
  （结构性问题）——本轮无法解决的问题；考虑更换会议/期刊或赛道（转交
```
select-venue
```
  处理）。需要重点检查的临界拒稿预测因素列在references/weakness-hunting.md中。

Output

输出内容

A simulated review packet, presented in chat (written to a file only if the user asks):

N independent reviews in the venue's form format, each opening with a Strengths section, then Weaknesses, then questions and subscores.
The meta-review, naming the biggest strength to preserve and the biggest shared risk to fix, and stating whether a champion exists.
The score table plus the
```
aggregate_scores.py
```
decision-risk readout.
The prioritized fix list, every item tagged
```
fix-now
```
/
```
rebuttal-defensible
```
/
```
structural
```
.

Every output carries the disclaimer: this is a simulation to improve the paper, not a prediction of the real outcome.

模拟评审数据包，以聊天形式呈现（仅在用户要求时写入文件）：

N份独立评审意见：采用目标会议/期刊的表单格式，每份先列出优点（Strengths）部分，再列出缺点（Weaknesses），然后是问题和分项评分。
元评审意见：指出需要保留的最大优势和需要解决的最大共性风险，并说明是否存在支持者。
评分表：加上
```
aggregate_scores.py
```
输出的决策风险结果。
优先级排序的修改清单：每项标注
```
fix-now
```
/
```
rebuttal-defensible
```
/
```
structural
```
。

所有输出均附带免责声明：本内容为提升论文质量的模拟评审，绝非对实际评审结果的预测。

Worked mini-example

小型示例

A 9-page submission to NeurIPS main track. After

python3 scripts/review_form.py venues/conferences/neurips-2026.yml --track Main

and a live-CFP check (scale and rebuttal format confirmed, tagged

verified-live

), the four personas are written independently. Convergence emerges: R2 (the empirical skeptic) and R4 (the adjacent-field expert) both, without seeing each other, land on the same gap.

R2, Weaknesses. "Table 2 reports a single run (§5.1). With ±std over 5 seeds, does the +1.3% gap over the baseline survive? No tuning-budget parity is stated for the baseline." — soundness 2, confidence 5.

R4, Weaknesses. "The §1 claim 'first to combine X with Y' needs the 2023 work on X-under-Y. If that line exists, the novelty claim narrows to an engineering delta." (Conditional — routed to
find-papers
; no fake citation stated.) — novelty 2, confidence 4.

Filling and aggregating

scores.json

(R1 5/4, R2 4/5, R3 6/2, R4 5/4) yields:

conf-weighted mean: 4.8   delta vs threshold: -0.133 (normalized)
DECISION RISK:      BORDERLINE-REJECT
drag dimensions:    soundness 2.5, reproducibility 2.75
flags:              strong detractor present (R2) — objection must be rebuttal-proof

The fix list leads with the convergent finding: fix-now
— add 5-seed ±std and tuning-budget parity to Table 2 (cheapest path off the soundness floor); fix-now
— verify the X-under-Y prior work and soften the "first" claim accordingly. The meta-review notes the only above-threshold score came from the low-confidence skimmer (no real champion), so at borderline this resolves downward unless the soundness objection is closed before submission.

一篇9页的NeurIPS主赛道投稿。运行

python3 scripts/review_form.py venues/conferences/neurips-2026.yml --track Main

并完成实时CFP检查（评分标准和回复格式已确认，标注为

verified-live

）后，四位角色设定的审稿人独立撰写评审意见。出现共性问题：R2（实证怀疑论者）和R4（相邻领域专家）在未参考彼此意见的情况下，发现了同一个漏洞。

R2，缺点：“表2仅报告了单次运行结果（§5.1）。如果基于5个随机种子计算±标准差，相比基线的+1.3%性能优势是否仍然存在？未说明基线模型的调参预算是否一致。”——严谨性得分2，置信度5。

R4，缺点：“§1中的声明‘首次将X与Y结合’需要参考2023年关于X-under-Y的研究。如果该研究存在，创新性声明将缩小为工程实现层面的改进。”（条件性表述——转交
find-papers
处理；未伪造引用。）——创新性得分2，置信度4。

填写并汇总

scores.json

（R1 5/4，R2 4/5，R3 6/2，R4 5/4）后得到：

conf-weighted mean: 4.8   delta vs threshold: -0.133 (normalized)
DECISION RISK:      BORDERLINE-REJECT
drag dimensions:    soundness 2.5, reproducibility 2.75
flags:              strong detractor present (R2) — objection must be rebuttal-proof

修改清单首项为共性问题：fix-now
——在表2中添加基于5个随机种子的±标准差和调参预算一致说明（脱离严谨性最低分的最便捷路径）；fix-now
——核实X-under-Y的已有研究，并相应弱化“首次”声明。元评审意见指出，唯一超过阈值的评分来自置信度较低的快速浏览型审稿人（无真正支持者），因此处于临界状态时，除非在提交前解决严谨性问题，否则将倾向于拒稿。

Adapt to your discipline

适配你的研究领域

Panels and scales are keyed on the venue

family:

field. For other fields, fork and add a calibration entry (personas + scale) for your community in

scripts/review_form.py

and a venue YAML — e.g. an APA-journal panel with action-editor + 2 reviewers and accept/minor/major/reject.

评审小组和评分标准与会议/期刊的

family:

字段关联。对于其他研究领域，可分叉本项目，在

scripts/review_form.py

中添加针对你的学术社区的校准条目（角色设定+评分标准），并添加对应的会议/期刊YAML配置文件——例如，针对APA期刊的评审小组，包含执行编辑+2位审稿人，评分结果分为录用/小修/大修/拒稿。

Guardrails

约束规则

Never present the simulation as a prediction ("your paper will get a 5 at NeurIPS"). Say: "the simulated panel scored it X; real panels vary widely."
Never fabricate citations, reviewer identities, or quotes — personas are archetypes, never named real researchers; missing-related-work claims go through
```
find-papers
```
+
```
verify-citations
```
or stay conditional.
Re-verify review-process facts against the live
```
cfp_url
```
(step 2 is not optional); flag any profile staleness in the report.
Process the paper transiently; quote at most one sentence per finding; never store paper text in this repo.
Never submit to, or post on, any review system on the user's behalf.
Format/compliance problems found along the way are out of scope — route them to
```
preflight-check
```
.

绝不能将模拟评审表述为预测（如“你的论文在NeurIPS会得5分”）。应表述为：“模拟评审小组给出的评分为X；真实评审小组的评分差异很大。”
绝不伪造引用、审稿人身份或原文引用——角色设定是典型archetype，绝非真实研究者；关于遗漏相关研究的质疑需通过
```
find-papers
```
+
```
verify-citations
```
核实，或采用条件性表述。
必须根据最新的
```
cfp_url
```
重新核实评审流程相关事实（步骤2为强制要求）；在报告中标记任何配置文件过时的情况。
仅临时处理论文；每个发现最多引用1句话；绝不将论文文本存储在本仓库中。
绝不代用户向任何评审系统提交或发布内容。
过程中发现的格式/合规问题不属于本Skill的处理范围——转交
```
preflight-check
```
处理。

Memory

记忆机制

This skill uses the shared

.paper-memory/

convention in the user's paper directory, following

paper-memory-convention.md

At start: read
```
.paper-memory/profile.yml
```
(vertical, risk appetite, venue tier) to calibrate the panel and how hard the personas press novelty claims, and read
```
lessons.md
```
to recall which weaknesses were already raised and which
```
recurring
```
ones this author repeats (e.g. "weak ablations" or "overclaimed contributions") so the meta-review leads with them.
At end: append the durable weaknesses in the shared format
```
- [YYYY-MM-DD] (simulate-reviewers | <scope>) weakness -> recommendation
```
(via
```
reflect-and-improve
```
's
```
reflect_log.py append
```
, which dedupes and dates). A structural habit seen across drafts is
```
recurring
```
; a draft-specific gap is
```
this-paper
```
. Do not log per-reviewer score noise.
Create
```
.paper-memory/
```
on demand if absent and offer to add it to the project
```
.gitignore
```
. It is local-only; never upload it or copy it into this repo.

本Skill遵循

paper-memory-convention.md

，使用用户论文目录中的共享

.paper-memory/

约定。

启动时：读取
```
.paper-memory/profile.yml
```
（研究方向、风险偏好、会议/期刊层级）以校准评审小组及角色设定的审稿人对创新性声明的严苛程度，读取
```
lessons.md
```
以回顾已提出的弱点以及作者反复出现的
```
recurring
```
（重复性）问题（如“消融实验不足”或“贡献夸大”），以便元评审意见重点提及这些问题。
结束时：采用共享格式
```
- [YYYY-MM-DD] (simulate-reviewers | <scope>) weakness -> recommendation
```
添加持久化的弱点（通过
```
reflect-and-improve
```
的
```
reflect_log.py append
```
命令，该命令会去重并添加日期）。跨版本草稿中出现的结构性问题标记为
```
recurring
```
；仅当前草稿存在的漏洞标记为
```
this-paper
```
。不得记录每位审稿人的评分噪声。
如果
```
.paper-memory/
```
不存在，则按需创建，并提议将其添加到项目的
```
.gitignore
```
中。该目录仅存储在本地；绝不上传或复制到本仓库中。