Back to Details

acceptance-gate

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

acceptance-gate — what "done" means

acceptance-gate——“完成”的含义

A result is done when it has survived a check that could have failed, run by a route that could not have known the answer. It is never done merely because the computation that produced it finished.
Why the gate exists: computation produces confident wrong answers at every stage, and none of them announce themselves. A fit converges to the wrong basin and reports small residuals. An integer-relation search "recognizes" numerical noise as a famous constant, because search enough constants and something always matches a short prefix. Two evaluators built on the same series expansion agree to great depth on the same wrong value. The agent that produced the answer — human or model — is the least qualified judge of it, because everything that produced the answer also produces reasons to believe it. So "done" is defined externally: agreement with an independent route, at points that entered no fit, to more digits than the fit could enforce, deepening when precision is raised, measured by machinery that has demonstrably failed on a wrong answer. Each clause of that sentence is there because its absence has, at some point, let a wrong result through.
The bar used for the BootLoops loop-integral results (bootloops.ai): at least thirty genuine held-back digits, two-precision stable, against an oracle that never fed the fit. Adopt a bar of that order for anything new. Scale the digit count to the problem; never scale away the structure.
一个结果只有在经受住了一项本可能失败的检查之后，才算完成；该检查必须由一条不可能预先知道答案的路径执行。仅仅因为产生该结果的计算结束，绝不意味着结果已经完成。
设置这道关卡的原因：计算在每个阶段都会产出自信的错误答案，而这些错误都不会自我宣告。一个拟合收敛到错误的吸引域，并报告很小的残差。一次整数关系搜索会“识别出”数值噪声，把它当成某个著名常数，因为只要搜索足够多的常数，总有某个常数能匹配一段短前缀。两个基于同一级数展开构建的评估器，会在同一个错误值上高度一致地吻合到很深程度。产生答案的 Agent——无论 human 还是 model——最不适合担任它的评判者，因为促成答案产生的一切因素，也会同时产生相信该答案的理由。因此，“完成”被定义为外部验证：与一条独立路径达成一致，验证点没有进入任何拟合，一致位数超过拟合所能强制维持的位数；提高精度时，一致程度应继续加深；并且所用机制已被证明能够对错误答案产生失败。句中的每一个条款都有其存在的原因，因为这些条款中的任何一项缺失，都曾在某个时候让错误结果蒙混过关。
用于 BootLoops loop-integral 结果（bootloops.ai）的门槛：至少保留三十个真实的留出数字，在两种精度下保持稳定，并与从未向拟合提供数据的 oracle 进行比较。新事物应采用同一量级的标准。根据问题调整数字位数；但绝不可弱化这一结构。

The procedure

操作流程

Run the steps in order. The order is itself part of the discipline: the check is designed before the answer exists, because a check designed after the answer exists is chosen — consciously or not — to pass.
0. Declare the gate before fitting. Before any fit, search, or model choice, write down: which evaluation points are reserved, what the independent route will be, what digit count will count as passing, and what the controls are. A gate written after the candidate exists inherits the candidate's blind spots.
1. Reserve never-fit points. Choose evaluation points, record them, and exclude them from every fit, every tuning decision, and every basis selection — not only the final fit but every exploratory run that shaped the method. Agreement at fitted points measures interpolation: a fit with enough parameters reproduces its own inputs exactly, and agreement there means nothing.
2. Audit the independent route. The check values must come from a route that shares no code, no series representation, and no fitted input with the derivation. Numerically integrating the original definition, when the candidate came from a fitted ansatz, is independent; a second wrapper around the same expansion is not. An oracle whose values entered the fit — even once, even only to pick a tolerance — is disqualified from certifying (see independence-bookkeeping). Where full disjointness is impossible, record exactly what is shared and treat the check as weakened by that much.
3. Count digits; do not describe them. Evaluate both sides at the reserved points and count the digits that genuinely agree. The count must exceed, by a wide margin, anything the fit's free parameters could have absorbed. The unit of evidence is a number — "34 digits at each of three reserved points" — never "excellent agreement".
4. Rerun at raised precision. Double the working precision and repeat. A true identity deepens: the count of agreeing digits grows with the precision. Agreement pinned at the same depth regardless of working precision is a truncated intermediate shared by both sides, a coincidence at the resolution of the search, or a bug. It is never a confirmation.
5. Run the controls, positive and negative. Positive: run the identical gate on a case where the answer is independently known, and confirm it passes. Negative: perturb the candidate — flip the sign of one term, alter the last fitted coefficient in its final digit — and confirm the gate fails, loudly, at the digit where it should. A gate that has never failed anything certifies nothing: every clause of it might be broken and you would see only passes.
6. Leave-one-out, where a fit determined the answer. For each anchor point that entered the fit: refit without it, evaluate the refit at the excluded point, and demand agreement at full depth. An identity survives every exclusion; an interpolation collapses at exactly the excluded point. This is the cheapest way to distinguish "found the formula" from "drew a curve through the data".
7. Package a standalone evaluator. The claim ships with a script that rebuilds the result from the exact data included with it and re-measures this gate at arbitrary requested precision — no hidden grids, no finite-precision constants baked in, no state that lived only in the session that produced the claim. If a stranger cannot rerun the gate, the gate ran once and its evidence is already decaying.
8. Report the true verdict. CLOSED means every part above passed and the counts are written down. Anything less is OPEN, reported with what was established and what remains. An honest OPEN is a result; a false CLOSED is damage, because later work builds on it and the cost of the unwinding grows with every week it stands.
按顺序执行以下步骤。顺序本身就是纪律的一部分：检查方案要在答案存在之前设计完成，因为在答案存在之后设计的检查，会有意或无意地被选择为能够通过。
0. 在拟合之前声明验收关卡。 在进行任何拟合、搜索或模型选择之前，写明：保留哪些评估点、独立路径是什么、多少位数字算作通过，以及控制项是什么。在候选答案出现之后才写下的关卡，会继承候选答案的盲点。
1. 保留从未参与拟合的点。 选择评估点并记录下来，将其排除在每一次拟合、每一次调参决策和每一次基函数选择之外——不仅排除在最终拟合之外，也排除在塑造了方法的每一次探索性运行之外。在拟合点处取得一致只能衡量插值能力：参数足够多的拟合可以精确复现自己的输入，在那里取得一致不能说明任何问题。
2. 审计独立路径。 检查值必须来自一条与推导过程不共享任何代码、任何级数表示和任何已拟合输入的路径。如果候选答案来自拟合 ansatz，那么对原始定义进行数值积分属于独立路径；围绕同一个展开再套一层包装器则不属于。即使只是曾经使用过、哪怕仅用于选择容差，其数值进入过拟合的 oracle，也不具备认证资格（见 independence-bookkeeping）。如果无法做到完全不相交，应准确记录共享了哪些部分，并据此相应削弱检查。
3. 统计位数；不要用形容词描述。 在保留点评估两侧，并统计真正一致的位数。该计数必须大幅超过拟合的自由参数可能吸收的误差范围。证据单位是一个数字——“在三个保留点处各自有 34 位一致”——而不是“吻合极佳”。
4. 在更高精度下重跑。 将工作精度加倍并重复。真实恒等式会加深：一致位数会随精度提高而增加。无论工作精度如何提高，一致深度都固定不变，说明双方共享了某个被截断的中间结果、这只是搜索分辨率下的巧合，或者存在 bug。它绝不能算作确认。
5. 运行正控制和负控制。 正控制：对一个答案已由独立方式确定的案例运行完全相同的关卡，并确认其通过。负控制：扰动候选答案——翻转其中一项的符号，或把最后一个拟合系数的末位数字改掉——并确认关卡会在本应失败的那一位上明显失败。一个从未让任何东西失败的关卡什么也证明不了：它的每个条款都可能已经损坏，而你只会看到通过。
6. 如果答案由拟合决定，执行 leave-one-out。 对进入拟合的每一个锚点：移除它后重新拟合，在被排除的点评估重拟合结果，并要求在完整深度上保持一致。恒等式经得起每一次排除；插值则恰好在被排除的点崩溃。这是区分“找到了公式”与“沿着数据画出一条曲线”最便宜的方法。
7. 打包一个独立评估器。 该声明应附带一个脚本，使用随附的精确数据重建结果，并能在任意请求的精度下重新测量这道关卡——不得有隐藏网格，不得烘焙有限精度常量，也不得依赖仅存在于生成该声明的会话中的状态。如果陌生人无法重跑这道关卡，那么关卡只运行过一次，而它的证据也已经开始失效。
8. 报告真实结论。 CLOSED 表示上述每一部分都已通过，并且计数已经写明。任何不足都属于 OPEN，并报告已经确立的内容和仍然遗留的内容。诚实的 OPEN 本身就是一个结果；虚假的 CLOSED 则会造成破坏，因为后续工作会建立在它之上，而撤销错误的成本会随着它存在的每一周不断增长。

Failure modes the gate exists to catch

这道关卡旨在捕捉的失败模式

Each of these is a class observed repeatedly in practice, from agents and from people. The vignette states the mechanism; learn the shape, not the example.
  • Satisficing. The first plausible answer is declared final: a handful of digits of numerical noise matches a known constant at the resolution of the search, and the search stops. The match was guaranteed by the size of the ring searched. The digit bar plus two-precision stability is the cure — a coincidence does not deepen.
  • Certifying at fitted points. The check runs at points the fit saw, agreement is perfect, and the perfection is reported as confirmation. It confirms only that the fit can reproduce its inputs. Reserved points, set aside before fitting, no exceptions.
  • Oracle contamination. Reference values get used casually during development — to pick a tolerance, choose a basis, decide when to stop — and are later reused to certify. The gate then verifies that the method remembers what it was shown. Track which values touched anything; certify only from the disjoint set.
  • False independence through a shared representation. Two separately written evaluators, both resting on the same series expansion or the same underlying library, agree to any depth you like — including on a wrong value, when the shared layer carries the bug. Their agreement measures the bug's consistency. Audit lineage; prefer a route different in kind, not merely in authorship.
  • Precision-pinned agreement. The digit count refuses to grow when working precision is raised: both sides share a truncated constant, a fixed grid, or a low-precision intermediate. This is the single most diagnostic symptom the gate produces. A pinned count is a failure, never "close enough".
  • The gate that cannot fail. The comparison harness itself is broken open: a tolerance wide enough to pass anything, a value compared against itself through an aliased variable, formatted output string-compared so both sides truncate identically, an assertion inside a branch that never executes. Every pass from such a gate is vacuous. The negative control is the only way to know a gate can fire: you have watched it fire.
  • Threshold drift. The tolerance is widened, once per awkward case, until the candidate passes. The finished gate is then a description of the candidate rather than a test of it, and a real error of the same size sails through. Thresholds are fixed at step 0 and never touched after the candidate exists; a candidate that needs the threshold moved has failed.
  • Adjectives in place of counts. "Essentially exact", "matches beautifully", "agrees to high precision". Every one of these has been used to describe agreement that fell apart under an actual digit count. Demand the number, the precision it was measured at, and whether the point was reserved.
  • Self-graded confidence. The producing agent's stated confidence offered as evidence. Confidence is an output of the same process being audited; it tracks fluency, not correctness. The gate is the evidence; there is no other kind.
  • Partials rounded up to done. "Verified" covering nine of ten cases with the tenth pending; "closed modulo one term"; a comparison run at lower precision than the claim states. The verdict vocabulary is the cure: CLOSED has a definition, and the definition is the full gate.
  • The unconditional success line. A driver prints its PASS banner outside the conditional that runs the comparison, or after a caught exception, so a run that compared nothing reports success. The verdict line must be printed by the comparison itself, from the measured counts — and the negative control catches this class too, because a broken driver passes the perturbed candidate.
以下每一种都是实践中反复观察到的类别，既出现在 Agent 中，也出现在人身上。示例说明的是机制；要学习的是其形态，而不是具体例子。
  • Satisficing（满足式求解）。 第一个看似合理的答案被宣布为最终答案：若干位数值噪声在搜索分辨率下匹配了一个已知常数，于是搜索停止。这种匹配是被搜索的常数环大小所必然保证的。位数门槛加上两种精度下的稳定性就是解决方案——巧合不会随精度加深。
  • 在已拟合点上进行认证。 检查在拟合见过的点上运行，一致性完美，并把这种完美报告为确认。它只证明拟合能够复现自己的输入。必须使用在拟合前留出的点，不设例外。
  • Oracle 污染。 开发期间随意使用参考值——用于选择容差、确定基、决定何时停止——之后又复用这些值进行认证。这样一来，关卡验证的只是方法是否记住了曾经展示给它的内容。跟踪哪些值曾影响任何环节；只能使用完全不相交的一组值进行认证。
  • 通过共享表示制造虚假独立性。 两个分别编写的评估器如果都依赖同一级数展开或同一个底层库，就可以在任意深度上一致——当共享层带有 bug 时，它们甚至会在错误值上保持一致。它们的一致性衡量的只是 bug 的一致性。审计来源谱系；优先选择本质上不同的路径，而不只是作者不同的路径。
  • 精度钉死的一致性。 提高工作精度后，位数计数仍拒绝增长：双方共享了一个被截断的常数、固定网格或低精度中间结果。这是该关卡所能识别的最具诊断意义的症状。计数被钉死就是失败，绝不是“已经足够接近”。
  • 无法失败的关卡。 比较测试框架本身被彻底弄坏：容差宽到足以让任何东西通过；一个值通过别名变量与自身比较；格式化输出通过字符串比较，导致双方以同样方式截断；或者断言位于永远不会执行的分支中。这种关卡给出的每一次通过都毫无意义。负控制是唯一能够知道关卡是否会触发的方法：你必须亲眼看着它触发。
  • 阈值漂移。 每遇到一个棘手案例就放宽一次容差，直到候选答案通过。完成后的关卡便成了对候选答案的描述，而不是对它的检验，同等规模的真实错误也会顺利通过。阈值在步骤 0 固定，候选答案出现后绝不再改动；需要移动阈值才能通过的候选答案，已经失败。
  • 用形容词代替计数。 “基本精确”“吻合得非常漂亮”“高度精确地一致”。这些说法都曾被用来描述在实际位数统计下会分崩离落的一致性。必须要求给出数字、测量时采用的精度，以及该点是否为保留点。
  • 自我评分的置信度。 把生成答案的 Agent 所声明的置信度当作证据。置信度是正在接受审计的同一过程的输出；它反映的是表达流畅度，而不是正确性。关卡才是证据；不存在其他证据。
  • 将未完成的部分四舍五入为已完成。 “已验证”只覆盖十个案例中的九个，而第十个仍待处理；“除一项外已 CLOSED”；或者比较运行所采用的精度低于声明中的精度。结论词汇就是解决方案：CLOSED 有明确含义，即必须通过完整的关卡。
  • 无条件的成功提示行。 驱动程序在执行比较的条件语句之外，或在捕获异常之后打印 PASS 横幅，因此一次什么都没比较的运行也会报告成功。结论行必须由比较本身根据测得的计数打印——负控制也能捕捉这一类问题，因为损坏的驱动程序会让扰动后的候选答案也通过。

Two micro-examples

两个微型示例

A fitted closed form. A candidate formula was fit from evaluations at a dozen points. The gate: three further points reserved before fitting; the original definition integrated numerically at those points by a method sharing nothing with the ansatz; both sides evaluated at two working precisions and the agreement counted, with the deepening confirmed; one sign in the candidate flipped and the gate watched to fail at the first affected digit; each of the twelve anchors dropped in turn and the refit checked at the dropped point. Only after all of that is the formula an identity rather than a fit.
A recognized constant. A computed number is matched against a declared ring of constants. The gate: the match must hold to far more digits than the search consumed; the digit count must grow when the value is recomputed at higher precision; and the same search, run on noise of the same length, must come back empty. A search that also identifies noise identifies nothing.
一个拟合得到的闭式。 候选公式是根据十二个点处的评估值拟合得到的。关卡：在拟合前保留另外三个点；在这三个点处，用一种与 ansatz 不共享任何内容的方法对原始定义进行数值积分；在两种工作精度下评估两侧并统计一致位数，同时确认一致深度会加深；翻转候选答案中的一个符号，并观察关卡在第一个受影响的数字位上失败；逐一移除十二个锚点中的一个，并在被移除的点检查重新拟合的结果。只有完成所有这些步骤，该公式才是一个恒等式，而不只是拟合。
一个被识别出的常数。 将计算出的数与预先声明的常数集合进行匹配。关卡：匹配必须保持到远多于搜索所消耗的位数；用更高精度重新计算该数时，位数计数必须增长；并且对长度相同的噪声运行同一搜索时，必须没有结果。一个连噪声也能识别出来的搜索，什么也没有识别出来。

Checklist before saying "done"

在说“完成”之前的检查清单

  • Gate declared — reserved points, route, digit bar, controls — before the fit ran.
  • Check route shares no code, no representation, no fitted input with the derivation; any shared remainder is written down.
  • Digit count at reserved points measured and recorded, not adjectivized.
  • Rerun at raised precision; the count deepened.
  • Positive control passed; negative control failed, visibly, at the expected digit.
  • Leave-one-out survived, if a fit determined the answer.
  • Standalone evaluator included: exact data, arbitrary precision, re-measures the gate.
  • Verdict is CLOSED only if every box above is checked; otherwise OPEN, with what remains stated.
Sources and acknowledgments. None of the ideas here is new; what is ours is their assembly into one gate for agent-produced numbers. Declaring the check before seeing the answer is blind analysis as particle physics practices it (Klein and Roodman, Annu. Rev. Nucl. Part. Sci. 55 (2005) 141; MacCoun and Perlmutter, Nature 526 (2015) 187) and preregistration as Nosek, Ebersole, DeHaven and Mellor argue for it (PNAS 115 (2018) 2600); reserved points and leave-one-out are Stone's cross-validatory assessment (J. R. Stat. Soc. B 36 (1974) 111); the warning that separately written routes fail together restates Knight and Leveson's multiversion experiment (IEEE Trans. Softw. Eng. SE-12 (1986) 96).
  • 已声明关卡——保留点、独立路径、位数门槛、控制项——且声明发生在拟合运行之前。
  • 检查路径与推导过程不共享代码、表示形式或已拟合输入；任何共享的剩余部分都已写明。
  • 已测量并记录保留点处的位数计数，而不是用形容词描述。
  • 已在提高精度后重跑，且计数确实加深。
  • 正控制已通过；负控制已在预期数字位上明显失败。
  • 如果答案由拟合决定，leave-one-out 已通过。
  • 已附独立评估器：包含精确数据，支持任意精度，并可重新测量该关卡。
  • 只有在上述每一项都勾选后，结论才是 CLOSED；否则为 OPEN，并说明仍然遗留的内容。
来源与致谢。 这里的任何想法都不新；属于我们的是将它们组装成一道供 Agent 生成数值使用的统一关卡。在看到答案之前声明检查，借鉴了粒子物理学实践中的盲分析（Klein and Roodman, Annu. Rev. Nucl. Part. Sci. 55 (2005) 141；MacCoun and Perlmutter, Nature 526 (2015) 187），而预注册则如 Nosek、Ebersole、DeHaven 和 Mellor 所主张（PNAS 115 (2018) 2600）；保留点和 leave-one-out 借鉴了 Stone 的交叉验证评估（J. R. Stat. Soc. B 36 (1974) 111）；关于分别编写的路径也会同时失败的警告，则重述了 Knight 和 Leveson 的多版本实验（IEEE Trans. Softw. Eng. SE-12 (1986) 96）。