loop-runner

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Skill: loop-runner

WHAT

Purpose và context

目的与上下文

Purpose: Drive an autonomous fix loop that CANNOT run away. VERIFY is a shell command whose exit-code 0 means pass (pytest / tsc / a lint) — preferred over LLM self-judgment, because the agent cannot argue its way around a red exit code. The termination guards are MANDATORY and all deterministic.
Trigger (when to use):
- An agent loops edit-then-check repeatedly (fix failing tests, satisfy a typechecker/lint).
- You need a hard guarantee it stops: bounded iterations, bounded wall-clock, stop-on-stall, escalate.
- You want a replayable run-log + a reflexion lesson trail, without trusting the model to "decide it's done".
Non-goals: not an LLM judge of "done"; not a place to tune guard values outside the config; no auto-invocation (
```
disable-model-invocation: true
```
).

目的： 驱动一个绝不会失控的自主修复循环。VERIFY 是一个 shell 命令，其 exit code 为 0 表示通过（pytest / tsc / lint）——优先于 LLM 自我判断，因为 Agent 无法通过自我判断把失败的 exit code 说成通过。终止护栏为强制项，且全部是确定性的。
触发条件（何时使用）：
- Agent 反复循环执行编辑后检查（修复失败测试、满足 typechecker/lint）。
- 需要以硬性保证其停止：有界迭代、有界墙钟时间、停滞即停止、升级至人工。
- 希望获得可重放的 run-log 和 Reflexion 经验轨迹，同时不依赖模型“自行决定已完成”。
非目标： 不是用于判断“完成”的 LLM judge；不是用于在 config 之外调整护栏值的位置；不支持自动调用（
```
disable-model-invocation: true
```
）。

Mental model

心智模型

PROPOSE (workspace state) → VERIFY (shell exit code) → GUARDS (max_iter · budget_seconds · no_progress_k · escalate_after_iter) → REFLEXION (lesson line) → CRITIQUE → REVISE (quarantined adapter) → repeat

. Everything except CRITIQUE/REVISE is deterministic.

Build-now / adapt-later boundary:

DETERMINISTIC (built + tested now): control loop, all guards, state-hash progress detection, run-log artifact, reflexion. →
```
harness/scripts/loop-runner.py
```
QUARANTINED adapter (
```
verified: false
```
): the LLM critique/revise step + the tunable guard values. →
```
harness/loop-runner.config.yaml
```
(the ONE file you edit to adapt). The default revise step is a no-op / shell stub, so the whole loop runs and self-tests WITHOUT an LLM.

PROPOSE（工作区状态） → VERIFY（shell exit code） → GUARDS（max_iter · budget_seconds · no_progress_k · escalate_after_iter） → REFLEXION（经验行） → CRITIQUE → REVISE（隔离适配器）→ 重复

。除 CRITIQUE/REVISE 之外，所有环节都是确定性的。

当前构建／后续适配边界：

DETERMINISTIC（已构建并测试）：控制循环、所有护栏、state-hash 进展检测、 run-log artifact、Reflexion。→
```
harness/scripts/loop-runner.py
```
QUARANTINED adapter（
```
verified: false
```
）：LLM critique/revise 步骤 + 可调整的护栏值。 →
```
harness/loop-runner.config.yaml
```
（唯一需要编辑以进行适配的文件）。默认 revise 步骤为 no-op / shell stub，因此整个循环无需 LLM 即可运行并完成自测。

Input và output contract

输入与输出契约

	Field	Required?	Ý nghĩa
In	`--verify`	có	shell cmd; exit 0 = pass
In	`--config`	không	guard defaults ( `harness/loop-runner.config.yaml` ); CLI flags override config
In	`--revise`	không	shell cmd or real LLM-revise adapter; default no-op stub
In	`--state`	không	glob for state-hash; without it `no_progress_k` is off
In	`--log`	không	run-log path
Out	Run-log JSON	có	iterations, verdicts, termination reason at `run_log.path`
Out	Episodic lessons	on failure	appended to `reflexion.episodic_memory_page` (lazily created with valid frontmatter + `## Origin` )
Out	exit code	có	0 SUCCESS · 2 MAX_ITER · 3 TIMEOUT · 4 NO_PROGRESS · 5 ESCALATE

	字段	是否必填？	含义
输入	`--verify`	是	shell 命令；exit 0 = 通过
输入	`--config`	否	护栏默认值（ `harness/loop-runner.config.yaml` ）；CLI flags 覆盖 config
输入	`--revise`	否	shell 命令或真实的 LLM-revise adapter；默认为 no-op stub
输入	`--state`	否	用于 state-hash 的 glob；未提供时 `no_progress_k` 关闭
输入	`--log`	否	run-log 路径
输出	Run-log JSON	是	迭代记录、判定结果和终止原因，存放于 `run_log.path`
输出	Episodic lessons	失败时	追加到 `reflexion.episodic_memory_page` （按需创建，并带有有效的 frontmatter + `## Origin` ）
输出	exit code	是	0 SUCCESS · 2 MAX_ITER · 3 TIMEOUT · 4 NO_PROGRESS · 5 ESCALATE

Rules và capabilities

规则与能力

RULE-01 (MUST): VERIFY is deterministic (exit code). NEVER let the LLM self-judge "done".
RULE-02 (MUST): The guards are mandatory — never remove one to "let it finish".
RULE-03 (MUST): Quarantined values (guard params + the revise hook) live ONLY in the config. The same value in two places is a leak.
RULE-04 (MUST): Under
```
verified: false
```
, behave fail-safe: assume the revise guess is wrong until conformance proves otherwise.
RULE-05 (MUST): The self-test must stay green and must NOT write into the real wiki (its episodic page is redirected to a temp dir).
Capabilities: run shell commands (verify/revise); hash workspace files; write run-log + episodic wiki page.

RULE-01（MUST）：VERIFY 必须是确定性的（依据 exit code）。绝不让 LLM 自行判断“完成”。
RULE-02（MUST）：护栏为强制项——绝不为了“让它完成”而移除任何护栏。
RULE-03（MUST）：隔离值（护栏参数 + revise hook）只能存在于 config 中。同一值在两处重复即构成泄漏。
RULE-04（MUST）：在
```
verified: false
```
时采用 fail-safe 行为：在一致性验证证明之前，假定 revise 的猜测是错误的。
RULE-05（MUST）：自测必须始终通过，并且绝不能写入真实 wiki（其 episodic 页面会被重定向到临时目录）。
能力：运行 shell 命令（verify/revise）；计算工作区文件哈希；写入 run-log + episodic wiki 页面。

Failure boundaries

失败边界

VERIFY exit 0 → succeeded (SUCCESS, exit 0).
```
max_iter
```
reached → failed (MAX_ITER, exit 2).
```
budget_seconds
```
exceeded → failed (TIMEOUT, exit 3).
State-hash unchanged K iterations → blocked (NO_PROGRESS, exit 4).
```
escalate_after_iter
```
hit → hand off to a human (ESCALATE, exit 5).

VERIFY exit 0 → 成功（SUCCESS，exit 0）。
达到
```
max_iter
```
→ 失败（MAX_ITER，exit 2）。
超过
```
budget_seconds
```
→ 失败（TIMEOUT，exit 3）。
state-hash 连续 K 次迭代保持不变 → 阻塞（NO_PROGRESS，exit 4）。
达到
```
escalate_after_iter
```
→ 移交给人工（ESCALATE，exit 5）。

HOW

Main workflow

主工作流

Step	Type	Inputs	Action	Outputs/exit	Failure/next
W01	deterministic	—	`python3 harness/scripts/loop-runner.py selftest` (5 guard scenarios, no LLM)	green	red → fix runner first, stop
W02	deterministic	workspace	PROPOSE — current workspace state	state	→ W03
W03	deterministic	`--verify`	VERIFY — run shell cmd	exit 0 → SUCCESS	non-zero → W04
W04	deterministic	counters, state-hash, clock	GUARDS — max_iter, budget_seconds, no_progress_k, escalate_after_iter	continue or terminal code 2/3/4/5	terminal → write run-log, stop
W05	effect	verdict	REFLEXION — append one lesson line to episodic page	lesson	→ W06
W06	judgment	failure output	CRITIQUE → REVISE via quarantined adapter (LLM, stub or shell cmd)	new workspace state	→ W02

Chi tiết từng bước (nguồn chân lý cho W01–W06):

步骤	类型	输入	操作	输出／退出码	失败／下一步
W01	确定性	—	`python3 harness/scripts/loop-runner.py selftest` （5 个护栏场景，无 LLM）	通过	失败 → 先修复 runner，然后停止
W02	确定性	工作区	PROPOSE——当前工作区状态	状态	→ W03
W03	确定性	`--verify`	VERIFY——运行 shell 命令	exit 0 → SUCCESS	非零 → W04
W04	确定性	计数器、state-hash、时钟	GUARDS——max_iter、budget_seconds、no_progress_k、escalate_after_iter	继续，或终止码 2/3/4/5	终止 → 写入 run-log，然后停止
W05	效果	判定结果	REFLEXION——向 episodic 页面追加一行经验	经验	→ W06
W06	判断	失败输出	通过隔离适配器执行 CRITIQUE → REVISE（LLM、stub 或 shell 命令）	新的工作区状态	→ W02

各步骤详情（W01–W06 的事实来源）：

How to run

运行方式

undefined

undefined

self-test — 5 deterministic guard scenarios, no LLM, no deps

python3 harness/scripts/loop-runner.py selftest

real loop around a verify command (config supplies the guard defaults)

python3 harness/scripts/loop-runner.py run
--config harness/loop-runner.config.yaml
--verify "pytest -q"
--revise "<shell cmd, or the real LLM-revise adapter>"
--state "src/**" --log harness/out/loop-runner-run.json

CLI flags override config. Process exit: 0 SUCCESS · 2 MAX_ITER · 3 TIMEOUT · 4 NO_PROGRESS · 5 ESCALATE.

CLI flags 覆盖 config。进程退出码：0 SUCCESS · 2 MAX_ITER · 3 TIMEOUT · 4 NO_PROGRESS · 5 ESCALATE。

The loop (one iteration)

循环（单次迭代）

PROPOSE — the current workspace state (initial proposal, or the last iteration's revise).
VERIFY — run the shell cmd; exit 0 → stop SUCCESS.
GUARDS (checked every iteration, all enforced):
- ```
max_iter
```
  — hard backstop (always on; cannot be disabled).
- ```
budget_seconds
```
  — wall-clock budget (0 = off).
- ```
no_progress_k
```
  — state-hash unchanged for K consecutive iterations → NO_PROGRESS (off if no
```
state_paths
```
  ).
- ```
escalate_after_iter
```
  — hand off to a human → ESCALATE (0 = off).
REFLEXION — append one "lesson" line to the episodic-memory wiki page on failure.
CRITIQUE → REVISE — the quarantined adapter (the LLM, or a stub / shell cmd). Then repeat.

PROPOSE——当前工作区状态（初始提案，或上一次迭代的 revise）。
VERIFY——运行 shell 命令；exit 0 → 停止并标记 SUCCESS。
GUARDS（每次迭代检查，全部强制执行）：
- ```
max_iter
```
  ——硬性兜底（始终开启；无法禁用）。
- ```
budget_seconds
```
  ——墙钟时间预算（0 = 关闭）。
- ```
no_progress_k
```
  ——state-hash 连续 K 次迭代保持不变 → NO_PROGRESS（未提供
```
state_paths
```
  时关闭）。
- ```
escalate_after_iter
```
  ——移交给人工 → ESCALATE（0 = 关闭）。
REFLEXION——失败时，向 episodic-memory wiki 页面追加一行“lesson”。
CRITIQUE → REVISE——通过隔离适配器执行（LLM、stub 或 shell 命令）。然后重复。

Output

输出

Run-log JSON (iterations, verdicts, termination reason) at
```
run_log.path
```
.
Episodic-memory lessons appended to
```
reflexion.episodic_memory_page
```
(lazily created with valid frontmatter +
```
## Origin
```
).

Run-log JSON（迭代记录、判定结果、终止原因）位于
```
run_log.path
```
。
Episodic-memory 经验教训会追加到
```
reflexion.episodic_memory_page
```
（按需创建，并带有有效的 frontmatter +
```
## Origin
```
）。

Branches

分支

ID	Kind	Guard	Hành vi	Skip / failure	Rejoin
B01	capability_optional	`--state` / `state_paths` given	enable `no_progress_k` state-hash stall detection	absent → guard off	W04
B02	user_optional	`budget_seconds` or `escalate_after_iter` > 0	enforce wall-clock budget / human escalation	0 → off ( `max_iter` stays always on)	W04
B03	capability_optional	real LLM-revise adapter wired	W06 uses the LLM; `verified: false` → fail-safe (RULE-04)	not wired → no-op / shell stub	W02

ID	类型	护栏	行为	跳过／失败	汇合点
B01	capability_optional	提供 `--state` / `state_paths`	启用 `no_progress_k` state-hash 停滞检测	未提供 → 关闭护栏	W04
B02	user_optional	`budget_seconds` 或 `escalate_after_iter` > 0	强制执行墙钟时间预算／人工升级	0 → 关闭（ `max_iter` 始终保持开启）	W04
B03	capability_optional	已接入真实的 LLM-revise adapter	W06 使用 LLM； `verified: false` → fail-safe（RULE-04）	未接入 → no-op / shell stub	W02

Validation và stopping

验证与停止

Stopping is decided only by exit codes and guards, never by the model.

max_iter

cannot be disabled, so every run terminates. Selftest must stay green and write nothing into the real wiki.

停止仅由 exit code 和护栏决定，绝不由模型决定。

max_iter

无法禁用，因此每次运行都会终止。Selftest 必须始终通过，且不得写入真实 wiki。

Examples

示例

Positive:
```
run --verify "pytest -q" --state "src/**"
```
, revise fixes the failing test on iteration 2 → VERIFY exit 0 → SUCCESS, process exit 0, run-log lists 2 iterations.
Boundary/failure: revise stub never changes files under
```
src/**
```
,
```
no_progress_k=2
```
→ state-hash equal 2 iterations → NO_PROGRESS, exit 4, lesson appended; the model cannot argue it's done.

正向：
```
run --verify "pytest -q" --state "src/**"
```
，revise 在第 2 次迭代修复失败测试 → VERIFY exit 0 → SUCCESS，进程退出码为 0，run-log 记录 2 次迭代。
边界／失败： revise stub 始终不改变
```
src/**
```
下的文件，
```
no_progress_k=2
```
→ state-hash 连续 2 次迭代相同 → NO_PROGRESS，exit 4，并追加 lesson；模型无法诡辩称其已完成。