• "ask the user" -- use your host's structured multi-choice question tool if you have one (e.g.

  AskUserQuestion

  ,

  request_user_input

  ). If the host has none, phrase the question as plain text in your next reply and wait for the user's answer.
• File paths like

  references/...

  -- relative to this

  SKILL.md

  ; resolve from the skill directory.
• Slash commands shown in user-facing copy (e.g.

  /evo:discover

  ) -- translate to your host's mention syntax when speaking to the user (e.g.

  $evo discover

  on Codex -- plugin namespace then skill name, separated by a space).

evo-version-check

evo --version

1. Exit 0,

   evo-version-check: OK (plugin=X, cli=X)

   -- continue to step 1.
2. Exit 1, "plugin manifest and installed CLI disagree" -- stop and show the user the script's stderr verbatim; it tells them the

   uv tool install --force evo-hq-cli==<version>

   command to run. Then re-invoke this skill.
3. Exit 2, "evo CLI not on PATH" -- stop and tell the user:

   evo-hq-cli

   isn't on your PATH. Install it once:

   uv tool install evo-hq-cli

   (or

   pipx install evo-hq-cli

   ). Then re-invoke this skill.

4. evo-version-check: command not found

   -- the host's plugin install is incomplete (missing the

   bin/

   wrapper). Fall back to running

   evo --version

   directly and check for

   evo-hq-cli

   in the output; if it's a different package (commonly

   evo 1.x

   -- the unrelated SLAM tool), tell the user to uninstall it and install

   evo-hq-cli

   in its place.

• Main stays clean. Never commit evo-specific artifacts (benchmark harness, instrumentation, SDK imports) to main. Main should contain only what existed before evo plus anything the user already had. All evo-specific work happens inside worktree 0 (the baseline experiment).
• Baseline is a worktree, not a main commit.

  evo init

  creates

  .evo/

  but nothing in main changes. The first real experiment (

  exp_0000

  , created by

  evo new --parent root

  ) is where the benchmark and instrumentation live.
• Ask the user as little as possible. Every question is a beat of friction. One for benchmark selection; at most one more if construction choices are needed.
• Relay the dashboard URL verbatim when it prints. This is the user's window into the run.

• The optimization target: which file(s) benefit from iterative optimization?
• Metric direction for each candidate: is higher better (

  max

  ) or lower better (

  min

  )?
• Critical behaviors worth gating: invariants that must never break regardless of score (e.g., "refund flow works", "core tests pass", "output is valid JSON"). Gates are commands that exit 0 on success, non-zero on failure.

• Full benchmarks: existing scripts that run end-to-end and output a score
• Partial evals: tests, notebooks, or logs with ground truth but not in runnable-score form
• Nothing at all

references/proposing-dimensions.md

• A handful of dimensions relevant to this specific repo (not generic categories).
• Ground each in repo signals: already-instrumented code, stated goals in READMEs, TODO/FIXME patterns, domain defaults.
• Rank by signal × slack × cost answered in prose (no numeric scores — they're vibes).

"I'm proposing these optimization targets for this repo:

[ranked list with one-line explanations, construction complexity, and whether an existing eval covers some of it]

Which should we optimize? Recommended: [default pick with reasoning]."

.evo/project.md

1. Selected benchmark already exists AND is already instrumented for evo (you can see

   from evo_agent import Run

   , an

   import { Run } from '@evo-hq/evo-agent'

   , or the inline

   log_task

   /

   logTask

   helpers in the benchmark source). No wiring needed. Skip this question entirely. Detect the instrumentation style from the source and pass the matching

   --instrumentation-mode <sdk|inline>

   value to

   evo init

   in step 7.
2. Selected benchmark already exists but is NOT instrumented (it just prints a score JSON, or it's a test runner that doesn't yet write per-task traces). Wiring is needed. Ask the question.
3. Selected benchmark needs to be constructed from scratch (case B or C from step 4). Wiring is needed. Ask the question.

"I can wire up the benchmark in one of two ways:

1. SDK mode -- install the SDK (Python:

   pip install evo-hq-agent

   / Node:

   npm install @evo-hq/evo-agent

   ). Richer per-task logs, ~5 lines of user code.
2. Inline mode -- paste a ~30-line helper directly into the benchmark. Zero new dependencies. Same data contract."

evo init

--instrumentation-mode <sdk|inline>

evo init

evo new

exp_0000

1. Tracked-but-modified files -- run

   git diff --name-only

   and

   git diff --cached --name-only

   . If any output line is the optimization target, an existing benchmark file, a gate-referenced script, or any of their import-graph dependencies, stop and ask the user to commit or stash before continuing. Do not commit on their behalf -- the user might be in the middle of an unrelated change.
2. Untracked files visible to git -- run

   git status --short --untracked-files=all

   and look for

   ??

   entries that the target or gates will reference. Classify each:
   • Part of the user's project (e.g., a smoke test they wrote but hadn't committed) -- stop and ask the user to commit it to main themselves.
   • Evo-specific new files (a new gate script you're about to write, a new test fixture) -- do not create these in main. Defer to step 10; they go into the baseline worktree and commit to experiment 0's branch. Every descendant experiment inherits via git branching.
3. Explicit paths inside soon-to-be-ignored directories -- inspect the benchmark command and every gate command for path references (e.g.,

   ./dist/eval-helper

   ,

   node_modules/some-tool/cli.js

   ,

   build/golden_outputs/

   ). For each such path, run

   git ls-files --error-unmatch <path>

   to confirm it's tracked. If any aren't, stop and ask the user to commit them. This catches dependencies that step 6b is about to hide from

   git status

   .

evo init --target <file> --benchmark "<command using {worktree} and {target}>" --metric <max|min> \
  --host <claude-code|codex|opencode|openclaw|hermes|generic> \
  --instrumentation-mode <sdk|inline> [--gate "<gate command>"]

--host

claude-code

codex

opencode

openclaw

hermes

generic

.evo/meta.json

evo dispatch

claude-code

discover

evo host set <value>

evo run

• {worktree}

  resolves to the absolute path of the experiment's worktree directory (e.g.

  /path/to/repo/.evo/run_0000/worktrees/exp_0000

  ). Use this to reference files that live on the experiment branch, not on main.

• {target}

  resolves to the absolute path of the target file inside that worktree (e.g.

  {worktree}/agent/solve.py

  ). Use this when your benchmark needs to load or exec the target dynamically.

evo run

{worktree}

{worktree}/benchmark.py

evo init \
  --target agent/solve.py \
  --benchmark "python3 {worktree}/benchmark.py --target {target}" \
  --metric max \
  --host claude-code

"poetry run python {worktree}/benchmark.py ..."

".venv/bin/python {worktree}/benchmark.py ..."

evo init

.evo/

root

Dashboard live: http://127.0.0.1:8080 (pid 12345)

evo run

{"score": 0.0}

• Test-suite gates --

  pytest

  ,

  cargo test

  ,

  npm test

  , etc. already exit non-zero on failure. Use them as-is.
• Score-threshold gates -- gate the benchmark on a minimum acceptable score. The benchmark script needs a flag like

  --min-score <float>

  that exits 1 when the computed score falls below the threshold. The

  inline_instrumentation.{py,js}

  helpers in

  references/

  show the pattern:

  write_result()

  returns the final score; the script can then compare and

  sys.exit(1)

  .

undefined

If a benchmark you constructed doesn't yet have a `--min-score` mode, add it now (a few lines: parse the threshold flag, compute the score, `sys.exit(1)` if below). Without it the gate is decorative.

Gate commands support `{target}` and `{worktree}` placeholders with the same semantics as benchmark commands (resolved at run time, not at registration). Registering a gate that references `{worktree}/benchmark.py` before the benchmark exists is safe -- the placeholder resolves only when the gate is evaluated, which happens during `evo run` after the benchmark is committed.

Verify registered gates:

```bash
evo gate list root

• If the selected benchmark already existed in the repo (not constructed from scratch): gates are optional at this step, but if you register any benchmark-derived gate, it must use a score-threshold (

  --min-score

  or equivalent) -- not a bare invocation. Subagents can add more during optimization.
• If the benchmark was constructed from scratch (case B or C from the A/B/C classification): a Goodhart-mitigation gate is mandatory before the baseline can run, AND that gate must be a real pass/fail check (score-threshold or correctness assertion that exits non-zero on regression), not a bare benchmark rerun. See

  references/constructing-benchmark.md

  section 6 on "Required gate pairing." Do not proceed to

  evo new

  or

  evo run

  without it. This is the safety against metric gaming -- it is not optional.

references/constructing-benchmark.md

• Design the scoring function (range, direction, meaningful-improvement threshold)
• Assemble test cases (10-20 for programmatic, 15-30 for fuzzy, realistic workload for perf)
• Write the runnable harness (helper/SDK writes the score JSON to

  $EVO_RESULT_PATH

  ; stdout and stderr are free for user output)
• Goodhart check (document gaming strategies, mitigate each with a gate or held-out slice)
• Held-out validation slice (60/70 training, 30/40 held-out) if the benchmark is hand-written

project.md

evo init

SKILL.md

• SDK mode: add

  from evo_agent import Run

  (Python) or

  import { Run } from '@evo-hq/evo-agent'

  (Node) to the benchmark script. Wrap the eval loop per

  references/sdk_python.py

  or

  references/sdk_node.js

  .
• Inline mode: copy the helper from

  references/inline_instrumentation.py

  (or

  .js

  ) into the benchmark. Use

  log_task

  /

  logTask

  per task and

  write_result

  /

  writeResult

  once at the end.

task_<id>.json

$EVO_TRACES_DIR

$EVO_RESULT_PATH

.evo/validate/

.evo/

<worktree>/.evo/validate/

git add

{worktree}

{target}

{worktree}

{target}

evo run

• WORKTREE

  = the worktree path returned by

  evo new

• TARGET

  =

  $WORKTREE/<relative target path, e.g. agent/solve.py>

• VALIDATOR

  =

  <absolute path to this skill dir>/scripts/validate_result.py

  -- resolve by taking the absolute path of this

  SKILL.md

  's directory and appending

  scripts/validate_result.py

undefined

"$ATTEMPT/stdout.log" 2>"$ATTEMPT/stderr.log"

Adapt the benchmark invocation (interpreter, args) to whatever you stored with `evo init`. The non-negotiable part is that the resulting bash command contains no literal `{worktree}`, `{target}`, or relative-script paths -- expand all of them to absolute paths before the shell runs the line. Each invocation gets a fresh attempt subdir, so re-running on failure is safe.

The validator asserts `result.json` exists, is non-empty, and is a JSON object with a numeric `score`. Also verify:

- All dependencies resolve and the command completes.
- Traces appear in `$EVO_TRACES_DIR` (if applicable).
- Each gate script runs cleanly on the unmodified target.

Fix any issues and re-validate before proceeding.

1. add: benchmark harness + test cases

2. add: instrumentation

   (only in SDK mode -- inline mode keeps the harness and instrumentation in one file, so this commit collapses into the previous one)

.gitignore

.evo/
__pycache__/
*.pyc
.pytest_cache/
node_modules/
dist/
build/

.pytest_cache/

.evo/

.evo/

evo init

• What the target does
• What can be changed by optimization vs what must stay stable
• How to interpret benchmark output (score meaning, direction)
• Benchmark determinism -- one line, pick what fits:

  • deterministic by construction

    -- pure code, no randomness, no network

  • uses LLMs with temp=0

    -- expected to be deterministic in practice; flag if it isn't

  • sampling-based, variance expected

    -- inherent noise; optimize will need multi-run strategies
• Environment requirements discovered during validation
• What each gate protects
• Benchmark gaming risks identified during the Goodhart check
• Future experiment candidates (the non-picked dimensions from step 3)

• The dashboard URL (if not already mentioned)
• The baseline experiment ID and score
• The chosen optimization dimension and why
• A one-liner on next steps: "Run

  /evo:optimize

  to start the optimization loop."
• Resume after crash: if the host, the shell, or the machine restarts mid-flow, re-invoke

  evo:optimize

  . Evo reads

  .evo/

  and resumes from the last committed experiment -- no special restore procedure.
• State is local to this machine: experiment commits on branches like

  evo/run_0000/exp_*

  survive

  git push --all

  , but orchestration state (graph, annotations, project notes) lives only in

  .evo/

  . If that history matters to you, back up

  .evo/

  separately (e.g.,

  tar -czf evo-state-$(date +%F).tar.gz .evo/

  ).

• Do NOT modify main after

  evo init

  unless the user explicitly asks. All new artifacts live in worktree 0.
• Do NOT install packages without the user's confirmation from step 5.
• Do NOT skip the held-out gate pairing when the benchmark was constructed from scratch. The gate is the safety net against Goodhart gaming, regardless of whether the benchmark is deterministic.
• Do NOT skip the Goodhart check when the benchmark was constructed from scratch. Gate pairing is mandatory, not optional.