• Contains

  --bucket-only

  → proceed to Phase B (lightweight recalibration)
• Contains

  --propose "<...>"

  → proceed to Phase 0~8 (full rubric bump)
• Neither → Ask user: "What do you want to do? 1) Adjust rubric formula / add/remove dimensions → --propose; 2) Only re-derive bucket boundaries → --bucket-only"

--propose

--bucket-only

• READINESS_HEURISTIC —
  • Default Reference: Calibration pool ≥5 samples + at least 1 cross-sample observation supported by ≥3 samples
  • But Claude can propose bump (even with few samples) if observation signals are exceptionally strong:
    • N=3 but there's a strong counterexample that completely overturns current rubric assumptions (e.g., composite 8.5 vs actual performance 50k, a ≥3x deviation)
    • A single post shows an extreme phenomenon (e.g., a single meme with ≥2000 likes in the comment section)
  • Claude can also reject bump (even with sufficient samples) if evidence is weak:
    • N=10 but observations are low-confidence fragmented patterns with no clear direction
    • User reviews contain numerous non-serious judgments like "just glanced at it"
  • Must be stated in prediction header or cheat-bump output: Whether this proposal is default-aligned or judgment-driven, providing users with a basis for review
• THRESHOLD = 0.8 — Consistency threshold between new ranking and actual performance ranking (4/5). This is hard-coded — statistical rigidity for bump validation
• CROSS_MODEL_AUDIT = true — Call external LLM for independent audit. false is only used for offline scenarios
• REQUIRE_CONFIRM = true — Require explicit user confirmation "yes, bump" before implementation

• No self-scored fallback accepted —

  /cheat-predict

  has a

  --skip-blind

  flag, but

  /cheat-bump

  does not. If Task tool is unavailable → abort bump, report to user "Resolve Task tool issues first before bumping"
• No "I only recalculate composite without re-scoring dimensions" — even if the new formula only adjusts weights without adding dimensions, all dimensions of each prediction must be re-reviewed by the sub-agent. Reason: Old dimension scores may already be contaminated; weight changes cannot guarantee old dimensions are still valid

1. Parse the prediction file to get the corresponding

   scripts/<id>.md

   path (from the

   Script Path

   header field)
2. Verify the script file exists + hash matches the header

   Script Hash

   ; if not → warn (script has been modified) but still spawn sub-agent
3. Spawn cheat-score-blind sub-agent via Task tool:

   Spawn cheat-score-blind sub-agent.
   
   Input:
     script_path: <Script Path from prediction header>
     rubric_notes_path: rubric_notes.md
     sidecar_path: .cheat-cache/bump-rescores/<prediction-id>.json
   
   Task: Score the script according to the current formula in rubric_notes (already updated to new version vN+1).
   Return strict JSON. Write to sidecar file for batch reading by the main bump process.
   
   Do not read state file / predictions/ / videos/ or any other files.
   Do not ask user — you have no user.
   Do not read this prediction file itself — only look at script + rubric.

4. Wait for sub-agent to complete → read sidecar JSON → main process calculates composite using new formula
5. Write "re-score table" to

   .cheat-cache/bump-rescores.json

   (summary). Mark each entry with

   blind: true

   — during Phase 5 cleanup of bump, write this field along with the new score to the

   Re-scored under v<N+1>

   line in the prediction file

For each sample:
  new_composite_rank: Rank sorted by new formula
  actual_plays_rank: Rank sorted by actual plays
  delta: |new_rank - actual_rank|

Output comparison table:
| Sample | composite (v2) | composite (v2.1) | rank (new) | actual | rank (actual) | delta |
|---|---|---|---|---|---|---|
| Hamster | 9.41 | 9.55 | 1 | 1.248M | 1 | 0 |
| Stop Expecting | 8.24 | 9.11 | 2 | 711K | 2 | 0 |
| Boss Nonsense | 7.65 | 8.11 | 4 | 396K | 3 | 1 |
| Job Hunting Paradox | 8.47 | 7.56 | 5 | 168K | 4 | 1 |
| Who Asked You | 8.24 | 7.00 | 6 | 117K | 5 | 1 |

Ranking consistency: 4/5 with |delta| ≤ 1
Pairwise no-regression: All pairs correctly ranked by old formula are not reversed under new formula ✓

• Ranking consistency < THRESHOLD (default 0.8) → Local rejection, explicitly report failure before proceeding to Phase 4
• Pairwise regression occurs → Local rejection

THRESHOLD

CROSS_MODEL_AUDIT=true

mcp__llm-chat__chat

prompt:
You are an independent reviewer. Below is a rubric formula that a content creator is preparing to upgrade.
Please independently judge two things:
1. Ranking consistency: Is the ranking of samples by the new formula consistent with the ranking of actual performance in ≥80% of samples?
2. Explanatory power: Does the new formula better explain the actual performance distribution of the calibration pool compared to the old formula?

Data:
Old formula: (ER×1.5 + SR×1.5 + HP×1.5 + QL + NA + AB + SAT) / 8.5 × 2.0
New formula: (ER×2.0 + HP×1.5 + MS×1.5 + QL + SR + TS + SAT) / 9.0 × 2.0

Calibration pool:
[Full JSON of re-score table from Phase 2]

Ranking comparison:
[Full JSON of table from Phase 3]

Output format:
- Judgment: PASS or REJECT
- Reason: ≥100 words
- Key risks: [List potential issues of new formula if any]

• Local PASS + External PASS → Pass, proceed to Phase 5
• Local PASS + External REJECT → Treat as REJECT. Conflict means at least one party's interpretation is unstable
• Local REJECT → Already terminated in Phase 3
• mcp__llm-chat__chat unavailable → Gracefully degrade to

  CROSS_MODEL_AUDIT=false

  , mark

  last_bump_self_audited: true

  in state file

CROSS_MODEL_AUDIT=false

• Only rely on local judgment
• Continuously mark in state file, cheat-status continuously prompts user "This bump was self-audited, it is recommended to configure mcp__llm-chat__chat"

• Update top metadata:

  • **Current Version**: vN+1

  • **Last bumped at**: <ISO 8601>

  • **Upgrade memos**: See [rubric-memo.md](rubric-memo.md)

    (pointer, do not copy memo content)
• Add a line to version quick reference table (only include version number + formula signature, no evidence samples)
• Update "Current Scoring Dimensions" section (remove NA / AB, add MS / TS)
• Derived evidence section if new dimensions need anchor explanation → Use general language:
  • ✅ Allowed: "Derived evidence: High abstract density samples → CC=1 → Low reach"
  • ❌ Prohibited: "Derived evidence: 'Stop Expecting' CC=1 → Actual performance 137K" (video name + actual performance number)
  • If prohibited pattern is hit → move this section to "Derived Evidence" sub-section in rubric-memo.md, replace with general language in place

• Trigger observation (include real observation ID)
• Evidence data (Full re-score table + ranking comparison of calibration pool, include real video names + actual performance)
• Derived evidence (Include real sample names + actual performance)
• Diagnosis
• New formula
• Cross-model audit conclusion reference (include model name + judgment + reason excerpt)
• Known limitations

rubric_notes.md

• Observations that have been absorbed into new dimensions → Delete (e.g., Observation E absorbed into MS → delete Observation E)
• Observations overturned by new data → Delete
• Unresolved observations → Move to "Pending Validation Hypotheses" section of new version
• Validated "rules" → Move to "Rule Precipitation Area"

• Re-read entire

  rubric_notes.md

  to ensure readers can understand current rules within 60 seconds → trigger additional cleanup if exceeding 600 lines
• Self-check leak guard: Run

  grep -E '\\d+\\s*[wWmMkK]|plays|actual performance|actual'

  on

  rubric_notes.md

  → if any hits → abort bump + rollback, prompt user "rubric_notes.md contains prohibited content (actual performance / play counts)". These contents should be in rubric-memo.md, not rubric_notes.md

ratio

absolute

percentile

--scheme

• --scheme ratio

  forces use of ratio (even if N≥5)

• --scheme absolute

  forces use of absolute

• --scheme percentile

  forces use of percentile (requires N≥3, otherwise error)

--scheme

Old design had

bucket_scheme

state field — removed in v1.1. All skills derive algorithm in real-time based on calibration_samples, no need to persist "which one is currently used". This avoids state inconsistency issues like "forgot to sync after switching scheme".

1. Edit the "Bucket Scheme" section in

   rubric_notes.md

   , replace with new table
2. Update

   baseline_plays

   field in

   .cheat-state.json

   (bucket scheme is not persisted — derived in real-time during next cheat-predict)
3. Append a change record to the top of the bucket section in

   rubric_notes.md

   :

   v2 buckets recalibrated on YYYY-MM-DD: scheme=absolute, baseline=42K (based on N=10 samples)

4. Do not modify any prediction files — bucket tags in historical predictions remain as they are (judgments made under the scheme at the time of writing the sample)

• No re-calculation of composite (formula remains unchanged)
• No re-review of observation sections (rubric remains unchanged)
• No cross-model audit (deterministic derivation requires no judgment)
• No strict sample count threshold (judged by Claude according to READINESS_HEURISTIC; ratio mode can run with N=1)

1. 5 steps cannot be skipped (only for full rubric bump). Reject any request to "run a simplified version first"
2. THRESHOLD is hard-coded (only for full rubric bump). Dynamic adjustment is not allowed
3. Cross-model audit is default (only for full rubric bump). Turning off audit requires explicit marking in state file
4. Cleanup pass is part of bump (only for full rubric bump). Bump cannot be completed without cleaning observation sections
5. REQUIRE_CONFIRM (both modes). Must get explicit user confirmation "yes, bump" or "yes, recalibrate" before final implementation
6. Bucket recalibration does not modify historical predictions. Buckets are prediction-time semantics, post-hoc rewriting destroys blindness

• "Skip calibration pool re-scoring, directly change formula" → Reject. Rule #2
• "Skip cheat-score-blind sub-agent, main Claude can re-score directly" → Reject. Bump does not accept any self-scored fallback — if sub-agent is unavailable → abort bump, do not accept "self-audit"
• "Skip external LLM audit" → Only allowed if

  CROSS_MODEL_AUDIT=false

  is explicitly set
• "Adjust THRESHOLD to 3/5 this time to let it pass" → Reject. Changing THRESHOLD is a meta-level bump
• "Keep all old observations as history" → Violates Rule #3
• "Bump first, do cleanup next time" → Reject. Cleanup is part of bump
• "Only recalculate composite without re-scoring dimensions" → Reject. New weights × old dimensions are still old contamination. Each dimension must be re-reviewed by sub-agent
• "Write full memo into top of rubric_notes.md for easy reading" → Reject. rubric_notes.md is whitelisted for blind sub-agent — containing video names / actual performance → leaks through whitelist. Memo is written in rubric-memo.md (outside whitelist), rubric_notes.md only contains formula + general language dimension definitions + pointer
• "Keep real video names in derived evidence section to make rubric more specific" → Reject. Must use general language in rubric_notes.md ("high abstract density samples"); derived evidence with video names is written in rubric-memo.md

• Upstream:

  /cheat-retro

  detects ≥3 same-direction deviations → propose running

  /cheat-bump

• Dependencies:

  mcp__llm-chat__chat

  (if configured) + Task tool (spawn cheat-score-blind)
• Modifications:

  • rubric_notes.md

    (structural update, never write real video names / actual performance)

  • rubric-memo.md

    (new — append full memo, including evidence + derived evidence)
  • All

    predictions/*.md

    (append Re-scored line, do not modify prediction section)

  • .cheat-state.json

• Downstream: Next

  /cheat-predict

  automatically uses new rubric_version for scoring