trace-to-training-data

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Trace To Training Data

轨迹转训练数据

This skill assumes

eval-harness-first

already graded the traces being converted here — goldens, graders, and

runs/<run-id>/results.json

all exist before conversion starts. This is the flywheel edge that skill names in its own flow: "the same labeled traces become the training set." Conversion happens here; grading already happened upstream.

Input: graded traces —

eval/goldens.jsonl

plus

runs/<run-id>/results.json

, each row carrying a

task_id

, a

verdict

from the grader, and a

reward

when the task supports a scalar score (judge score, execution partial-credit, or an RLVR verifier):

json

{"task_id": "t-042", "trace_id": "t-042-a3",
 "messages": [{"role": "user", "content": "..."}],
 "verdict": "pass", "reward": 0.91,
 "grader": "exact_match"}

Output format: rows shaped exactly like

dataset-curation

's Format Selection table — SFT

messages

rows or DPO

prompt

chosen

rejected

pairs — so this skill's output is that skill's input with no reshaping step in between.

该技能假定

eval-harness-first

已完成待转换轨迹的分级——在转换开始前，goldens（黄金示例）、分级器以及

runs/<run-id>/results.json

文件均已存在。这正是该技能在自身流程中提到的飞轮优势：“相同的带标签轨迹成为训练集”。转换操作在此处进行，而分级已在上游完成。

输入： 已分级的轨迹——

eval/goldens.jsonl

与

runs/<run-id>/results.json

，每行包含

task_id

、分级器给出的

verdict

（判定结果），以及当任务支持标量评分时的

reward

（奖励值，如评审分数、执行部分得分或RLVR验证器评分）：

json

{"task_id": "t-042", "trace_id": "t-042-a3",
 "messages": [{"role": "user", "content": "..."}],
 "verdict": "pass", "reward": 0.91,
 "grader": "exact_match"}

输出格式： 行结构完全匹配

dataset-curation

（数据集整理）的格式选择表——SFT的

messages

行或DPO的

prompt

chosen

rejected

对，因此该技能的输出可直接作为

dataset-curation

的输入，无需额外的格式转换步骤。

The Principle

核心原则

The eval harness already did the labeling work: every trace in

results.json

carries a verdict, and often a reward, before this skill ever touches it. Converting a graded trace into a training row is mechanical — pick a shape from

dataset-curation

's table, map fields, write JSONL. Curation is the work that remains — which traces clear a quality bar, which pairs are informative, and which rows must never enter the training set at all.

Treat any conversion step that requires re-judging a trace as a sign the harness is missing a grader, not a gap this skill should paper over. A trace with no verdict or reward isn't convertible yet — route it back to

eval-harness-first

first, don't hand-label it here to unblock conversion.

评估框架已完成标注工作：在该技能处理之前，

results.json

中的每条轨迹都带有判定结果，通常还包含奖励值。将已分级的轨迹转换为训练行是机械性操作——从

dataset-curation

的格式表中选择一种结构，映射字段，写入JSONL文件。剩余的工作是数据整理——筛选出符合质量标准的轨迹、具备信息价值的样本对，以及绝对不能进入训练集的行。

若转换过程中需要重新判定轨迹，应将其视为评估框架缺少对应分级器的信号，而非该技能需要弥补的漏洞。没有判定结果或奖励值的轨迹暂无法转换——先将其送回

eval-harness-first

处理，不要在此处手动标注以推进转换。

SFT From Traces

从轨迹生成SFT数据

Keep the top-reward fraction of successful trajectories, not every passing one. Rank passing traces by reward and take a fraction (the Agent-lightning pattern) rather than every trace that merely cleared the pass bar — a trace that barely passed is a weaker SFT signal than one that scored well above threshold.
Expert-corrected failures become gold SFT examples directly (the Langfuse pattern) — when a human edits a failing trace's output into a correct one, that correction needs no reward threshold; a human already validated it. Route corrections straight into the SFT set.
Step-level masking beats whole-trajectory discard for multi-step traces. When only some steps in a multi-step trajectory are bad, mask the loss on the bad steps and keep the good ones, rather than discarding the whole trajectory. SRFT reports 32.2% vs. 30.9% on SWE-bench for step-level critic masking over trajectory discard — a real, if modest, gap from the finer-grained cut.

保留成功轨迹中奖励值排名靠前的部分，而非所有通过的轨迹。按奖励值对通过的轨迹排序，选取其中一部分（即Agent-lightning模式），而非仅保留刚好达到及格线的所有轨迹——勉强通过的轨迹作为SFT信号的强度远低于得分远超阈值的轨迹。
经专家修正的失败轨迹可直接成为黄金SFT示例（即Langfuse模式）——当人工将失败轨迹的输出编辑为正确结果时，该修正无需设置奖励阈值，因为已由人工验证。将修正后的轨迹直接纳入SFT数据集。
对于多步骤轨迹，步骤级掩码优于丢弃整个轨迹。当多步骤轨迹中仅部分步骤存在问题时，对有问题的步骤进行损失掩码并保留良好步骤，而非丢弃整个轨迹。SRFT在SWE-bench上的测试显示，步骤级批评掩码的效果为32.2%，而丢弃整个轨迹的效果为30.9%——这种细粒度处理带来了切实（尽管幅度不大）的性能差距。

Preference Pairs From Traces

从轨迹生成偏好对

Build pairs from passing-vs-failing trajectories on the SAME task, never from unrelated best- and worst-scoring traces pulled across different tasks — cross-task pairs teach the model to prefer one task over another, not one response over another.
Select the rejected member at μ−2σ of the reward distribution for that task, never the absolute minimum.
```
preference-optimization
```
's Pair Construction section owns the full selection formula; this skill supplies the graded trajectories it consumes.
Judge-scored delta selection cuts pair volume without cutting signal. Score each candidate pair by chosen-minus-rejected judge delta and keep only the highest-delta subset — the top 5k of a 16.5k candidate pool matched the full pool's downstream result. Build the full candidate set first, then filter by delta; don't cap generation at 5k up front.

基于同一任务的成功与失败轨迹构建样本对，绝不能从不同任务中选取无关的最高得分与最低得分轨迹——跨任务样本对会让模型学会偏好某一任务而非某一响应。
选取该任务奖励分布中μ−2σ位置的轨迹作为被拒绝样本，而非绝对最小值。
```
preference-optimization
```
（偏好优化）的样本对构建部分包含完整的选择公式；该技能为其提供所需的已分级轨迹。
基于评审分数差值筛选样本对可减少样本量且不损失信号。通过选中样本与被拒绝样本的评审分数差值对候选样本对评分，仅保留差值最高的子集——在16.5k候选样本池中选取前5k，其下游效果与完整样本池相当。先构建完整候选集，再通过差值筛选；不要预先将生成数量限制为5k。

Hygiene

数据规范

Scan for secrets and PII before any row ships, and redact what's found. Traces sourced from production logs can carry credentials, API keys, tokens, or customer data — run a secret/PII scan over every SFT and DPO row and redact matches; conversion fails closed (the row is dropped, not shipped with the raw content) if sensitive fields remain after redaction. Never commit secrets.
Eval goldens must never leak into training data. Hold every
```
eval/goldens.jsonl
```
ID out of every converted SFT and DPO set — a trace that also appears as a golden trains on the exact item the checkpoint gets graded against later, silently inflating every subsequent eval run.
Dedup against the training set, not just within the newly converted rows — exact-match or embedding-similarity, matching
```
dataset-curation
```
's dedup method field, run against whatever training data already exists before this batch merges in.
Provenance goes into the dataset card. Every converted row must trace back to its source
```
run_id
```
and
```
trace_id
```
—
```
dataset-curation
```
's Provenance field checks for exactly this link back to
```
trace-to-training-data
```
output; a row with no traceable source isn't ready to merge.

在输出任何数据行前扫描机密信息与PII（个人可识别信息），并对发现的内容进行脱敏处理。源自生产日志的轨迹可能包含凭证、API密钥、令牌或客户数据——对每条SFT和DPO数据行进行机密信息/PII扫描，并对匹配项脱敏；若脱敏后仍存在敏感字段，转换将终止（该行会被丢弃，不会以原始内容输出）。绝不提交机密信息。
评估黄金示例绝不能泄露到训练数据中。将
```
eval/goldens.jsonl
```
中的所有ID排除在所有转换后的SFT和DPO数据集之外——若某条轨迹同时作为黄金示例，模型会在后续评估时针对该精确样本进行训练，从而悄无声息地抬高后续所有评估的分数。
与现有训练集去重，而非仅在新转换的数据行内去重——采用与
```
dataset-curation
```
去重方法字段匹配的精确匹配或嵌入相似度方法，在合并新批次数据前，针对已存在的所有训练数据进行去重。
溯源信息需纳入数据集卡片。每条转换后的数据行必须可追溯至其来源的
```
run_id
```
和
```
trace_id
```
——
```
dataset-curation
```
的溯源字段会检查该与
```
trace-to-training-data
```
输出的关联；无法追溯来源的数据行暂不具备合并条件。