finetuning-method-selection

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Fine-Tuning Method Selection

微调方法选择

This is the router skill for the fine-tuning lifecycle: it decides whether fine-tuning is the right tool at all, and if so, which method and which base-model size class. Every other skill in this plugin assumes this routing already happened — start here before opening

lora-qlora-recipes

preference-optimization

, or

grpo-rlvr-training

这是微调生命周期中的路由技能：它会判断微调是否是合适的工具，如果是，则确定使用哪种方法以及哪个规模级别的基础模型。此插件中的所有其他技能都假设已经完成了这一路由步骤——在打开

lora-qlora-recipes

、

preference-optimization

或

grpo-rlvr-training

之前，请先从这里开始。

When to Use This Skill

何时使用此技能

Starting any fine-tuning effort, before a framework or base model has been chosen.
Unsure whether RAG or prompt engineering would solve the problem more cheaply than training.
Choosing between preference optimization (DPO family) and a reinforcement method (GRPO/RLVR) for the same underlying task.
Sizing a candidate model/method combination before committing to a run.

启动任何微调工作时，在选择框架或基础模型之前。
不确定RAG或提示工程是否能比训练更低成本地解决问题时。
针对同一底层任务，在偏好优化（DPO系列）和强化学习方法（GRPO/RLVR）之间做选择时。
在投入运行前，评估候选模型/方法组合的规模时。

Quick Reference

快速参考

Situation	Route
Facts change often (prices, docs, news)	RAG, not fine-tuning
Desired behavior still being figured out	Prompt engineering
Stable domain knowledge, ≥500MB text	CPT then SFT — see Off-Ramps First
Have input/output demonstrations	SFT — see `lora-qlora-recipes`
Have preference pairs or thumbs-up/down	DPO/ORPO/KTO — see `preference-optimization`
Have a verifiable pass/fail signal	GRPO+RLVR — see `grpo-rlvr-training`
No eval harness yet	Stop — see `eval-harness-first`

场景	选择方案
事实频繁变化（价格、文档、新闻）	RAG，而非微调
期望行为仍在摸索中	提示工程
稳定的领域知识，文本量≥500MB	先进行CPT再做SFT——请先查看替代方案
拥有输入/输出演示样本	SFT——查看 `lora-qlora-recipes`
拥有偏好配对或点赞/差评数据	DPO/ORPO/KTO——查看 `preference-optimization`
拥有可验证的通过/失败信号	GRPO+RLVR——查看 `grpo-rlvr-training`
尚无评估工具链	停止操作——查看 `eval-harness-first`

Off-Ramps First

先考虑替代方案

Most requests that sound like "fine-tune this" are served better and cheaper elsewhere. Check these off-ramps before opening a training run:

Knowledge-bound and volatile (the gap is facts that change — prices, docs, current events): route to RAG, not fine-tuning. A fine-tuned model bakes in a snapshot; volatile facts go stale immediately.
Behavior-bound and shifting (the desired behavior is still being figured out, or changes per request): route to prompt engineering. Fine-tuning locks in a behavior; don't lock in one that hasn't stabilized yet.
Stable, dense domain knowledge: this is where continued pretraining (CPT) enters, sized by how much domain text exists:

Domain text volume	Route
<10MB	RAG only
10MB–500MB	RAG + fine-tune
500MB–10GB	CPT, then SFT
>10GB	CPT required

CPT learning rate ≈ 10% of the pretraining LR. CPT is guidance-only in this plugin — sizing and LR guidance live here, but this plugin does not execute a CPT run.

大多数听起来像是“对此进行微调”的需求，通过其他方式处理会更高效、成本更低。在启动训练之前，请先检查以下替代方案：

知识相关且易变（差距在于频繁变化的事实——价格、文档、时事）：选择RAG，而非微调。微调后的模型会固化一个快照；易变的事实会立即过时。
行为相关且不断变化（期望行为仍在摸索中，或随请求变化）：选择提示工程。微调会锁定一种行为；不要锁定尚未稳定的行为。
稳定、密集的领域知识：此时需要用到持续预训练（CPT），具体取决于领域文本的规模：

领域文本量	选择方案
<10MB	仅使用RAG
10MB–500MB	RAG + 微调
500MB–10GB	先CPT，再SFT
>10GB	必须进行CPT

CPT的学习率≈预训练学习率的10%。此插件中仅提供CPT的指导——规模和学习率指导在此处，但插件不会执行CPT运行。

Method Router

方法路由

Once the off-ramps are ruled out, this is the full decision tree (verbatim from the research this plugin is built on):

New FACTS?  volatile → RAG | stable+dense → CPT (LR ~10% of pretrain) → SFT
New BEHAVIOR? shifting → prompt-engineering | stable:
  demos → SFT (LoRA/QLoRA, all-linear, α=2r)
  preference pairs → DPO (SimPO if length-bias, ORPO if memory-bound)
  unpaired 👍/👎 → KTO
  verifiable success → RLVR + GRPO (DAPO/GSPO/Dr.GRPO per failure mode)
Deploy: FP8 (Hopper+) | NVFP4 (Blackwell scale) | AWQ (older) | GGUF+imatrix (edge)
BEFORE ANY OF THIS: the eval harness must exist first.

Read the tree top-down: answer "new facts or new behavior," then follow the branch that matches the data shape in hand (demos, preference pairs, thumbs up/down, or verifiable success/failure). The data shape picks the method — not the other way around.

在排除替代方案后，以下是完整的决策树（直接取自构建此插件的研究内容）：

New FACTS?  volatile → RAG | stable+dense → CPT (LR ~10% of pretrain) → SFT
New BEHAVIOR? shifting → prompt-engineering | stable:
  demos → SFT (LoRA/QLoRA, all-linear, α=2r)
  preference pairs → DPO (SimPO if length-bias, ORPO if memory-bound)
  unpaired 👍/👎 → KTO
  verifiable success → RLVR + GRPO (DAPO/GSPO/Dr.GRPO per failure mode)
Deploy: FP8 (Hopper+) | NVFP4 (Blackwell scale) | AWQ (older) | GGUF+imatrix (edge)
BEFORE ANY OF THIS: the eval harness must exist first.

自上而下阅读决策树：回答“是新增事实还是新增行为”，然后选择与手头数据形态匹配的分支（演示样本、偏好配对、点赞/差评、可验证的成功/失败）。数据形态决定方法——而非反过来。

Worked Routing Examples

路由示例

"Users want the assistant to follow our support macros exactly." Behavior is stable and demonstrable from transcripts → demos → SFT.
"We have pairs of good/bad responses from reviewer thumbs-up/down, unpaired." → unpaired signal → KTO, not DPO (DPO needs paired preferences).
"The model can already solve some of these math problems and we can grade correctness automatically." → verifiable success signal → GRPO+RLVR, and only after confirming the model succeeds at least sometimes (see Key Routing Facts below).
"We want the model to know this week's pricing page." → volatile facts → RAG, no training run at all.

“用户希望助手严格遵循我们的支持宏指令。” 行为稳定且可通过转录样本演示→演示样本→SFT。
“我们有来自审核者点赞/差评的好坏响应对，但未配对。”→未配对信号→KTO，而非DPO（DPO需要配对的偏好数据）。
“模型已经能解决部分数学问题，我们可以自动评分正确性。”→可验证的成功信号→GRPO+RLVR，且需先确认模型至少能偶尔成功（请参阅下方关键路由事实）。
“我们希望模型了解本周的定价页面。”→易变事实→RAG，完全无需训练。

Key Routing Facts

关键路由事实

Loss-function choice is low-leverage. A 240-H100-run study found method choice worth ~1 percentage point versus ~50 points for model scale, and zero of 20 DPO variants beat vanilla DPO. Don't spend a routing decision agonizing over DPO-variant selection — spend it on getting the data shape and scale right.
DPO is for taste, GRPO+RLVR is for reasoning. Preference pairs that encode a subjective judgment (tone, style, "which answer is better") route to DPO. Tasks with a verifiable pass/fail signal (math, code, tool calls) route to GRPO+RLVR instead.
RL is not the fix for a model that never succeeds. GRPO and other RL methods sharpen an existing capability — they don't teach one from zero. If the model doesn't yet understand the task or output format, run SFT first; only bring in RL once the model succeeds at least sometimes.

损失函数的选择影响较小。一项240次H100运行的研究发现，方法选择的影响约为1个百分点，而模型规模的影响约为50个百分点，且20种DPO变体中没有一种能击败原生DPO。不要在路由决策上纠结于DPO变体的选择——应专注于确保数据形态和规模正确。
DPO适用于主观偏好，GRPO+RLVR适用于推理。编码主观判断（语气、风格、“哪个答案更好”）的偏好配对应选择DPO。具有可验证通过/失败信号的任务（数学、代码、工具调用）应选择GRPO+RLVR。
强化学习无法修复完全无法成功的模型。GRPO和其他强化学习方法会强化现有能力——无法从零开始教授能力。如果模型尚未理解任务或输出格式，请先运行SFT；只有当模型至少能偶尔成功时，再引入强化学习。

Common Routing Mistakes

常见路由错误

Reaching for fine-tuning to fix facts that change weekly — that's a RAG problem, and fine-tuning will just go stale faster than the source data does.
Picking a DPO variant before checking whether the actual bottleneck is data quality or model scale — variant choice is the ~1pp lever, not the ~50pp one.
Starting an RL run on a model that fails every rollout — route to SFT first so RL has something to sharpen.
Treating CPT as the default for "the model doesn't know our domain" — check the data volume thresholds first; under 500MB, RAG or RAG+fine-tune iterates faster than a CPT run.

试图通过微调来解决每周变化的事实——这是RAG的适用场景，微调只会比源数据更快过时。
在检查实际瓶颈是数据质量还是模型规模之前就选择DPO变体——变体选择的影响约为1个百分点，而非50个百分点。
在每次输出都失败的模型上启动强化学习运行——应先选择SFT，让强化学习有可以优化的基础。
将CPT作为“模型不了解我们的领域”的默认方案——先检查数据量阈值；低于500MB时，RAG或RAG+微调的迭代速度比CPT运行更快。

Model Selection

模型选择

Base-model choice is size-class first, family second, and it goes stale fast — so it lives in exactly one place:

references/model-catalog.md

. That file is the only place in this plugin (and in the DGX Spark ops plugin) that names a base model family. Neither this skill nor

references/memory-math.md

names one; both describe models by size class only (for example, "8B-class LoRA," not a model name).

The catalog is dated on purpose — model rankings turn over quarterly. It carries a "last verified" date and a refresh checklist. Before trusting a row, check that date; if stale, work the refresh checklist in the catalog before recommending a model from it.

Precedence when the catalog and a method skill disagree: the catalog's per-row Notes column states hardware/size-class feasibility, not a method recommendation —

lora-qlora-recipes

's LoRA vs QLoRA vs Full FT table (routed by task shape) governs the actual method choice.

基础模型选择应优先考虑规模级别，其次是模型家族，且信息更新很快——因此相关内容仅存于一个位置：

references/model-catalog.md

。该文件是此插件（以及DGX Spark ops插件）中唯一提及基础模型家族的地方。此技能和

references/memory-math.md

都不会提及具体模型名称；两者仅按规模级别描述模型（例如，“8B级LoRA”，而非具体模型名）。

该目录带有日期标记——模型排名每季度更新一次。它包含“最后验证日期”和更新清单。在信任某一行内容之前，请检查该日期；如果已过期，请先执行目录中的更新清单，再推荐其中的模型。

当目录与方法技能存在分歧时的优先级：目录每行的“备注”列说明了硬件/规模级别的可行性，而非方法推荐——

lora-qlora-recipes

中的LoRA vs QLoRA vs 全微调表格（根据任务形态路由）决定实际方法选择。

Memory Feasibility

内存可行性

Before committing to a method, size it: total memory ≈ params × dtype bytes + optimizer state + gradients + activations. Work each term for the chosen dtype and method (full fine-tune, LoRA, or QLoRA) — worked worksheets and size-class examples live in

references/memory-math.md

On DGX Spark specifically, unified-memory behavior breaks the naive estimate (transient load peaks,

nvidia-smi

underreporting, thermal throttling on long runs). Once the

dgx-spark-ops

plugin is installed, defer Spark-specific feasibility calls to its

spark-memory-thermal-ops

skill rather than re-deriving them here.

在确定方法之前，请评估其规模：总内存≈参数数量×数据类型字节数 + 优化器状态 + 梯度 + 激活值。针对所选的数据类型和方法（全微调、LoRA或QLoRA）计算每个项——已计算的工作表和规模级别示例位于

references/memory-math.md

中。

特别是在DGX Spark上，统一内存行为会打破简单的估算（瞬时负载峰值、

nvidia-smi

报告不足、长时间运行时的热节流）。安装

dgx-spark-ops

插件后，Spark特定的可行性评估请交由其

spark-memory-thermal-ops

技能处理，无需在此重新推导。