Back to Details

hyperloom-workload-optimizer

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese
<!-- Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved. See LICENSE for license information. -->
<!-- Copyright (c) 2026 Advanced Micro Devices, Inc. All rights reserved. See LICENSE for license information. -->

Hyperloom Workload Optimizer

Hyperloom工作负载优化器

You are the catalog entry point for Hyperloom optimization on AMD Instinct GPUs. Bootstrap the workspace, prepare the runtime environment, collect workload parameters, then install, launch, and monitor the optimizer. This skill owns the orchestration and the launcher gates; environment prep and workload intake are delegated to the skills the Hyperloom wheel installs, and 
@${HYPERLOOM_SKILL_PATH}
(
inference_optimizer
) is the execution baseline.
Do not manually optimize inside chat unless debugging.
你是AMD Instinct GPU上Hyperloom优化的目录入口点。引导工作区搭建、准备运行环境、收集工作负载参数，然后安装、启动并监控优化器。本技能负责编排和启动管控；环境准备和工作负载收集由Hyperloom安装包附带的技能处理，
@${HYPERLOOM_SKILL_PATH}
（
inference_optimizer
）是执行基准。
除非调试，否则不要在聊天中手动进行优化。

Prerequisites

前置条件

  • AMD Instinct GPU host (MI300X / MI325X / MI355X) with ROCm
  • /dev/kfd
    and 
    /dev/dri
    present; 
    amd-smi
    or 
    rocm-smi
    works
  • Python 3.10+ and network access to install the Hyperloom wheel
  • Anthropic (or compatible) LLM credentials for agent backends
  • A dedicated agent workspace directory
Every command in this skill runs on that GPU host. Confirm the shell you are in is on it before Phase 0, so a bootstrap does not land on a machine with no GPU.
The Hyperloom runtime ships via 
pip install
of the published wheel.
  • 搭载ROCm的AMD Instinct GPU主机（MI300X / MI325X / MI355X）
  • 存在
    /dev/kfd
    和
    /dev/dri
    ；
    amd-smi
    或
    rocm-smi
    可正常运行
  • Python 3.10+，且有网络权限用于安装Hyperloom安装包
  • 用于智能体后端的Anthropic（或兼容）LLM凭证
  • 专用的智能体工作区目录
本技能中的所有命令都在该GPU主机上运行。在Phase 0之前确认当前所在的Shell处于该主机上，避免引导操作落到无GPU的机器上。
Hyperloom 运行时通过
pip install
已发布的安装包提供。

What Hyperloom runs

Hyperloom运行内容

The CLI starts a Python Coordinator that coordinates:
  • Orchestration — baseline, explore, specialist, integrate_patch, sweep
  • Kernel — trace_analyze, run_optimization, integrate
  • Critic — proposal review (default 
    --critic-agent
    )
  • Robustness — health monitoring and RCA (default 
    --robustness-agent
    )
State lives under a session directory per run; run-state root is 
$USER_DATA_PATH
(default 
/workspace/hyperloom
), independent of the install directory (
INSTALL_DIR
, where the wheel and 
.env
live) and may point to shared storage. Layout: 
$USER_DATA_PATH/runtime/
(install.sh outputs, 
kernel-agent.env.sh
), 
logs/
, and 
<model_basename>/<UTC_ts>/
per session holding 
manifest.json
, 
state.json
, 
runs/
, 
reports/
, 
optimizer_runs/
.
CLI启动一个Python协调器，负责协调：
  • 编排 — 基准测试、探索、专家处理、补丁集成、扫描
  • 内核 — 追踪分析、运行优化、集成
  • 评审 — 提案审核（默认
    --critic-agent
    ）
  • 鲁棒性 — 健康监控与根本原因分析（默认
    --robustness-agent
    ）
状态存储在每次运行对应的会话目录下；运行状态根目录为
$USER_DATA_PATH
（默认
/workspace/hyperloom
），与安装目录（
INSTALL_DIR
，存放安装包和
.env
）相互独立，可指向共享存储。目录结构：
$USER_DATA_PATH/runtime/
（install.sh输出、
kernel-agent.env.sh
）、
logs/
，以及每个会话对应的
<model_basename>/<UTC_ts>/
，其中包含
manifest.json
、
state.json
、
runs/
、
reports/
、
optimizer_runs/
。

Workflow overview

工作流概述

Match 
hyperloom-custom-advanced
section order — do not ask workload questions while writing 
.env
or during 
/hyperloom-setup
.
  • Phase 0 Bootstrap — 
    pip install
    , 
    /hyperloom-setup
    → 
    .env
    (credentials + run mode only)
  • Phase 1 Environment — custom-advanced §Setup Configuration (baremetal: confirm host; docker: start container + setup inside, contract in setup.md)
  • Phase 2 Workload intake — custom-advanced §Advanced Configuration → Model Resolution → show launch plan → user confirms
  • Phase 3 Execute — install.sh → preflight → launch → monitor → report
Load 
hyperloom-custom-advanced
at Phase 1 and follow its sections in order (discovery: 
.cursor/
/ 
.claude/
/ 
.agents/skills/hyperloom-custom-advanced/SKILL.md
). If it is not on disk, stop and tell the user to restart the agent so the newly installed skills are picked up — do not improvise the environment or workload sections from memory, since the wheel is the source of truth for both. For deeper optimizer behavior read 
@${HYPERLOOM_SKILL_PATH}
(
inference_optimizer
); Iron Rules + CLI reference: reference.md.
匹配
hyperloom-custom-advanced
章节顺序 — 编写
.env
或执行
/hyperloom-setup
期间，不要询问工作负载相关问题。
  • Phase 0 引导 — 
    pip install
    、
    /hyperloom-setup
    → 生成
    .env
    （仅包含凭证和运行模式）
  • Phase 1 环境准备 — custom-advanced §Setup Configuration（裸金属：确认主机；Docker：启动容器并在内部完成配置，相关约定见setup.md）
  • Phase 2 工作负载收集 — custom-advanced §Advanced Configuration → 模型解析 → 展示启动计划 → 用户确认
  • Phase 3 执行 — install.sh → 预检 → 启动 → 监控 → 报告
在Phase 1加载
hyperloom-custom-advanced
并按其章节顺序执行（查找路径：
.cursor/
/ 
.claude/
/ 
.agents/skills/hyperloom-custom-advanced/SKILL.md
）。如果该文件不在磁盘上，请停止操作并告知用户重启智能体以加载新安装的技能 — 不要凭记忆临时编写环境或工作负载相关内容，因为安装包是这两部分内容的权威来源。如需了解优化器的更深层行为，请阅读
@${HYPERLOOM_SKILL_PATH}
（
inference_optimizer
）；核心规则与CLI参考见reference.md。

Iron Rules (launcher gates)

核心规则（启动管控）

Run order is always IR-2 → IR-1 → launch. Full text in reference.md.
  • IR-1 — GPU unoccupied. Before every 
    optimize
    (fresh or 
    --resume
    ), every visible GPU must have zero foreign serving PIDs (
    sglang.launch_server
    / 
    vllm.entrypoints
    / 
    Magpie
    ) and ≲ 500 MiB VRAM in use.
  • IR-2 — install.sh before launch. Run 
    install.sh
    and source 
    kernel-agent.env.sh
    in the same shell that spawns 
    optimize
    .
  • Resume carve-out: 
    --resume
    may skip install only when 
    install.sh
    exited 0 earlier in the same shell, 
    kernel-agent.env.sh
    is still sourced, and the session's 
    manifest.json
    exists. Any failure → re-run 
    install.sh
    .
运行顺序始终为 IR-2 → IR-1 → 启动。完整文本见reference.md。
  • IR-1 — GPU空闲。在每次执行
    optimize
    （全新运行或
    --resume
    恢复）之前，所有可见GPU必须没有外部服务进程（
    sglang.launch_server
    / 
    vllm.entrypoints
    / 
    Magpie
    ），且已使用VRAM ≲ 500 MiB。
  • IR-2 — 先运行install.sh再启动。在生成
    optimize
    进程的同一个Shell中运行
    install.sh
    并加载
    kernel-agent.env.sh
    。
  • 恢复运行例外：仅当以下条件全部满足时，
    --resume
    可跳过install步骤：此前在同一个Shell中
    install.sh
    已成功退出（返回码0）、
    kernel-agent.env.sh
    仍处于加载状态、且会话的
    manifest.json
    存在。任何失败情况 → 重新运行
    install.sh
    。

Phase discipline (do not skip)

阶段规范（请勿跳过）

One phase at a time. Each phase asks only its own questions, waits for the user's answers, completes its exit condition, then moves on. Never batch questions from different phases into one prompt. In particular, never ask workload questions (model, framework, TP/EP, precision, ISL/OSL, hours…) during Phase 0 or Phase 1 — those belong to Phase 2 only.
一次仅处理一个阶段。每个阶段仅询问自身相关问题，等待用户回答，完成退出条件后再进入下一阶段。切勿将不同阶段的问题批量放入同一个提示中。特别注意：在Phase 0或Phase 1期间，绝对不要询问工作负载相关问题（模型、框架、TP/EP、精度、ISL/OSL、时长…） — 这些问题仅属于Phase 2。

Phase 0 — Bootstrap

Phase 0 — 引导

Skip completed steps (idempotent). Ask only about the install directory and credentials/run mode here. Do not ask about the model or workload yet.
跳过已完成的步骤（操作具有幂等性）。此阶段仅询问安装目录和凭证/运行模式相关问题。暂时不要询问模型或工作负载相关内容。

Confirm the install directory

确认安装目录

The wheel installs into a target directory with 
pip install --target <dir>
, which also holds 
.env
and runtime artifacts. Do not silently use the current directory. Show the resolved current directory (
pwd
) and confirm it with the user, or let them choose another dedicated path. Wait for the answer, then 
cd
into the chosen directory before installing.
安装包将通过
pip install --target <dir>
安装到目标目录，该目录还将存放
.env
和运行时产物。请勿默认使用当前目录。显示当前解析后的目录（
pwd
）并让用户确认，或允许用户选择其他专用路径。等待用户回答后，进入选定目录再执行安装。

Install the Hyperloom wheel

安装Hyperloom安装包

Skip when 
hyperloom/
(wheel) or 
src/hyperloom/
(source) already exists in the confirmed directory.
The runtime is published to PyPI as 
hyperloom-inference-optimizer
. List the releases, tell the user the newest one, and ask whether to install it or a version they name.
List with 
--pre
so prereleases are visible, and install an exact 
==
version so a later bootstrap installs the same runtime.
bash
cd "$INSTALL_DIR"   # the directory confirmed above
pip index versions hyperloom-inference-optimizer --pre
pip install hyperloom-inference-optimizer==<version the user approved> --target .
Confirm 
hyperloom/inference_optimizer/assets/install.sh
exists. Restart the agent if wheel skills are not visible.
如果确认目录中已存在
hyperloom/
（安装包）或
src/hyperloom/
（源码），则跳过此步骤。
运行时已发布到PyPI，包名为
hyperloom-inference-optimizer
。列出所有版本，告知用户最新版本，并询问用户是安装最新版本还是指定版本。
使用
--pre
参数列出预发布版本，并安装精确的
==
版本，确保后续引导操作安装相同的运行时。
bash
cd "$INSTALL_DIR"   # 上述确认的目录
pip index versions hyperloom-inference-optimizer --pre
pip install hyperloom-inference-optimizer==<version the user approved> --target .
确认
hyperloom/inference_optimizer/assets/install.sh
存在。如果安装包技能不可见，请重启智能体。

Credentials and run mode

凭证与运行模式

Run 
/hyperloom-setup
(installed to 
.cursor/skills/hyperloom-setup/
). It writes 
.env
, sets 
USER_DATA_PATH
, 
HYPERLOOM_RUN_MODE
, and 
HYPERLOOM_SKILL_PATH
, and on bare metal runs 
install_baremetal.sh
.
Phase 0 is done when all hold:
  • hyperloom/inference_optimizer/assets/install.sh
    exists
  • .env
    exists with non-placeholder LLM secrets
  • USER_DATA_PATH
    , 
    HYPERLOOM_RUN_MODE
    , and 
    HYPERLOOM_SKILL_PATH
    are set
More bootstrap detail: setup.md.
运行
/hyperloom-setup
（安装到
.cursor/skills/hyperloom-setup/
）。该命令会写入
.env
，设置
USER_DATA_PATH
、
HYPERLOOM_RUN_MODE
和
HYPERLOOM_SKILL_PATH
，并在裸金属环境下运行
install_baremetal.sh
。
当以下条件全部满足时，Phase 0完成：
  • hyperloom/inference_optimizer/assets/install.sh
    存在
  • .env
    存在且包含非占位符的LLM密钥
  • USER_DATA_PATH
    、
    HYPERLOOM_RUN_MODE
    和
    HYPERLOOM_SKILL_PATH
    已设置
更多引导细节见setup.md。

Phase 1 — Environment prep

Phase 1 — 环境准备

Load 
hyperloom-custom-advanced
and follow its Setup Configuration section only.
Baremetal (
HYPERLOOM_RUN_MODE=baremetal
): confirm 
install_baremetal.sh
finished and the serving framework from setup is importable. Do not ask workload questions yet.
Docker (
HYPERLOOM_RUN_MODE=docker
): image choice, 
docker run
, and the in-container setup are owned entirely by custom-advanced Setup Configuration — follow it, do not restate its commands or flags here. Do not ask workload questions until the container is up and in-container setup succeeded, and never run 
optimize
on the host.
Phase 1 is done when the target environment (host or container) is ready.
加载
hyperloom-custom-advanced
并仅遵循其Setup Configuration章节。
**裸金属环境（
HYPERLOOM_RUN_MODE=baremetal
）：**确认
install_baremetal.sh
已完成，且设置中指定的服务框架可正常导入。暂时不要询问工作负载相关问题。
**Docker环境（
HYPERLOOM_RUN_MODE=docker
）：**镜像选择、
docker run
命令以及容器内的配置完全由custom-advanced的Setup Configuration章节负责 — 遵循该章节内容，不要在此处重复其命令或参数。在容器启动并完成内部配置之前，不要询问工作负载相关问题，且绝对不要在主机上运行
optimize
。
当目标环境（主机或容器）准备就绪时，Phase 1完成。

Phase 2 — Workload intake

Phase 2 — 工作负载收集

Enter only after Phase 0 and Phase 1 exit conditions hold. This is the first and only phase that asks workload questions.
Now follow custom-advanced Advanced Configuration, Default Values, and Model Resolution. Use the agent's structured question UI when available. Never copy API keys into chat output.
FieldCLI flagDefaultNotes
Model path
--model
requiredLocal dir with 
config.json
, or HF cache
Framework
--framework
sglang
or 
vllm
; prefer 
.env
FRAMEWORK
when set
TP / EP
--tp
/ 
--ep
1
/ 
1
tensor / expert parallel
CONC
--conc
64
client concurrency
ISL / OSL
--isl
/ 
--osl
1024
/ 
1024
input / output seq lengths
PRECISION
--precision
bf16
match checkpoint; 
fp8
for FP8 models
MAX_HOURS
--max-hours
CLI 
2.0
offer 
3
(quick) or 
12
(full); see below
TARGET_GAIN
--target-gain
30
desired % gain
Optional: 
--no-explore
, 
--no-enable-conc-sweep
, 
--gpu-type
, 
--server-args
, 
--compare-against-gpu
, 
--quantize
prelude.
Infer 
PRECISION
from the model name when obvious (e.g. an 
FP8
model implies 
--precision fp8
) and confirm it — do not silently keep the 
bf16
default.
仅在Phase 0和Phase 1的退出条件全部满足后进入此阶段。这是唯一询问工作负载相关问题的阶段。
现在遵循custom-advanced的Advanced Configuration、Default Values和Model Resolution章节。如果可用，请使用智能体的结构化问题UI。切勿在聊天输出中复制API密钥。
字段CLI参数默认值说明
模型路径
--model
必填包含
config.json
的本地目录，或HF缓存
框架
--framework
sglang
或
vllm
；优先使用
.env
中设置的
FRAMEWORK
TP / EP
--tp
/ 
--ep
1
/ 
1
张量并行 / 专家并行
并发数
--conc
64
客户端并发数
ISL / OSL
--isl
/ 
--osl
1024
/ 
1024
输入/输出序列长度
精度
--precision
bf16
与检查点匹配；FP8模型使用
fp8
最大时长
--max-hours
CLI默认
2.0
提供
3
（快速）或
12
（完整）选项；详见下文
目标提升率
--target-gain
30
期望的性能提升百分比
可选参数： 
--no-explore
、
--no-enable-conc-sweep
、
--gpu-type
、
--server-args
、
--compare-against-gpu
、
--quantize
预处理参数。
当模型名称明显暗示精度时（例如
FP8
模型意味着
--precision fp8
），自动推断
PRECISION
并确认 — 不要默认保留
bf16
。

Budget and flags — offer these three

预算与参数 — 提供以下三个选项

Offer all three and let the user pick one. The flags in each are a set: pass them together, and do not ask for a budget and then ask separately which phases to run. The two demos take the workload and flags of the Hyperloom demo skill of the same budget — treat those as given and skip the table above. The user may name their own model instead of the demo's; for the 3-hour demo keep it at 8B or below. Confirm everything in the launch plan. Only Custom collects workload answers.
1. 3-hour demo (
hyperloom-qwen3-8b-3h
) — 
Qwen/Qwen3-8B
unless the user names another 8B-or-smaller model, TP=1, CONC=64, ISL=OSL=1024, 
--precision bf16
, serving and config parameters only, no kernel rewrites. Resolve the model per custom-advanced Model Resolution; download it from Hugging Face when it is not already local. Match 
--precision
to the chosen checkpoint. Expect a modest validated gain, or an honest 0% when the workload has no parameter headroom.
text
--max-hours 3 --precision bf16
--no-framework-agent --no-kernel --no-enable-conc-sweep --no-enable-roofline
--max-minutes-explore-pct 0.39 --max-minutes-sweep-pct 0.01
--explore-force-exit-budget-pct 0.01 --explore-force-exit-hours-remaining 0.05
2. 12-hour demo (
hyperloom-qwen3-14b-fp8-12h
) — 
Qwen/Qwen3-14B-FP8
unless the user names another model, TP=1, CONC=64, ISL=OSL=1024, 
--precision fp8
matched to the chosen checkpoint, every lever with kernel rewrites included. The kernel agent needs room to profile, rewrite and revalidate, which is where the larger gains come from.
text
--max-hours 12 --precision fp8
--max-minutes-framework-pct 0.01 --max-minutes-explore-pct 0.42
--max-minutes-kernel-pct 0.42
3. Custom — the user brings their own model or workload instead of taking a demo. Walk through the fields in the table above and the phase toggles, one question at a time, and derive the flags from the answers rather than asking for flags. Whichever levers they pick, a budget of 3 hours or less keeps the 3-hour demo's flag set. Optional flags come from the list above; show the full flag list in the launch plan either way.
提供全部三个选项让用户选择。每个选项的参数是一组：需同时传递，不要先询问预算再单独询问要运行哪些阶段。两个演示选项使用对应预算的Hyperloom演示技能的工作负载和参数 — 视为既定内容，跳过上述表格。用户可指定自己的模型而非演示模型；对于3小时演示，模型需为8B或更小规模。确认启动计划中的所有内容。仅自定义选项需要收集工作负载相关回答。
1. 3小时演示（
hyperloom-qwen3-8b-3h
） — 默认使用
Qwen/Qwen3-8B
，除非用户指定其他8B或更小规模的模型，TP=1，CONC=64，ISL=OSL=1024，
--precision bf16
，仅调整服务和配置参数，不进行内核重写。根据custom-advanced的Model Resolution章节解析模型；如果模型未在本地存在，则从Hugging Face下载。确保
--precision
与所选检查点匹配。预期获得适度的验证性能提升，若工作负载无参数优化空间则可能为0%。
text
--max-hours 3 --precision bf16
--no-framework-agent --no-kernel --no-enable-conc-sweep --no-enable-roofline
--max-minutes-explore-pct 0.39 --max-minutes-sweep-pct 0.01
--explore-force-exit-budget-pct 0.01 --explore-force-exit-hours-remaining 0.05
2. 12小时演示（
hyperloom-qwen3-14b-fp8-12h
） — 默认使用
Qwen/Qwen3-14B-FP8
，除非用户指定其他模型，TP=1，CONC=64，ISL=OSL=1024，
--precision fp8
与所选检查点匹配，包含所有优化维度（包括内核重写）。内核智能体需要足够时间进行分析、重写和重新验证，这也是获得较大性能提升的关键。
text
--max-hours 12 --precision fp8
--max-minutes-framework-pct 0.01 --max-minutes-explore-pct 0.42
--max-minutes-kernel-pct 0.42
3. 自定义 — 用户提供自己的模型或工作负载，而非使用演示内容。逐一询问上述表格中的字段和阶段开关，根据回答生成参数，而非直接询问参数。无论用户选择哪些优化维度，若预算为3小时或更短，则使用3小时演示的参数集。可选参数来自上述列表；无论哪种情况，都需在启动计划中显示完整的参数列表。

Confirmation gate (required before Phase 3)

确认环节（Phase 3前必填）

The Coordinator has no in-loop 
setup
/ 
classify
— a value not asked here is silently lost to its default. Before running any Phase 3 command, present the full launch plan (including defaulted fields) and get explicit user confirmation.
Print the plan in the reply body as this aligned block:
text
Launch plan — please confirm:
  MODEL_PATH    /wekafs/models/Qwen3-14B-FP8
  FRAMEWORK     vllm
  TP=1  EP=1  CONC=64
  ISL=1024  OSL=1024
  PRECISION=fp8
  MAX_HOURS=3     TARGET_GAIN=20%
  profile       3-hour demo — no kernel, no framework agent, no roofline
  flags         --no-framework-agent --no-kernel --no-enable-conc-sweep
                --no-enable-roofline
                --max-minutes-explore-pct 0.39 --max-minutes-sweep-pct 0.01
                --explore-force-exit-budget-pct 0.01
                --explore-force-exit-hours-remaining 0.05
  RUN_MODE      baremetal
Never put the plan inside the confirmation prompt itself. A prompt renders as one wrapped paragraph, which collapses the alignment above into an unreadable blob the user has to search for 
MAX_HOURS
in. Keep the prompt to a single short question such as 
Approve this launch plan?
, and if you offer a "change something" option, name the field to change rather than making the user retype it as free text.
Do not run 
install.sh
or launch 
optimize
until the user approves this plan.
协调器没有循环内的
setup
/
classify
功能 — 此处未询问的值将默认丢失。在运行任何Phase 3命令之前，展示完整的启动计划（包括默认字段）并获得用户明确确认。
按以下对齐格式在回复正文中打印计划：
text
启动计划 — 请确认：
  MODEL_PATH    /wekafs/models/Qwen3-14B-FP8
  FRAMEWORK     vllm
  TP=1  EP=1  CONC=64
  ISL=1024  OSL=1024
  PRECISION=fp8
  MAX_HOURS=3     TARGET_GAIN=20%
  profile       3小时演示 — 无内核优化，无框架智能体，无roofline分析
  flags         --no-framework-agent --no-kernel --no-enable-conc-sweep
                --no-enable-roofline
                --max-minutes-explore-pct 0.39 --max-minutes-sweep-pct 0.01
                --explore-force-exit-budget-pct 0.01
                --explore-force-exit-hours-remaining 0.05
  RUN_MODE      baremetal
切勿将计划放入确认提示本身。提示会渲染为一个换行的段落，导致上述对齐格式变成难以阅读的一团，用户需要在其中查找
MAX_HOURS
等字段。提示应仅包含一个简短问题，例如
是否批准此启动计划？
，如果提供“修改内容”选项，请明确指出要修改的字段，而非让用户自由输入文本。
在用户批准此计划之前，不要运行
install.sh
或启动
optimize
。

Persist the plan (required — shells do not share exports)

持久化计划（必填 — Shell不共享环境变量）

Agent shells do not persist exports between calls, so write the confirmed values to 
$RUN_DIR/workload.env
right after approval. Every Phase 3 block sources it; without this, launch silently falls back to 
${TP:-1}
/ 
${CONC:-64}
defaults and 
--model ""
. Fill each value from the approved plan.
bash
export USER_DATA_PATH="${USER_DATA_PATH:?run /hyperloom-setup first}"
export RUN_DIR="${USER_DATA_PATH}/optimizer_runs"
mkdir -p "$RUN_DIR"
智能体Shell不会在多次调用之间保留环境变量，因此在获得批准后立即将确认的值写入
$RUN_DIR/workload.env
。后续每个Phase 3步骤都会加载该文件；如果不执行此操作，启动会默认回退到
${TP:-1}
/
${CONC:-64}
等默认值，且
--model
为空。根据批准的计划填充每个值。
bash
export USER_DATA_PATH="${USER_DATA_PATH:?run /hyperloom-setup first}"
export RUN_DIR="${USER_DATA_PATH}/optimizer_runs"
mkdir -p "$RUN_DIR"

Quoted heredoc (<<'EOF'): values are written literally, so a MODEL_PATH with

带引号的here文档（<<'EOF'）：值将按字面写入，因此包含空格、$或$(...)的MODEL_PATH不会被展开或执行。根据计划修改每个值。

spaces, $, or $(...) is not expanded or executed. Edit each value to the plan.

—
cat > "$RUN_DIR/workload.env" <<'EOF' export MODEL_PATH=/wekafs/models/Qwen3-14B-FP8 export FRAMEWORK=vllm export TP=1 export EP=1 export CONC=64 export ISL=1024 export OSL=1024 export PRECISION=fp8 export MAX_HOURS=3 export TARGET_GAIN=20
cat > "$RUN_DIR/workload.env" <<'EOF' export MODEL_PATH=/wekafs/models/Qwen3-14B-FP8 export FRAMEWORK=vllm export TP=1 export EP=1 export CONC=64 export ISL=1024 export OSL=1024 export PRECISION=fp8 export MAX_HOURS=3 export TARGET_GAIN=20

The whole flag set for the approved profile, space-separated. The 3-hour

批准的配置对应的完整参数集，空格分隔。此处展示的是3小时演示的参数；12小时运行需替换为对应的参数集。

demo is shown; a 12-hour run swaps in its own set.

—
export OPT_FLAGS="--no-framework-agent --no-kernel --no-enable-conc-sweep --no-enable-roofline --max-minutes-explore-pct 0.39 --max-minutes-sweep-pct 0.01 --explore-force-exit-budget-pct 0.01 --explore-force-exit-hours-remaining 0.05" EOF
undefined
export OPT_FLAGS="--no-framework-agent --no-kernel --no-enable-conc-sweep --no-enable-roofline --max-minutes-explore-pct 0.39 --max-minutes-sweep-pct 0.01 --explore-force-exit-budget-pct 0.01 --explore-force-exit-hours-remaining 0.05" EOF
undefined

Phase 3 — Install (IR-2)

Phase 3 — 安装（IR-2）

INSTALL_DIR
is the directory confirmed in Phase 0, the one holding 
hyperloom/
and 
.env
. Every Phase 3 block below rebuilds it from the current directory, so run them from there; the check refuses a directory that is not it.
Resolve paths for wheel or source layout:
bash
export INSTALL_DIR="$(pwd -P)"
[ -d "${INSTALL_DIR}/hyperloom" ] || [ -d "${INSTALL_DIR}/src/hyperloom" ] || {
  echo "ERROR: ${INSTALL_DIR} holds no hyperloom/ -- cd to the Phase 0 install directory" >&2; exit 1; }
set -a; . "${INSTALL_DIR}/.env"; set +a
export USER_DATA_PATH="${USER_DATA_PATH:?USER_DATA_PATH missing}"
. "${USER_DATA_PATH}/optimizer_runs/workload.env"   # confirmed Phase 2 values
export PYTHONPATH="${INSTALL_DIR}:${PYTHONPATH:-}"
ulimit -Sn 65536 || true

INSTALL_SH="${INSTALL_DIR}/hyperloom/inference_optimizer/assets/install.sh"
[ -f "$INSTALL_SH" ] || INSTALL_SH="${INSTALL_DIR}/src/hyperloom/inference_optimizer/assets/install.sh"

bash "$INSTALL_SH"
. "${KERNEL_AGENT_ENV:-${USER_DATA_PATH}/runtime/kernel-agent.env.sh}"
export PYTHONPATH="${INSTALL_DIR}:${PYTHONPATH:-}"
In Docker mode, run this inside the container.
INSTALL_DIR
是Phase 0中确认的目录，即存放
hyperloom/
和
.env
的目录。以下每个Phase 3步骤都会从当前目录重新构建该路径，因此需从该目录运行；检查会拒绝非指定目录。
解析安装包或源码布局的路径：
bash
export INSTALL_DIR="$(pwd -P)"
[ -d "${INSTALL_DIR}/hyperloom" ] || [ -d "${INSTALL_DIR}/src/hyperloom" ] || {
  echo "ERROR: ${INSTALL_DIR} holds no hyperloom/ -- cd to the Phase 0 install directory" >&2; exit 1; }
set -a; . "${INSTALL_DIR}/.env"; set +a
export USER_DATA_PATH="${USER_DATA_PATH:?USER_DATA_PATH missing}"
. "${USER_DATA_PATH}/optimizer_runs/workload.env"   # Phase 2确认的值
export PYTHONPATH="${INSTALL_DIR}:${PYTHONPATH:-}"
ulimit -Sn 65536 || true

INSTALL_SH="${INSTALL_DIR}/hyperloom/inference_optimizer/assets/install.sh"
[ -f "$INSTALL_SH" ] || INSTALL_SH="${INSTALL_DIR}/src/hyperloom/inference_optimizer/assets/install.sh"

bash "$INSTALL_SH"
. "${KERNEL_AGENT_ENV:-${USER_DATA_PATH}/runtime/kernel-agent.env.sh}"
export PYTHONPATH="${INSTALL_DIR}:${PYTHONPATH:-}"
在Docker模式下，需在容器内部运行此步骤。

Phase 3 — Preflight (IR-1)

Phase 3 — 预检（IR-1）

install.sh
exports 
$PYTHON
; the fallback below covers agent sandboxes that do not persist exports between shell calls.
bash
export SKILL_DIR="${SKILL_DIR:?absolute path of the directory holding this SKILL.md}"
. "${USER_DATA_PATH}/optimizer_runs/workload.env"   # confirmed Phase 2 values
export PYTHON="${PYTHON:-$(command -v python3)}"
"$PYTHON" "${SKILL_DIR}/scripts/preflight.py"
The gate exits non-zero — do not launch — when 
MODEL_PATH
is missing or has no 
config.json
, torch sees no GPU, a foreign serving process still holds a card, or any GPU holds more than 
IR1_VRAM_LIMIT_MIB
(default 500) MiB.
It also blocks when VRAM cannot be read at all: no 
amd-smi
/
rocm-smi
on 
PATH
, a probe that exits non-zero, or output it cannot parse. An unreadable probe cannot rule out a busy GPU, and a foreign process holding VRAM under a different name would slip through. Confirm the GPUs are idle by hand before re-running with 
IR1_ALLOW_UNVERIFIED_VRAM=1
.
Never print API keys or tokens. 
scripts/tests/test_preflight.py
covers the probe shapes this gate must reject.
install.sh
会导出
$PYTHON
；以下回退方案适用于不会在Shell调用之间保留导出变量的智能体沙箱。
bash
export SKILL_DIR="${SKILL_DIR:?absolute path of the directory holding this SKILL.md}"
. "${USER_DATA_PATH}/optimizer_runs/workload.env"   # Phase 2确认的值
export PYTHON="${PYTHON:-$(command -v python3)}"
"$PYTHON" "${SKILL_DIR}/scripts/preflight.py"
当以下情况发生时，该管控环节会返回非零值 — 不要启动：
MODEL_PATH
缺失或无
config.json
、torch未检测到GPU、外部服务进程仍占用GPU、或任何GPU已使用VRAM超过
IR1_VRAM_LIMIT_MIB
（默认500）MiB。
当完全无法读取VRAM时也会阻止启动：
PATH
中无
amd-smi
/
rocm-smi
、探测命令返回非零值、或输出无法解析。无法读取探测结果则无法排除GPU繁忙的情况，且外部进程可能以不同名称占用VRAM而未被检测到。在使用
IR1_ALLOW_UNVERIFIED_VRAM=1
重新运行之前，请手动确认GPU处于空闲状态。
切勿打印API密钥或令牌。
scripts/tests/test_preflight.py
涵盖了此管控环节必须拒绝的探测结果类型。

Phase 3 — Launch

Phase 3 — 启动

After IR-2 and IR-1 pass, launch. 
setsid nohup
is required for runs longer than 5 minutes, so the run outlives the agent shell.
bash
export INSTALL_DIR="$(pwd -P)"
export SKILL_DIR="${SKILL_DIR:?absolute path of the directory holding this SKILL.md}"
bash "${SKILL_DIR}/scripts/launch.sh"
Every workload value comes from the confirmed 
workload.env
; the script has no 
${VAR:-default}
fallbacks, so a missing value fails loudly instead of launching a different config. Put any optional Phase 2 flags (
--no-kernel
, 
--no-explore
, 
--gpu-type
, 
--model-class
, 
--server-args
, 
--compare-against-gpu
, 
--quantize
, phase budget flags) into 
OPT_FLAGS
in 
workload.env
. 
OPT_FLAGS
is word-split, so quote any flag value that contains spaces, e.g. 
export OPT_FLAGS='--server-args "--foo bar"'
.
在IR-2和IR-1通过后，启动优化器。对于时长超过5分钟的运行，必须使用
setsid nohup
，确保运行进程不会随智能体Shell结束而终止。
bash
export INSTALL_DIR="$(pwd -P)"
export SKILL_DIR="${SKILL_DIR:?absolute path of the directory holding this SKILL.md}"
bash "${SKILL_DIR}/scripts/launch.sh"
所有工作负载值均来自已确认的
workload.env
；脚本没有
${VAR:-default}
回退，因此缺失的值会导致启动失败，而非使用不同配置启动。将Phase 2中的任何可选参数（
--no-kernel
、
--no-explore
、
--gpu-type
、
--model-class
、
--server-args
、
--compare-against-gpu
、
--quantize
、阶段预算参数）放入
workload.env
的
OPT_FLAGS
中。
OPT_FLAGS
会按空格拆分，因此包含空格的参数值需加引号，例如
export OPT_FLAGS='--server-args "--foo bar"'
。

Launch health check (30 s after start)

启动健康检查（启动后30秒）

Required after every launch and resume. The PID recorded at launch is the setsid wrapper, which exits immediately — it is NOT the optimizer. This reads the real 
.pid
and 
.session_dir
from the launch-info JSON, rewrites the PID file so the monitor watches the right process, and records both in 
$RUN_DIR/last_launch.env
for the later phases.
bash
export INSTALL_DIR="$(pwd -P)"
export SKILL_DIR="${SKILL_DIR:?absolute path of the directory holding this SKILL.md}"
bash "${SKILL_DIR}/scripts/launch_health.sh"
It exits non-zero when the launch-info JSON never appeared, no optimizer process can be found, or 
session_dir
is still unset — inspect the reported run log in those cases. Never guess 
session_dir
from a timestamp; concurrent sessions share 
USER_DATA_PATH
.
每次启动和恢复运行后都必须执行此步骤。启动时记录的PID是setsid包装进程，该进程会立即退出 — 它不是优化器进程。此步骤会从启动信息JSON中读取真实的
.pid
和
.session_dir
，重写PID文件以便监控进程能跟踪正确的进程，并将两者记录到
$RUN_DIR/last_launch.env
供后续阶段使用。
bash
export INSTALL_DIR="$(pwd -P)"
export SKILL_DIR="${SKILL_DIR:?absolute path of the directory holding this SKILL.md}"
bash "${SKILL_DIR}/scripts/launch_health.sh"
当启动信息JSON从未生成、无法找到优化器进程、或
session_dir
仍未设置时，此步骤会返回非零值 — 此时需检查报告的运行日志。切勿根据时间戳猜测
session_dir
；并发会话共享
USER_DATA_PATH
。

Phase 3 — Monitor

Phase 3 — 监控

Poll at most every 5 minutes unless debugging a startup failure. Use the state reader the wheel ships rather than parsing 
state.json
by hand — it also prints the recent lifecycle events.
bash
export INSTALL_DIR="$(pwd -P)"
. "${USER_DATA_PATH}/optimizer_runs/last_launch.env"   # SESSION_DIR from launch
STATE_TOOL="${INSTALL_DIR}/hyperloom/inference_optimizer/tools/read_optimizer_state.py"
[ -f "$STATE_TOOL" ] || STATE_TOOL="${INSTALL_DIR}/src/hyperloom/inference_optimizer/tools/read_optimizer_state.py"
"${PYTHON:-python3}" "$STATE_TOOL" "$SESSION_DIR"
For recent action counts grouped by category, the wheel also ships 
tools/event_counts.py
, invoked the same way.
Report session id + log path, 
baseline_tput
/ 
current_best
/ 
cumulative_gain
, explore accepted/rejected, last kernel opt (correctness, speedup, KEEP/REVERT), and process-alive vs 
stop_reason
. See reference.md Report fields.
除非调试启动失败，否则最多每5分钟轮询一次。使用安装包附带的状态读取工具，而非手动解析
state.json
— 该工具还会打印最近的生命周期事件。
bash
export INSTALL_DIR="$(pwd -P)"
. "${USER_DATA_PATH}/optimizer_runs/last_launch.env"   # 启动时的SESSION_DIR
STATE_TOOL="${INSTALL_DIR}/hyperloom/inference_optimizer/tools/read_optimizer_state.py"
[ -f "$STATE_TOOL" ] || STATE_TOOL="${INSTALL_DIR}/src/hyperloom/inference_optimizer/tools/read_optimizer_state.py"
"${PYTHON:-python3}" "$STATE_TOOL" "$SESSION_DIR"
如需按类别查看最近的操作计数，安装包还附带了
tools/event_counts.py
，调用方式相同。
报告会话ID + 日志路径、
baseline_tput
/
current_best
/
cumulative_gain
、探索阶段的接受/拒绝情况、最近的内核优化（正确性、加速比、KEEP/REVERT），以及进程存活状态与
stop_reason
。详见reference.md中的报告字段。

Resume

恢复运行

Resume runs in a fresh shell. Re-run the IR-2 and IR-1 gates first, exactly as for a fresh launch — the script does not re-check them.
bash
export INSTALL_DIR="$(pwd -P)"
export SKILL_DIR="${SKILL_DIR:?absolute path of the directory holding this SKILL.md}"
bash "${SKILL_DIR}/scripts/resume.sh"
bash "${SKILL_DIR}/scripts/launch_health.sh"
It resumes the session recorded in 
last_launch.env
and always passes 
--resume-from
explicitly, because a bare 
--resume
auto-picks the newest session and can target the wrong run. Resume writes its own log (
run_resume-*.log
) so the original run log is preserved. Reuse the IR-2 carve-out rules; re-run 
install.sh
if the shell or env changed.
stop_reason
Action
time_exhausted
--resume
same session
no_more_leverage
stop; resume only if user changes strategy
policy_loop
inspect 
policy_denial_history
; clear stale prunes
在新的Shell中恢复运行。首先重新执行IR-2和IR-1管控环节，与全新启动完全一致 — 脚本不会自动重新检查这些条件。
bash
export INSTALL_DIR="$(pwd -P)"
export SKILL_DIR="${SKILL_DIR:?absolute path of the directory holding this SKILL.md}"
bash "${SKILL_DIR}/scripts/resume.sh"
bash "${SKILL_DIR}/scripts/launch_health.sh"
它会恢复
last_launch.env
中记录的会话，并始终显式传递
--resume-from
参数，因为仅使用
--resume
会自动选择最新会话，可能会定位到错误的运行。恢复运行会写入自己的日志（
run_resume-*.log
），以保留原始运行日志。遵循IR-2的例外规则；如果Shell或环境发生变化，需重新运行
install.sh
。
stop_reason
操作
time_exhausted
--resume
恢复同一会话
no_more_leverage
停止；仅当用户更改策略时才恢复
policy_loop
检查
policy_denial_history
；清除过时的修剪规则

Expected optimizer flow

预期优化器流程

  1. Establish 
    baseline_tput
    .
  2. Coordinator runs roofline/profile analysis after baseline.
  3. explore
    tests serving parameters incrementally.
  4. Kernel-agent runs on hot paths with compile + correctness evidence.
  5. sweep
    validates concurrency around the best candidate.
  6. Final report under 
    $SESSION_DIR/reports/
    .
  1. 建立
    baseline_tput
    基准吞吐量。
  2. 基准测试后，协调器运行roofline/性能分析。
  3. explore
    阶段逐步测试服务参数。
  4. 内核智能体针对热点路径运行编译 + 正确性验证。
  5. sweep
    阶段围绕最优候选方案验证并发数。
  6. 最终报告存放在
    $SESSION_DIR/reports/
    下。

When to defer

何时转交其他技能

  • Plain serving only — use 
    serving-llms-on-instinct
    .
  • ROCm driver broken — diagnose the ROCm stack first (e.g. a 
    rocm-doctor
    skill if published); do not start the optimizer on a broken driver.
  • Edge cases — read 
    @${HYPERLOOM_SKILL_PATH}
    for multi-node, atom framework (IR-8), critic/robustness backends, cache topology, and the full failure matrix.
  • 仅需普通服务 — 使用
    serving-llms-on-instinct
    技能。
  • ROCm驱动损坏 — 先诊断ROCm栈（例如已发布的
    rocm-doctor
    技能）；不要在驱动损坏的情况下启动优化器。
  • 边缘情况 — 阅读
    @${HYPERLOOM_SKILL_PATH}
    了解多节点、atom框架（IR-8）、评审/鲁棒性后端、缓存拓扑以及完整的故障矩阵。

Further reading

扩展阅读

  • Bootstrap detail: setup.md
  • Iron Rules + CLI reference: reference.md
  • Authoritative runtime skill: 
    hyperloom/inference_optimizer/SKILL.md
  • 引导细节：setup.md
  • 核心规则 + CLI参考：reference.md
  • 权威运行时技能：
    hyperloom/inference_optimizer/SKILL.md