nemo-rl-brev-etiquette

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Brev Etiquette

Brev操作规范

Operate as though

/home/ubuntu/RL

is the source checkout and

/ephemeral

is the working storage for generated experiment state. Keep the repo small, reproducible, and easy to inspect. Move bulky run outputs to

/ephemeral

before launching anything expensive.

将

/home/ubuntu/RL

视为源代码检出目录，

/ephemeral

作为生成实验状态的工作存储目录。保持仓库体积小巧、可复现且易于检查。在启动任何资源密集型任务前，将庞大的运行输出移至

/ephemeral

。

Storage Rules

存储规则

Keep code edits, small config changes, committed experiment hypotheses, and concise reproducibility records under
```
/home/ubuntu/RL
```
.
Put generated experiment assets under
```
/ephemeral
```
, including checkpoints, run logs, Ray temp directories, W&B offline files, profiler traces, evaluation dumps, rollout samples, and per-experiment artifacts.
Keep reusable caches under one shared
```
/ephemeral
```
cache root per user, not under each experiment. This includes Hugging Face models, dataset caches, PyTorch caches, Triton caches,
```
uv
```
caches, and pip caches.
Before a campaign or long run, check capacity with
```
df -h /home/ubuntu/RL /ephemeral
```
and avoid starting if
```
/ephemeral
```
is missing or nearly full.
Create a campaign root such as
```
/ephemeral/nemo-rl/${USER:-ubuntu}/nemo-rl-auto-research/<campaign>
```
and use one subdirectory per experiment.
Do not leave large files, cache directories, or generated outputs in the git checkout. If a tool defaults to the repo, override its output/cache path before running it.

将代码编辑、小型配置修改、已提交的实验假设和简洁的可复现记录保存在
```
/home/ubuntu/RL
```
下。
将生成的实验资产放在
```
/ephemeral
```
下，包括检查点、运行日志、Ray临时目录、W&B离线文件、分析器追踪数据、评估输出、滚动样本和每个实验的工件。
将可复用缓存放在每个用户专属的共享
```
/ephemeral
```
缓存根目录下，而非每个实验目录下。这包括Hugging Face模型、数据集缓存、PyTorch缓存、Triton缓存、
```
uv
```
缓存和pip缓存。
在启动任务或长时间运行前，使用
```
df -h /home/ubuntu/RL /ephemeral
```
检查容量，若
```
/ephemeral
```
缺失或接近满容量则避免启动。
创建任务根目录，例如
```
/ephemeral/nemo-rl/${USER:-ubuntu}/nemo-rl-auto-research/<campaign>
```
，并为每个实验创建一个子目录。
不要在git检出目录中遗留大文件、缓存目录或生成的输出。如果工具默认将输出保存到仓库，运行前需覆盖其输出/缓存路径。

Environment Secrets

环境密钥

Treat
```
/home/ubuntu/RL/.env
```
as the local secret store. It may contain keys such as
```
WANDB_API_KEY
```
,
```
HF_TOKEN
```
, or
```
HUGGING_FACE_HUB_TOKEN
```
.
Before any run that may need external auth, load
```
/home/ubuntu/RL/.env
```
when it exists. Never print,
```
cat
```
, log, commit, or summarize secret values.
If
```
/home/ubuntu/RL/.env
```
is absent, or a required key is still unset after loading it, remind the user to add the needed key to that file before launching authenticated work.

bash

if [ -f /home/ubuntu/RL/.env ]; then
  set -a
  . /home/ubuntu/RL/.env
  set +a
else
  echo "Missing /home/ubuntu/RL/.env; add required keys such as WANDB_API_KEY or HF_TOKEN before authenticated runs."
fi

将

/home/ubuntu/RL/.env

视为本地密钥存储。它可能包含

WANDB_API_KEY

、

HF_TOKEN

或

HUGGING_FACE_HUB_TOKEN

等密钥。

在任何可能需要外部认证的运行前，若
```
/home/ubuntu/RL/.env
```
存在则加载该文件。绝不要打印、
```
cat
```
查看、记录、提交或总结密钥值。
若
```
/home/ubuntu/RL/.env
```
不存在，或加载后仍缺少所需密钥，请提醒用户在启动认证工作流前将所需密钥添加到该文件中。

bash

if [ -f /home/ubuntu/RL/.env ]; then
  set -a
  . /home/ubuntu/RL/.env
  set +a
else
  echo "Missing /home/ubuntu/RL/.env; add required keys such as WANDB_API_KEY or HF_TOKEN before authenticated runs."
fi

Auto-Research Pattern

自动研究模式

When using

nemo-rl-auto-research

, keep the git ledger in the repo and heavy evidence on

/ephemeral

bash

if [ -f /home/ubuntu/RL/.env ]; then
  set -a
  . /home/ubuntu/RL/.env
  set +a
fi

BREV_ROOT=/ephemeral/nemo-rl/${USER:-ubuntu}
CACHE_ROOT=$BREV_ROOT/cache
CAMPAIGN_ROOT=$BREV_ROOT/nemo-rl-auto-research/<campaign>
EXP_DIR=$CAMPAIGN_ROOT/<experiment>
mkdir -p "$EXP_DIR"/{logs,checkpoints,artifacts,ray,tmp,wandb}
mkdir -p "$CACHE_ROOT"/{huggingface,torch,triton,uv,pip,xdg,wandb}

export HF_HOME=$CACHE_ROOT/huggingface
export HF_HUB_CACHE=$HF_HOME/hub
export HF_DATASETS_CACHE=$HF_HOME/datasets
export TRANSFORMERS_CACHE=$HF_HOME/transformers
export TORCH_HOME=$CACHE_ROOT/torch
export TRITON_CACHE_DIR=$CACHE_ROOT/triton
export UV_CACHE_DIR=$CACHE_ROOT/uv
export PIP_CACHE_DIR=$CACHE_ROOT/pip
export XDG_CACHE_HOME=$CACHE_ROOT/xdg
export WANDB_CACHE_DIR=$CACHE_ROOT/wandb
export RAY_TMPDIR=$EXP_DIR/ray
export TMPDIR=$EXP_DIR/tmp
export WANDB_DIR=$EXP_DIR/wandb

Record the absolute

/ephemeral

paths in the nemo-rl-auto-research TSV fields for log path, checkpoint path, artifacts, shared cache root, and command. If the TSV itself may grow large, store the full TSV in

/ephemeral

and keep a small pointer file or summary in the repo.

使用

nemo-rl-auto-research

时，将git记录保存在仓库中，将占用大量空间的实验数据存储在

/ephemeral

上。

bash

if [ -f /home/ubuntu/RL/.env ]; then
  set -a
  . /home/ubuntu/RL/.env
  set +a
fi

BREV_ROOT=/ephemeral/nemo-rl/${USER:-ubuntu}
CACHE_ROOT=$BREV_ROOT/cache
CAMPAIGN_ROOT=$BREV_ROOT/nemo-rl-auto-research/<campaign>
EXP_DIR=$CAMPAIGN_ROOT/<experiment>
mkdir -p "$EXP_DIR"/{logs,checkpoints,artifacts,ray,tmp,wandb}
mkdir -p "$CACHE_ROOT"/{huggingface,torch,triton,uv,pip,xdg,wandb}

export HF_HOME=$CACHE_ROOT/huggingface
export HF_HUB_CACHE=$HF_HOME/hub
export HF_DATASETS_CACHE=$HF_HOME/datasets
export TRANSFORMERS_CACHE=$HF_HOME/transformers
export TORCH_HOME=$CACHE_ROOT/torch
export TRITON_CACHE_DIR=$CACHE_ROOT/triton
export UV_CACHE_DIR=$CACHE_ROOT/uv
export PIP_CACHE_DIR=$CACHE_ROOT/pip
export XDG_CACHE_HOME=$CACHE_ROOT/xdg
export WANDB_CACHE_DIR=$CACHE_ROOT/wandb
export RAY_TMPDIR=$EXP_DIR/ray
export TMPDIR=$EXP_DIR/tmp
export WANDB_DIR=$EXP_DIR/wandb

在nemo-rl-auto-research的TSV字段中记录

/ephemeral

的绝对路径，包括日志路径、检查点路径、工件、共享缓存根目录和命令。如果TSV本身可能变得很大，将完整TSV存储在

/ephemeral

中，并在仓库中保留一个小型指针文件或摘要。

Launch Checklist

启动检查清单

Inspect disk first:
```
df -h /home/ubuntu/RL /ephemeral
```
.
Choose a unique
```
/ephemeral
```
run root before editing recipes or launching jobs.
Reuse a shared cache root such as
```
/ephemeral/nemo-rl/${USER:-ubuntu}/cache
```
across experiments unless a run explicitly requires a clean cache.
Override recipe output paths, logger paths, checkpoint paths, and temp paths to point under the experiment directory.
Override cache paths to point under the shared cache root.
Stream stdout/stderr to
```
$EXP_DIR/logs/run.log
```
or an equivalent file under
```
/ephemeral
```
.
Periodically check disk during long runs with
```
df -h /ephemeral
```
and stop gracefully if the volume is approaching exhaustion.
At the end, summarize the important metrics and paths in the repo ledger; do not copy bulky artifacts back into
```
/home/ubuntu/RL
```
.

首先检查磁盘：
```
df -h /home/ubuntu/RL /ephemeral
```
。
在编辑脚本或启动作业前，选择一个唯一的
```
/ephemeral
```
运行根目录。
在多个实验间复用共享缓存根目录（如
```
/ephemeral/nemo-rl/${USER:-ubuntu}/cache
```
），除非运行明确要求使用干净缓存。
将脚本输出路径、日志记录器路径、检查点路径和临时路径覆盖为指向实验目录下的路径。
将缓存路径覆盖为指向共享缓存根目录下的路径。
将stdout/stderr流输出到
```
$EXP_DIR/logs/run.log
```
或
```
/ephemeral
```
下的等效文件。
在长时间运行期间定期使用
```
df -h /ephemeral
```
检查磁盘，若卷即将耗尽则优雅停止运行。
运行结束后，在仓库记录中总结重要指标和路径；不要将庞大的工件复制回
```
/home/ubuntu/RL
```
。

Cleanup

清理步骤

Clean only files that belong to the current campaign or experiment.
Prefer pruning clearly named experiment directories under
```
/ephemeral/nemo-rl/...
```
; never remove shared caches or another user's run directory without an explicit instruction.
Preserve enough small metadata in the repo to reproduce a result after
```
/ephemeral
```
is cleaned.

仅清理属于当前任务或实验的文件。
优先删除
```
/ephemeral/nemo-rl/...
```
下命名清晰的实验目录；未经明确指示，绝不要删除共享缓存或其他用户的运行目录。
在仓库中保留足够的小型元数据，以便在
```
/ephemeral
```
被清理后仍能复现结果。