Validate a Workflow
Purpose
Run the selected workflow run mode through the unified launcher, inspect the completed recording, and report simulator success.
Instructions
- Resolve the base checkout and a live workflow run mode.
- Run the unified launcher in the foreground.
- Require the final episode success summary.
- Inspect visible behavior and every rollout artifact.
Resolve live support
bash
export I4H_WORKFLOWS_REPO_URL="${I4H_WORKFLOWS_REPO_URL:-https://github.com/isaac-for-healthcare/i4h-workflows}"
I4H_REPO_DIR_NAME="${I4H_WORKFLOWS_REPO_URL%/}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME##*/}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME##*:}"
I4H_REPO_DIR_NAME="${I4H_REPO_DIR_NAME%.git}"
[ -n "$I4H_REPO_DIR_NAME" ] || { echo "Cannot derive a checkout name from I4H_WORKFLOWS_REPO_URL" >&2; exit 2; }
ROOT="${I4H_WORKFLOWS:-$(git rev-parse --show-toplevel 2>/dev/null)}"
if [ ! -d "$ROOT/workflows/i4h_workflows" ]; then
ROOT="${I4H_WORKFLOWS:-$HOME/$I4H_REPO_DIR_NAME}"
[ -d "$ROOT/workflows/i4h_workflows" ] || git clone "$I4H_WORKFLOWS_REPO_URL" "$ROOT"
fi
export I4H_WORKFLOWS="$ROOT"
cd "$ROOT"
./run.sh list
Treat the resolver above as part of the skill contract: a hosted copy may run outside the base repository, so never assume the current checkout contains
.
selects the clone source. When
is unset, derive the fallback directory from that URL; set
only to reuse or choose a specific destination. Never replace an existing checkout.
Treat
output as the complete authoritative workflow-by-mode table; it is dependency-light and faster than scanning workflow modules. Do not duplicate that mutable table in this skill. Map the user's natural name to a listed workflow id, then choose only a mode shown on that same line:
| User intent | Required live mode | Launcher argument |
|---|
| Ordinary learned-policy evaluation | | |
| Requested or only available local controller | | |
| Explicit named alternative such as N1.7 | matching listed mode such as | |
Inspect
./run.sh show <workflow> --mode <mode>
, the workflow module, Scene manifest, and selected Task manifest when model, prompt, checkpoint, goal, or step-cap behavior matters.
Use precise readiness language:
- Structurally valid: , per-mode lint, and pass.
- Launchable: the selected simulator mode starts and every required backend/checkpoint preloads.
- Rollout-validated: the requested episodes complete and the recorded success evidence passes inspection.
Do not report “validated” without stating which level was actually reached.
Foreground execution rule
Keep
as this agent's foreground tool call. Do not use a subagent, monitor task, shell backgrounding,
,
, or a detached process. Poll a yielded session until exit and inspect the final episode summary before responding. When the selected Task is remote, the policy backend subprocess internally owned by
is expected; a simulator-compatible exported RSL-RL Task runs in-process.
Run visibly by default. If the user explicitly requests headless execution, or a documented environment constraint makes it necessary, say so before launch and include
; never switch to headless silently.
Policy:
bash
./run.sh <workflow> --policy \
--episodes <N> --attempts 3 \
--record verify.hdf5
Rule-based:
bash
./run.sh <workflow> --rule-based \
--episodes <N> --attempts 3 \
--record verify.hdf5
The launcher creates a unique canonical run directory and anchors the relative
inside it. Resolve
from the
line or machine-readable
; do not recreate the launcher's timestamp. Use
only when a caller-selected location must be shared with another stage; the launcher creates it. Absolute recording paths remain supported.
For another declared mode, use
. For a supplied/new checkpoint, resolve its exact path, confirm it belongs to the selected policy Task, and pass
--checkpoint /absolute/path
; an exported RSL-RL Task expects its TorchScript
, while a remote Task expects the owning backend's loadable checkpoint format. For “300 timesteps,” pass
.
Never raise the Scene manifest's cap.
may only lower it. Remote inference waits do not consume simulation steps. Use a unique
when another run of the same workflow is active.
Verify
Require exit status 0 and final
. A failed attempt followed by a successful retry counts as a successful requested episode; report attempts and retries.
bash
RUN_DIR="<absolute run_dir from run.json or launcher output>"
uv run --project tools/dataset i4h-dataset inspect "$RUN_DIR/verify.hdf5" --segments
Inspect episode metadata, action/state widths, camera frames, node segments, and success flags. For visible runs, observe Scene/camera behavior and final task outcome. On failure, use the first actionable backend, contract, graph, or simulator error; never switch modes or increase the cap silently. Run
after crashes that leave processes.
For a success rule that excludes collision, inspect the owning contact-sensor configuration and require a forced-contact negative test at least once after authoring or changing that rule. A sensor that initializes but has never produced a non-zero filtered force does not establish collision rejection.
Troubleshooting
Use the first actionable backend, graph, contract, or simulator error. Retry within the same mode and cap only after correcting it.
Prerequisites
Require synced simulator assets and any backend/checkpoint declared by the selected run mode.
Limitations
Only modes from
are supported, and a runtime step override may lower but never raise the validated Scene cap.
Examples
Evaluate scissor pick and place for 2 episodes.
Run surgical_reach_psm in rule-based mode for 1 episode.
Completion gate
Report workflow, mode, model/checkpoint source, requested successes, attempts/retries, completion steps, visible outcome, HDF5 path and inspection, final exit status, and first unresolved failure if any.