python3 scripts/detect.py            # add --host user@box for a remote host

python3 scripts/validate.py --image <image from data/epyc.json> --generation <epyc_generation from detect>

python3 scripts/check_model.py --model-id <model> --revision <rev or main> --vllm-version <stack.vllm from validate, else vllm_version from data/epyc.json>

python3 scripts/estimate_memory.py --model-id <model> --revision <rev or main> --ram-gb <memory_gb from detect> --max-model-len <4096 or user value> --num-prompts <1 or desired concurrency>

eval "$(python3 scripts/cpu_tune.py)"      # or --format json to inspect

RT="<runtime from validate.py: docker | podman>"
$RT rm -f vllm-epyc 2>/dev/null               # clear any leftover container from a prior run (name collision otherwise)
$RT pull <image from data/epyc.json>          # agent pulls; do not ask the user to
$RT run -d --name vllm-epyc \
  <run_flags from data/epyc.json>            # --ipc=host --network=host (NO --shm-size: it conflicts with --ipc=host on podman)
  <hf_cache_mount> \
  <container_cpuset from cpu_tune>             # --cpuset-cpus=<cores> --cpuset-mems=<nodes>
  --env VLLM_CPU_OMP_THREADS_BIND="$VLLM_CPU_OMP_THREADS_BIND" \
  --env VLLM_CPU_KVCACHE_SPACE=$VLLM_CPU_KVCACHE_SPACE \
  --env HF_TOKEN=${HF_TOKEN} \
  <image from data/epyc.json> \
  vllm serve <model> --dtype bfloat16 --port <port> --max-model-len <len>

<conda_launch_prefix from cpu_tune> vllm serve <model> --dtype bfloat16 --port <port> --max-model-len <len> &

Optional throughput flags are **opt-in and must move together** (see Gotchas):
`TORCHINDUCTOR_FREEZING=1` + `VLLM_USE_AOT_COMPILE=0` (+ `ZENTORCH_WEIGHT_PREPACK=1`).
The base launch sets none of them.

undefined

Then exercise the endpoint the model actually supports. Use deterministic
sampling and a small output cap for the smoke check:

```bash

Confirm the response is JSON with a non-error `choices[0]` (chat: `message.content`;
completion: `text`). An HTTP 200 that carries an `error` payload is **not** success.
Resource sanity (your validation list): `$RT stats --no-stream vllm-epyc`.

**If the server never becomes healthy, `/v1/models` omits the model, or the
endpoint returns an error/empty `choices`: print the container/process logs,
state the failing phase, and STOP. Do not retry. Do not start a debugging loop.**

curl -s http://localhost:<port>/v1/chat/completions -H 'Content-Type: application/json' \
  -d '{"model":"<served-model>","messages":[{"role":"user","content":"Hello"}],"max_tokens":128,"temperature":0.7}'

curl -s http://localhost:<port>/v1/completions -H 'Content-Type: application/json' \
  -d '{"model":"<served-model>","prompt":"Hello, world","max_tokens":128,"temperature":0.7}'

from openai import OpenAI

client = OpenAI(base_url="http://localhost:<port>/v1", api_key="EMPTY")
model = client.models.list().data[0].id

Argument guidance to pass along (see [reference.md](reference.md) for the full list):
- `max_tokens` caps the **output**; `prompt_tokens + max_tokens` must be `<= --max-model-len`.
- `temperature` (0 = deterministic/greedy, higher = more random); tune `top_p` *or*
  `temperature`, not both.
- `stream: true` streams tokens (SSE) instead of one blocking response.
- The model's `generation_config.json` can set sampling defaults; pass explicit
  values to be sure.