serverless-deploy

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Serverless deploy

Serverless部署

You take any deployable source and turn it into a working, queue-backed Serverless endpoint that you have proven with one real job, and you leave the user knowing exactly what now exists and what it bills. The steps never change: read the source, decide the engine and the hardware, provision, warm, smoke-test one real job, hand back the URL and a

curl

, state what is billing. Tear down only when the user framed the deploy as a throwaway test or asks for cleanup.

Two decisions are yours to make without being told: the engine family (which worker image is correct for this modality and model format) and the provisioning path (bake weights into the image vs download-on-start vs stage on a network volume, each with a different cold-start cost). Size hardware from the model's real metadata, never a guess. When several suitable workers exist for the modality, prefer the smallest, most-proven image for a smoke-able deploy. A giant image turns the first job into a cold start of many minutes, and the smoke test's job is proof of the journey, not maximal quality. Name the trade-off if the user's request implies a heavier engine. Creating an endpoint is billable: state the hourly cost of the GPU class before you create it, and remember a smoke job costs money.

For code-first Python (writing endpoint functions in a project and iterating with Runpod's Python dev loop, runpod-flash), this MCP-only bundle has no path. Say so instead of improvising one. This skill deploys an existing source (Hub repo, container image, ComfyUI workflow, Hub release) to an endpoint through the MCP tools.

你可以将任意可部署来源转化为可运行的、带队列支持的Serverless端点，并通过一个真实任务验证其可用性，让用户清楚了解已部署的资源及对应的计费情况。流程固定不变：读取来源信息、确定引擎和硬件配置、资源调配、预热、执行一次真实任务的冒烟测试、返回URL和

curl

示例、说明计费项。仅当用户明确表示本次部署为一次性测试或要求清理时，才拆除相关资源。

你需要自主做出两个决策：引擎系列（哪种worker镜像适配当前的任务模态和模型格式）以及部署路径（权重预打包进镜像 vs 启动时下载 vs 暂存到网络卷，每种方式的冷启动成本不同）。必须根据模型的真实元数据确定硬件规格，严禁猜测。当同一模态有多个适用的worker镜像时，优先选择最小、最成熟的镜像来完成可冒烟验证的部署。过大的镜像会导致首次任务的冷启动长达数分钟，而冒烟测试的目的是验证部署流程的可行性，而非追求最高质量。如果用户的需求暗示需要使用更重的引擎，需明确说明其中的权衡。创建端点会产生费用：在创建前必须说明对应GPU规格的小时费用，且冒烟测试本身也会产生费用。

对于代码优先的Python部署（在项目中编写端点函数，并通过Runpod的Python开发循环runpod-flash进行迭代），本纯MCP工具包不支持该模式。请直接说明，不要临时拼凑方案。本技能仅通过MCP工具将现有来源（Hub仓库、容器镜像、ComfyUI工作流、Hub发布版）部署到端点。

Required capabilities

所需能力

Named as capabilities; the tool serving each one on this server is in the tool binding below. This skill reads the catalog before it creates, and creates only what the user asked to deploy.

Read GPU catalog + price + live stock, per-data-center availability, and the CUDA versions that have capacity: sizing and stock (delegate the sizing math to
```
discovery
```
).
Browse the Hub worker catalog, with the release config, to find a prebuilt worker or a community template and read its hardware/env schema.
Deploy a Hub release directly as an endpoint: the one-call Hub path.
Create a template (image, disk, ports, env, registry, mounts): the reusable worker preset.
Create a Serverless endpoint (GPU pools, worker min/max, scaler, data centers, network volumes).
Create a network volume to stage large weights in a GPU-co-located data center.
Pin GPUs / set GPU count per worker:
```
gpu.count
```
and the pool plus
```
excludedTypes
```
selection;
```
set-endpoint-gpus
```
builds the exclusion list and preserves the other settings.
Read endpoint health and list workers: warmup and readiness, and the cold-start-vs-crash-loop evidence together with the worker logs (a crash-looping container can still report RUNNING or INITIALIZING).
Submit a job and block on it: the smoke test (
```
run-endpoint
```
, then
```
get-job-status wait:300000
```
on the job id until terminal;
```
runsync-endpoint
```
only for a warm endpoint and a fast job, since its result is kept for one minute).
Read a worker's logs when that tool is available to you: model-load / crash evidence. When it is not,
```
endpoint-health
```
+
```
list-endpoint-workers
```
give the worker-health counts, which narrow the cause but cannot settle a crash loop on their own.
Delete the endpoint, template, and any network volume: teardown of everything created.

以下为各项能力；本服务器上对应每项能力的工具见下方的工具绑定表。本技能在创建资源前会先读取目录，且仅创建用户明确要求部署的内容。

读取GPU目录、价格、实时库存、各数据中心可用性，以及有容量的CUDA版本：用于规格确定和库存校验（规格计算工作委托给
```
discovery
```
）。
浏览Hub worker目录及发布配置，查找预构建的worker或社区模板，并读取其硬件/环境配置 schema。
直接将Hub发布版部署为端点：单步调用的Hub部署路径。
创建模板（镜像、磁盘、端口、环境变量、镜像仓库、挂载）：可复用的worker预设配置。
创建Serverless端点（GPU资源池、worker最小/最大数量、扩缩容器、数据中心、网络卷）。
创建网络卷，用于在GPU同驻地数据中心暂存大体积权重文件。
固定GPU型号 / 设置每个worker的GPU数量：通过
```
gpu.count
```
、资源池及
```
excludedTypes
```
选择实现；
```
set-endpoint-gpus
```
工具会生成排除列表并保留其他配置不变。
读取端点健康状态并列出worker：包括预热和就绪情况，以及冷启动与崩溃循环的判断依据和worker日志（处于崩溃循环的容器仍可能报告状态为RUNNING或INITIALIZING）。
提交任务并等待其完成：即冒烟测试（调用
```
run-endpoint
```
，然后对任务ID执行
```
get-job-status wait:300000
```
直到任务进入终态；
```
runsync-endpoint
```
仅适用于已预热的端点和快速任务，因为其结果仅保留一分钟）。
在你有权限使用对应工具时读取worker日志：获取模型加载/崩溃的证据。如果没有该工具，可通过
```
endpoint-health
```
+
```
list-endpoint-workers
```
获取worker健康统计，这能缩小原因范围，但无法单独定位崩溃循环问题。
删除端点、模板及所有相关网络卷：拆除所有已创建的资源。

Phase 0 — identify the source and pick the engine (unprompted decision)

阶段0 — 识别来源并选择引擎（自主决策）

Read the source the user gave you and classify it. The engine family follows from the modality and the model format, not from a default. Never serve a non-text-generation task on a vLLM or ComfyUI worker.

Source / modality	Correct engine family	Worker image kind
Text-generation LLM, `safetensors` weights	vLLM or SGLang	`runpod/worker-v1-vllm` (or SGLang worker)
Text-generation LLM, `GGUF` quantized weights	llama.cpp / Ollama	a GGUF-capable worker — not vLLM
Image/diffusion `workflow.json`	ComfyUI	`runpod/worker-comfyui` (reference/comfyui-serverless.md)
Speech-to-text (transcription)	a Whisper-family model	a speech-to-text worker, not an LLM worker
Text-to-speech / voice cloning	a TTS model	a TTS worker
Speaker diarization	a diarization pipeline	a diarization worker from the Hub or public catalog
OCR / document extraction	a classic OCR engine	an OCR worker, not a VLM chat worker
Text embeddings	an embeddings server (e.g. text-embeddings-inference)	an embeddings worker
A custom handler you were given	your own image	a custom container image

Format detection. Inspect the repo's files before choosing.
```
*.safetensors
```
/
```
*.bin
```
→ vLLM/SGLang.
```
*.gguf
```
→ a llama.cpp/Ollama worker. Deploying a GGUF repo on worker-vllm is a common wrong-engine mistake: vLLM will not load it. State the format you detected and the engine it implies in the final message; the detection is part of the answer, not only the decision.
OCR sizing. For an OCR/document-extraction smoke deploy, default to a classic lightweight OCR worker (16–24 GB). A VLM-based OCR (7B+ params) is a big-model deploy: size it like one (48 GB class) or don't pick it for a smoke test.
Community-template source. When the modality has no first-party Runpod worker, browse the Hub / public catalog for a community (non-Runpod) template that serves it. Name the repo and its owner from the listing, never assert a repo from memory. Once you have found a fitting worker, deploy it and run one real job. Do not stop at "I found X," and do not block on a credential the config makes optional (see the runtime-optional-token rule in phase 2); request the capability through the job input the worker's config documents. Only if the catalog returns nothing that fits do you say "none found" and stop; never invent a repo name or a Docker image.
Custom image. When the user brings their own handler image, that image is the engine. Deploy exactly the image named, never a substitute. When its required env is missing, apply the model-default rule in phase 2 rather than asking; the observed worker states are the report, not a question back to the user. After the deploy, read the worker states (
```
list-endpoint-workers
```
or
```
endpoint-health
```
) and report what you observed. For a custom image the workers' observed health IS the deliverable: one completed job does not show a crash-looping sibling worker, so report a crash-looping custom image as failing. If it is in a private registry, the endpoint needs a registry credential (create/reference it; see
```
lifecycle-crud
```
); never echo the registry password. For a load-balancer (custom-HTTP, direct-routing) endpoint rather than the queue, the cardinal rule is that the worker must serve HTTP on
```
PORT
```
(default 80) AND answer the health check at
```
HEALTH_CHECK_PATH
```
(default
```
/ping
```
) on
```
PORT_HEALTH
```
(default 80). Miss either and workers never pass health, so the endpoint looks stuck with zero ready workers; the endpoint read returns the exact health URL being polled. The endpoint type is also immutable after create, and a load-balancer endpoint has no queue: when every worker is busy, requests fail instead of waiting, so size
```
workers.max
```
for the peak.

读取用户提供的来源并进行分类。引擎系列由任务模态和模型格式决定，而非使用默认值。严禁将非文本生成任务部署到vLLM或ComfyUI worker上。

来源 / 任务模态	正确的引擎系列	Worker镜像类型
文本生成LLM， `safetensors` 权重	vLLM 或 SGLang	`runpod/worker-v1-vllm` （或SGLang worker）
文本生成LLM， `GGUF` 量化权重	llama.cpp / Ollama	支持GGUF的worker —— 不能是vLLM
图像/扩散模型 `workflow.json`	ComfyUI	`runpod/worker-comfyui` （reference/comfyui-serverless.md）
语音转文本（转录）	Whisper系列模型	语音转文本worker，而非LLM worker
文本转语音 / 声音克隆	TTS模型	TTS worker
说话人分离	说话人分离流水线	来自Hub或公共目录的说话人分离worker
OCR / 文档提取	经典OCR引擎	OCR worker，而非VLM聊天worker
文本嵌入	嵌入服务（例如text-embeddings-inference）	嵌入worker
用户提供的自定义handler	用户自有镜像	自定义容器镜像

格式检测。选择引擎前先检查仓库中的文件。
```
*.safetensors
```
/
```
*.bin
```
→ vLLM/SGLang。
```
*.gguf
```
→ llama.cpp/Ollama worker。将GGUF仓库部署到worker-vllm上是常见的引擎选择错误：vLLM无法加载GGUF格式。在最终回复中说明你检测到的格式及其对应的引擎；检测过程本身也是答案的一部分，而非仅给出决策结果。
OCR规格配置。对于OCR/文档提取的冒烟部署，默认使用经典的轻量级OCR worker（16–24 GB）。基于VLM的OCR（7B+参数）属于大模型部署：需按大模型配置规格（48 GB级），否则不要在冒烟测试中选用。
社区模板来源。如果某一模态没有Runpod官方worker，可浏览Hub/公共目录寻找提供该能力的**社区（非Runpod）**模板。需明确列出仓库及其所有者的名称（从目录列表中获取），严禁凭记忆编造仓库信息。找到合适的worker后，必须完成部署并运行一个真实任务。不要仅停留在“我找到了X”的阶段，也不要因为配置中标记为可选的凭证而阻塞流程（参见阶段2中的运行时可选token规则）；通过worker配置文档中说明的任务输入参数来请求对应能力。仅当目录中没有找到任何适配项时，才说明“未找到”并停止；严禁编造仓库名或Docker镜像。
自定义镜像。如果用户提供了自己的handler镜像，该镜像就是引擎。必须严格部署指定的镜像，严禁替换。如果缺少所需的环境变量，应应用阶段2中的模型默认规则，而非询问用户；将观察到的worker状态作为报告内容，而不是反过来向用户提问。部署完成后，读取worker状态（
```
list-endpoint-workers
```
或
```
endpoint-health
```
）并报告观察结果。对于自定义镜像，worker的实际健康状态就是交付成果：单个成功完成的任务无法反映其他worker是否处于崩溃循环，因此如果自定义镜像出现崩溃循环，应报告为部署失败。如果镜像位于私有仓库，端点需要配置镜像仓库凭证（创建/引用凭证，参见
```
lifecycle-crud
```
）；严禁回显镜像仓库密码。对于使用负载均衡器（custom-HTTP、direct-routing）而非队列的端点，核心规则是worker必须在
```
PORT
```
（默认80）上提供HTTP服务，并且必须在
```
PORT_HEALTH
```
（默认80）的
```
HEALTH_CHECK_PATH
```
（默认
```
/ping
```
）路径上响应健康检查。缺少任意一项，worker都无法通过健康检查，导致端点看起来一直卡在零就绪worker的状态；端点读取接口会返回当前正在轮询的准确健康检查URL。端点类型在创建后也是不可变的，且负载均衡器端点没有队列：当所有worker都繁忙时，请求会直接失败而非排队等待，因此需根据峰值负载配置
```
workers.max
```
。

Phase 1 — size the hardware (read the catalog, cite

discovery

)

阶段1 — 确定硬件规格（读取目录，引用

discovery

结果）

Delegate the params×precision + KV/overhead VRAM math to

discovery

; this skill applies its result to a real GPU pick.

Read the model's real metadata (parameter count, precision, and for diffusion the checkpoint/LoRA/VAE asset sizes). Do not size from the repo name alone when the config is readable.
Pinning to one card. When the user wants the endpoint locked to an exact GPU, select the pool that contains the card and list the pool's other GPU types in
```
gpu.excludedTypes
```
(
```
gpu.pools
```
takes pool ids, not GPU type ids, and at least one type must remain), either on the create or with
```
set-endpoint-gpus
```
afterwards. The read-back quotes pools and exclusions to prove the pin, and every other setting is confirmed unchanged from the read.
Pick the smallest GPU class whose VRAM fits, then confirm live stock from the GPU-availability read.
```
include:["AVAILABILITY"]
```
with
```
product:["SERVERLESS"]
```
(the two go together, and stock differs by product) also gives the per-data-center picture when the endpoint needs a specific data center, and the same read's
```
cudaVersions
```
shows which CUDA versions have capacity when a CUDA floor is in play. Pin that floor on the create:
```
gpu.minCudaVersion
```
at the image's CUDA line (12.8 for a CUDA 12 image; 13.0 only for an image built on CUDA 13), or
```
gpu.allowedCudaVersions
```
for an exact set; the two are mutually exclusive (400 if both are sent), and the floor must have stock in
```
cudaVersions
```
. Never create into a GPU class with no stock: a scale-to-zero endpoint that can never acquire a worker looks deployed but never runs.
Multi-GPU when one card can't hold the model. When required VRAM exceeds the largest single card you can get with stock, set GPUs-per-worker > 1 and enable tensor parallelism in the worker env (e.g. the vLLM worker's tensor-parallel setting = GPU count). State that you are going multi-GPU and why (the model does not fit on one card), and that multi-GPU workers cost proportionally more per hour. Before committing, price the alternative in the answer: quote the largest in-stock single card (its VRAM, live stock, $/hr) against the multi-GPU pick.
Pin the pool on Hub deploys.
```
deploy-hub-repo
```
inherits the release's GPU pool list when
```
gpuIds
```
is omitted, which can silently acquire a top-tier card for a task a 24 GB card serves, at several times the hourly rate. Pass
```
gpuIds
```
from YOUR sizing decision on every Hub deploy unless the release config pins exactly the pool your sizing chose.
CPU serverless.
```
create-endpoint
```
takes one hardware choice: the
```
gpu
```
object (pools and count) or the
```
cpu
```
array of CPU flavor configurations, sized from the CPU catalog read (
```
list-cpu-types
```
). For a CPU workload send
```
cpu
```
and no
```
gpu
```
(never force a GPU pool onto a CPU workload), and quote the CPU flavor's hourly price before creating.
Set the defaults you mean. A REST create defaults to FlashBoot
```
OFF
```
,
```
workers.max
```
3,
```
workers.idleTimeout
```
10 s and
```
timeout
```
300 s; set
```
flashboot
```
and the worker bounds explicitly instead of inheriting them. The sum of
```
workers.max
```
across the account's endpoints is capped by a balance-tiered quota (5 to start), checked at create, so a second endpoint at the default of 3 can be refused.
State the cost. Before creating, quote the chosen GPU's hourly rate (×GPU count) from the catalog read. Creating the endpoint and running the smoke test both bill.

将“参数量×精度 + KV缓存/开销”的VRAM计算工作委托给

discovery

；本技能会将其计算结果应用于实际的GPU选型。

读取模型的真实元数据（参数量、精度，对于扩散模型还包括checkpoint/LoRA/VAE资源大小）。当配置文件可读时，严禁仅根据仓库名估算规格。
固定单卡型号。如果用户要求端点锁定到特定GPU型号，需选择包含该型号的资源池，并在
```
gpu.excludedTypes
```
中列出该资源池内的其他GPU型号（
```
gpu.pools
```
接收的是资源池ID而非GPU型号ID，且至少要保留一个可用型号），可在创建时配置或之后通过
```
set-endpoint-gpus
```
设置。通过回读结果中的资源池和排除列表来证明固定配置生效，同时确认所有其他配置与读取结果一致未被修改。
选择VRAM容量满足要求的最小GPU规格，然后通过GPU可用性读取结果确认实时库存。使用
```
include:["AVAILABILITY"]
```
和
```
product:["SERVERLESS"]
```
（这两个参数需配合使用，且库存因产品类型而异），当端点需要部署在特定数据中心时，还能获取各数据中心的库存情况；当有最低CUDA版本要求时，同一次读取的
```
cudaVersions
```
字段会显示哪些CUDA版本有可用容量。在创建端点时设置最低版本要求：根据镜像的CUDA版本设置
```
gpu.minCudaVersion
```
（CUDA 12的镜像设为12.8；仅当镜像基于CUDA 13构建时才设为13.0），或通过
```
gpu.allowedCudaVersions
```
指定精确的版本集合；这两个参数互斥（同时发送会返回400错误），且指定的最低版本必须在
```
cudaVersions
```
中有库存。严禁在无库存的GPU规格上创建端点：无法获取worker的缩容到零端点看起来已部署，但永远无法运行任务。
单卡无法容纳模型时使用多GPU。当所需VRAM超过当前有库存的最大单卡容量时，将每个worker的GPU数量设为大于1，并在worker环境变量中启用张量并行（例如vLLM worker的tensor-parallel设置等于GPU数量）。需明确说明将使用多GPU配置及其原因（模型无法在单卡上容纳），以及多GPU worker的小时费用会按比例增加。在确认配置前，需在回复中给出替代方案的价格对比：列出当前有库存的最大单卡的VRAM、实时库存、小时费用，与多GPU方案进行对比。
Hub部署时固定资源池。如果省略
```
gpuIds
```
参数，
```
deploy-hub-repo
```
会继承发布版的GPU资源池列表，这可能导致原本用24 GB卡就能运行的任务被分配到顶级显卡，小时费用高出数倍。每次Hub部署都应传入你根据规格计算确定的
```
gpuIds
```
，除非发布版配置恰好固定了你所选的资源池。
CPU Serverless。
```
create-endpoint
```
仅接受一种硬件配置：要么是
```
gpu
```
对象（资源池和数量），要么是CPU规格配置数组
```
cpu
```
，需根据CPU目录读取结果（
```
list-cpu-types
```
）确定规格。对于CPU工作负载，仅传入
```
cpu
```
参数，不要传
```
gpu
```
（严禁将GPU资源池强制用于CPU工作负载），并在创建前说明对应CPU规格的小时费用。
明确设置默认值。REST创建接口的默认配置为：FlashBoot关闭、
```
workers.max
```
为3、
```
workers.idleTimeout
```
为10秒、
```
timeout
```
为300秒；应显式设置
```
flashboot
```
和worker数量边界，而非继承默认值。账户下所有端点的
```
workers.max
```
总和受余额分层配额限制（初始为5），创建时会校验，因此第二个默认配置为3的端点可能会被拒绝。
说明费用。创建前，根据目录读取结果报出所选GPU的小时费率（×GPU数量）。创建端点和运行冒烟测试都会产生费用。

Phase 2 — provision

阶段2 — 资源部署

Two provisioning paths; choose by source.

Hub release. If the source is a Hub repo, deploy its listed release directly (one call). Read the release config first (hardware requirements + env schema) so you pass only env keys the release accepts, and so you know its GPU/disk defaults. To reproduce a known-good build, pin the exact release rather than "latest".
Everything else. Create a template that names the engine image, the disk size, the ports the worker exposes, and the env (model id, engine settings, tokens); then create the endpoint referencing it. Start scale-to-zero (min workers 0) unless the user asked for warm capacity; a nonzero minimum is an always-on billed worker. An image hosted on
```
registry.runpod.net
```
is a Hub build: deploy it through
```
deploy-hub-repo
```
(the release carries the registry auth); a plain template with that image loops on a registry-auth failure.
Provisioning path and cold start. Bake-into-image = fastest cold start, biggest image; download-on-start = small image, slow first request per new worker; network volume = weights staged once in a GPU-co-located data center and mounted by every worker (writable, so coordinate concurrent writes), so new workers skip the download. Choose a network volume when weights are large (roughly > 30 GB) or shared across workers. Size it to the real asset total with headroom, create it in the same data center you place the endpoint in (see
```
lifecycle-crud
```
), and say which path you took and its cold-start consequence.
Multi-region / HA (no auto-sync). For a multi-region endpoint you attach one network volume per data center. These per-DC volumes do not auto-sync: you replicate data across them yourself (e.g. an
```
aws s3
```
sync between the volumes' S3 endpoints), or each region serves stale or missing weights.
Required env, gated weights, and runtime-optional tokens. Read the release/worker config (
```
includeConfig: true
```
) and SET every required env key that has no default before you deploy (e.g. an embeddings worker's model-id env such as
```
MODEL_NAMES
```
). Deployed without its model env, a worker loads nothing and 400s every job. When the user named no model for a text-generation worker (a vLLM image with no
```
MODEL_NAME
```
), pick the worker's documented default model when its config has one, otherwise the smallest instruct model that fits the card (a 1–2B instruct model on a 16 GB class), state the choice, and deploy. Do not ask which model. For a gated HuggingFace repo whose weights 403 without a token at download time, pass the token in env, and if the user gave none, say the download will 403 and ask. Do not deploy a build that cannot fetch its weights. A token needed only for an optional runtime capability, one whose config default is empty (a worker whose optional feature needs a token), branches on one check you make FIRST: did the user provide a token (pasted in the request, set in the environment, or named as available)?
- Token provided → using it is the job. Set it as the worker env var at create (the token env the worker documents), say that you configured it (by env-var name, never the value), and expect the gated capability to work. Never deploy without wiring a credential the user handed you.
- No token → deploy anyway. The deploy is NOT conditional on the token: don't park the whole deploy on an optional credential. Deploy, run the job, report what the endpoint returned, and then name the token as the unlock for the gated capability. Never print a token value; reference it as an env var.

有两种部署路径，根据来源类型选择。

Hub发布版。如果来源是Hub仓库，直接部署其列出的发布版（单步调用）。需先读取发布版配置（硬件要求 + 环境变量schema），这样你只会传入发布版支持的环境变量键，也能了解其默认的GPU/磁盘配置。要复现已知可用的构建，需固定到具体的发布版本，而非使用“latest”。
其他所有来源。创建一个模板，指定引擎镜像、磁盘大小、worker暴露的端口以及环境变量（模型ID、引擎设置、token）；然后引用该模板创建端点。除非用户要求预热容量，否则默认以缩容到零模式启动（worker最小数量为0）；非零的最小数量意味着worker会持续运行并计费。托管在
```
registry.runpod.net
```
上的镜像是Hub构建产物：应通过
```
deploy-hub-repo
```
部署（发布版自带镜像仓库认证）；直接用该镜像创建普通模板会因镜像仓库认证失败而循环报错。
部署路径与冷启动。预打包进镜像 = 冷启动最快，镜像体积最大；启动时下载 = 镜像体积小，每个新worker的首次请求慢；网络卷 = 权重文件在GPU同驻地数据中心暂存一次，所有worker都挂载使用（可写，因此需协调并发写入），新worker无需下载权重。当权重文件较大（约>30 GB）或需要在多个worker间共享时，选择网络卷方案。根据实际资源总大小预留余量来配置卷容量，将其创建在与端点相同的数据中心（参见
```
lifecycle-crud
```
），并说明你选择的部署路径及其对冷启动的影响。
多区域 / 高可用（无自动同步）。对于多区域端点，每个数据中心都需要挂载一个网络卷。这些跨数据中心的卷不会自动同步：你需要自行在卷之间复制数据（例如通过
```
aws s3
```
在各卷的S3端点之间同步），否则不同区域可能会提供过期或缺失的权重。
必填环境变量、受限权重与运行时可选token。读取发布版/worker配置（
```
includeConfig: true
```
），并在部署前设置所有无默认值的必填环境变量（例如嵌入worker的模型ID环境变量，如
```
MODEL_NAMES
```
）。如果部署时没有配置模型相关环境变量，worker不会加载任何模型，所有任务都会返回400错误。如果用户没有为文本生成worker指定模型（vLLM镜像没有设置
```
MODEL_NAME
```
），当配置中有记录的默认模型时选择该默认模型，否则选择能适配当前显卡的最小指令微调模型（16 GB级显卡选1–2B参数的指令模型），说明选择后再部署。不要询问用户要选哪个模型。对于受限的HuggingFace仓库，如果没有token下载权重时会返回403，需在环境变量中传入token；如果用户没有提供，需说明下载会返回403并向用户索要。不要部署无法获取权重的构建。对于仅用于可选运行时能力的token（即配置默认值为空，worker的可选功能需要该token），需首先检查一个条件：用户是否提供了token（粘贴在请求中、设置在环境里，或说明有可用token）？
- 已提供token → 必须使用。在创建时将其设置为worker的环境变量（即worker文档中说明的token环境变量名），说明你已配置该变量（仅说变量名，绝不泄露值），并预期受限功能可以正常工作。严禁在部署时忽略用户提供的凭证。
- 未提供token → 仍继续部署。部署不以token为前提：不要因为一个可选凭证而暂停整个部署。完成部署、运行任务、报告端点返回结果，然后说明该token是解锁受限能力的关键。严禁打印token值；仅以环境变量名引用。

Phase 3 — warm and smoke-test one real job

阶段3 — 预热并执行一次真实任务的冒烟测试

Prove the endpoint before you call it deployed: block on the smoke job with server-side holds instead of handing the user a poll loop; the tools can carry the wait. Report what you observed once the job is terminal; if waiting stops being productive, report the evidenced state instead.

Submit, then wait server-side — do not tight-loop. Fire a single small job (the smallest real input for the modality) with
```
run-endpoint
```
, then hold on it with
```
get-job-status
```
and
```
wait: 300000
```
, the maximum server-side hold. A cold first job outlives
```
runsync
```
's hold and its result is kept for only a minute, so
```
runsync-endpoint
```
is for a warm endpoint and a fast job. The point is to prove the worker boots with the real asset set and returns the right shape of output, not only that the endpoint exists.
Expect a cold start of 1–5+ minutes on the first job — a fresh worker pulls the image and loads the model. This is normal, not a failure. If the job outlives a hold, keep blocking with
```
get-job-status
```
using
```
wait: 300000
```
(one server-side hold, not a rapid re-poll). Poll the job id you already have; do not re-submit, which starts a new job. Between holds check
```
list-endpoint-workers
```
:
```
unhealthy
```
workers, a job still queued beside ready workers, or zero capacity → stop waiting and read the logs (see below). Three server-side holds per smoke job, counted: at most three
```
get-job-status
```
calls with
```
wait: 300000
```
on the same job id. After the third hold the evidenced report is the answer: no fourth hold and no tight poll. A job that outlives three holds is reported, not waited out. When continued waiting stops being informative, stop and report the state with the worker-state evidence: endpoint created correctly, job submitted, the worker never became ready, here is what the worker states show. That evidenced report is a complete answer; an endless grind is not.
Diagnose a stuck job from the reads you always have:
endpoint-health
,
list-endpoint-workers
, and the
workerHealth
block
get-job-status
attaches on
IN_QUEUE
.
```
initializing
```
workers on the first job = the cold start may still be in progress → keep waiting, but read the worker logs (
```
source=system
```
) before the next hold: a container that crash-loops on an image pull or a CUDA mismatch can still report
```
INITIALIZING
```
or
```
RUNNING
```
while the job sits in
```
IN_QUEUE
```
.
```
unhealthy
```
workers, or repeated container starts with no handler output in the logs = a crash loop (a real worker failure, not slow warmup) → switch the worker image or GPU pool, don't wait it out.
```
total: 0
```
for more than a couple of minutes, or
```
throttled
```
above 0 = no capacity in the selected pools and data centers → widen them. A job that never left
```
IN_QUEUE
```
was never parsed: its payload was never read by any worker, so NEVER speculate the input was malformed from queue-stuck evidence. The honest report is the queue/cold-start fact with the worker states quoted. The same discipline covers every stuck-queue anomaly: never present a cause as "conclusive" that your own reads contradict or don't show (ready workers beside a queued job with nothing in progress point to a broken image; say so from the logs, not from the status alone). Prefer a broad, in-stock GPU pool over a scarce high-VRAM tier (e.g. the 48 GB cards) so the endpoint can acquire a worker.
Work with the tools you were granted. In an MCP-only session, do not reach for a web-fetch tool, a shell, or a browser you do not have in order to look up a worker's input schema or read its README. A tool you do not have cannot be called, and the deploy stalls with the job unfinished. Get the input shape from
```
list-hub-repos
```
with
```
includeConfig: true
```
and from the error the endpoint returns, then correct the payload and re-run. If
```
stream-worker-logs
```
is likewise unavailable, diagnose from
```
endpoint-health
```
/
```
list-endpoint-workers
```
; never let an unfinished job hinge on a tool you may not have.
On a terminal
```
FAILED
```
job, report the decisive detail the result gives you (missing file, gated 403, OOM, wrong-format load error): what it says, not what it probably says. OOM → the GPU is too small for the real memory pattern; step up a class. Wrong-format load error on a GGUF repo → the engine is wrong (phase 0). A
```
400
```
on the payload → the input shape is wrong; fix it from the release config and re-run, do not abandon.

在宣布部署完成前必须验证端点可用性：通过服务端等待机制阻塞等待冒烟任务完成，而非让用户自行轮询；工具本身可以承担等待工作。任务进入终态后报告观察结果；如果继续等待不再有意义，则基于已有证据报告当前状态。

提交任务，然后服务端等待 —— 不要紧密轮询。使用
```
run-endpoint
```
提交一个小型任务（对应模态的最小真实输入），然后通过
```
get-job-status
```
的
```
wait: 300000
```
参数（服务端最大等待时长）阻塞等待。首次冷启动的任务会超出
```
runsync
```
的等待时长，且其结果仅保留一分钟，因此
```
runsync-endpoint
```
仅适用于已预热的端点和快速任务。冒烟测试的目的是证明worker能加载真实资源集启动并返回正确格式的输出，而不仅仅是证明端点存在。
首次任务预计有1–5分钟以上的冷启动时间 —— 新worker需要拉取镜像并加载模型。这是正常现象，并非故障。如果任务在一次等待后仍未完成，继续使用
```
get-job-status
```
的
```
wait: 300000
```
参数阻塞等待（一次服务端等待，而非快速重复轮询）。轮询你已有的任务ID；不要重新提交，那会启动新的任务。在两次等待之间检查
```
list-endpoint-workers
```
：如果出现
```
unhealthy
```
worker、有就绪worker但任务仍在队列中、或容量为零 → 停止等待并读取日志（见下文）。每个冒烟任务最多执行三次服务端等待： 对同一个任务ID最多调用三次带
```
wait: 300000
```
的
```
get-job-status
```
。第三次等待后，基于已有证据的报告就是最终答案：不再进行第四次等待，也不紧密轮询。超过三次等待仍未完成的任务直接报告状态，不再继续等待。当继续等待无法获得更多信息时，停止等待并结合worker状态证据报告当前情况：端点已正确创建、任务已提交、worker从未进入就绪状态，以下是worker状态显示的信息。这种有证据支撑的报告就是完整的答案；无休止的等待没有意义。
通过你始终可用的读取接口诊断卡住的任务：
endpoint-health
、
list-endpoint-workers
，以及
get-job-status
在任务处于
IN_QUEUE
状态时附带的
workerHealth
块。首次任务时worker处于
```
initializing
```
状态 = 冷启动可能仍在进行中 → 继续等待，但在下一次等待前读取worker日志（
```
source=system
```
）：因镜像拉取或CUDA不匹配而崩溃循环的容器，在任务处于
```
IN_QUEUE
```
时仍可能报告
```
INITIALIZING
```
或
```
RUNNING
```
状态。如果出现
```
unhealthy
```
worker，或日志显示容器反复启动但没有handler输出 = 崩溃循环（是真实的worker故障，而非预热慢） → 切换worker镜像或GPU资源池，不要继续等待。如果
```
total: 0
```
持续超过几分钟，或
```
throttled
```
大于0 = 所选资源池和数据中心没有容量 → 扩大选择范围。从未离开
```
IN_QUEUE
```
状态的任务意味着从未被解析：其payload从未被任何worker读取，因此绝对不要仅凭任务卡在队列中的证据就推测输入格式错误。诚实的报告是引用worker状态，说明队列/冷启动的事实。同样的原则适用于所有队列卡住的异常情况：永远不要将你自己的读取结果矛盾或未显示的原因说成“结论性的”（例如有就绪worker但任务仍在队列中且没有进行中的任务，说明镜像可能有问题；这需要从日志中得出结论，而非仅靠状态判断）。优先选择范围广、有库存的GPU资源池，而非稀缺的高VRAM规格（例如48 GB显卡），这样端点更容易获取到worker。
使用你被授予的工具。在纯MCP会话中，不要试图使用你没有的网页抓取工具、shell或浏览器来查找worker的输入schema或读取其README。你没有的工具无法调用，会导致部署停滞、任务无法完成。通过带
```
includeConfig: true
```
的
```
list-hub-repos
```
以及端点返回的错误来获取输入格式，然后修正payload并重新运行。如果
```
stream-worker-logs
```
同样不可用，通过
```
endpoint-health
```
/
```
list-endpoint-workers
```
进行诊断；永远不要让未完成的任务依赖于你可能没有的工具。
对于终态为
```
FAILED
```
的任务，报告结果中给出的决定性细节（文件缺失、受限403、OOM、格式错误加载失败）：如实报告结果内容，而非推测内容。OOM → GPU显存太小，无法满足实际内存需求；提升一个规格。GGUF仓库出现格式错误加载失败 → 引擎选择错误（阶段0）。payload返回
```
400
```
→ 输入格式错误；根据发布版配置修正后重新运行，不要放弃。

Phase 4 — hand back

阶段4 — 交付结果

The hand-back is a complete final summary the user never has to scroll back from: the endpoint id; the worker image or Hub repo (and which read it came from); the GPU pool and its hourly price; the smoke job's id and terminal outcome with its proof (the output reference, the quoted failure, or the evidenced timeout report); the sync and async URLs; whether the endpoint was kept (and what is now billing) or deleted (read-back confirmed); and a ready-to-run
```
curl
```
that references
```
$RUNPOD_API_KEY
```
(never the literal key). The sizing arithmetic, format finding, and any caching/placement facts belong in it too.
Cleanup. The default is to KEEP what the user asked for: tell them exactly what is now billing (the endpoint, any min-worker, any network volume) and how to delete it later. Tear down (the endpoint, its template, any network volume, each confirmed gone by a read-back) only when the user framed the journey as a throwaway test or asks for cleanup.

交付内容应是完整的最终总结，用户无需向上翻找历史信息：包括端点ID；worker镜像或Hub仓库（以及来自哪次读取）；GPU资源池及其小时价格；冒烟任务的ID和终态结果及证明（任务返回的输出，以原文引用或参考形式呈现）；同步和异步URL；端点是被保留（以及当前哪些资源在计费）还是被删除（已通过回读确认）；以及一个可直接运行的
```
curl
```
示例，引用
```
$RUNPOD_API_KEY
```
（绝不使用明文密钥）。规格计算过程、格式检测结果，以及任何缓存/部署位置的相关信息也应包含在内。
清理。默认保留用户要求部署的资源：明确告知用户当前哪些资源在计费（端点、任何最小worker、任何网络卷），以及后续如何删除。仅当用户明确表示本次部署为一次性测试或要求清理时，才拆除资源（端点、其模板、任何网络卷，每个资源都需通过回读确认已删除）。

Hard rules

硬性规则

Deploy only the source the user named. Never create an endpoint they did not ask for, and never touch or delete another endpoint, template, or volume.
Pick the engine from the modality and the model format, never a default. A GGUF repo is not a vLLM job; transcription/TTS/OCR/embeddings are not chat-worker jobs.
State the hourly GPU cost before creating a billable endpoint, and remember the smoke job bills too.
Never invent a model URL, a community template repo, or a Docker image. If the source can't be resolved from what the user gave you plus the catalog, ask once or say "none found" and stop.
Prove the endpoint with one real job before calling it deployed. A create call that returned an id is not a working endpoint.
Carry the smoke job to a terminal state with server-side holds (
```
run-endpoint
```
, then
```
get-job-status wait:300000
```
on the job id), expecting a 1–5+ minute first-job cold start. Hold server-side rather than handing the user a poll loop for a wait the tools can carry. If continued waiting stops being informative, end with the evidenced report (worker states, job status). That is a complete answer; silently stopping mid-wait is not.
Work with the tools you were granted. Never make the journey depend on a web-fetch or shell tool you do not have for looking up a worker's schema. Derive the input shape from
```
list-hub-repos includeConfig
```
and the endpoint's own error, and correct the payload in place.
Set every required env key (no config default) at deploy: an embeddings worker without
```
MODEL_NAMES
```
loads no model and fails every job. An optional-capability token (e.g. the token env the worker documents, empty default): if the user provided one, set it at create (never ignore a handed-over credential); if not, deploy anyway, run, and report what came back, then name the token as the unlock.
Start scale-to-zero unless warm capacity was requested; never set a nonzero worker minimum silently; it bills around the clock.
Sizing a big model is shown arithmetic, never a bare total: give the parameter count and where you read it, the bytes per parameter at the chosen precision, the weights total, the KV-cache/overhead allowance, and the GPU decision they add up to.
When the user asked for specific behavior settings (warm/minimum workers, idle timeout, FlashBoot, a GPU pin), read the endpoint back after the create/update and confirm those exact fields landed. An unread write is an unverified write, because the write call's own response is not a read. The REST PATCH is sparse (omitted fields are left untouched) except for the GPU selection:
```
gpu.pools
```
sent without
```
gpu.excludedTypes
```
clears the exclusions, so send both, and after a pin confirm the scaling fields too. The read-back result belongs in the final answer.
When deploying a new endpoint, cold-start asks have four levers; for an endpoint that already exists, go to endpoint-ops and change nothing here. Name all four with their cost trade-offs: FlashBoot (faster worker start, no standing cost, best effort), minimum/warm workers (no cold start; bills that worker continuously, so quote $/hr × 730/mo from the price read), idle timeout (a longer window keeps a warm worker between bursts: fewer cold starts, more billed idle seconds), model caching (the model files on a network volume, in Runpod's model repository through
```
runpodctl model add
```
, or baked into the image; the load, not the boot, dominates big-model cold starts) (the endpoint's model reference is set in the console or with runpodctl; hand that step to the user). Then create the endpoint with the levers that add no standing charge (FlashBoot on, a longer idle timeout), read it back, and report the warm worker as the one lever left with its monthly price. Set a minimum worker only when the user asked for one or accepts that price. A question about which option to pick, with nothing created, does not answer a set-up request.
When weights are too large to bake into the image, stage them on a network volume. Create the network volume in a data center that has the target GPU, attach it to the endpoint, point the worker's model path env at the volume mount, keep minimum workers 0, and confirm the wiring by reading the endpoint and volume back. When the user asked for the storage setup rather than a served model, those read-backs are the proof; a job on a large model costs money and download time, so it waits for their ask. The first worker's download seeds the cache. Do not create a staging pod the user did not ask for; offer it as the option if they want the cache warm before the first request. State two facts of the architecture you built, without hedging: the first worker downloads the weights to the volume once and every later cold start mounts the cache instead of re-downloading; and the volume must live in the same data center as the GPUs or it cannot mount. (The volume CRUD discipline is lifecycle-crud's; this is the serving-side wiring.)
Never echo an HF/Civitai/registry token. Reference it as an env var.
On a smoke-test failure, read the log and quote it; do not retry blindly or assert a cause you did not read.
A terminal job's
```
output
```
/
```
error
```
field IS worker-level evidence: quote it verbatim when explaining a failure (a gated-model rejection, a load error). Never tell the user you cannot quote evidence while holding a terminal job result; lacking a log-stream tool does not excuse dropping the evidence the job itself returned. Empty or missing output on COMPLETED jobs from a gated-capability worker is itself quotable evidence: report it as such, pair it with the config fact (the token env the worker documents was not set), and state the gate.

仅部署用户指定的来源。严禁创建用户未要求的端点，严禁触碰或删除其他端点、模板或卷。
根据任务模态和模型格式选择引擎，绝不使用默认值。GGUF仓库不是vLLM的任务；转录/TTS/OCR/嵌入不是聊天worker的任务。
在创建会产生费用的端点前，必须说明GPU的小时费用，且冒烟测试也会产生费用。
严禁编造模型URL、社区模板仓库或Docker镜像。如果根据用户提供的信息加上目录仍无法确定来源，询问一次或说明“未找到”后停止。
在宣布部署完成前，必须通过一个真实任务验证端点可用性。仅返回ID的创建调用不代表端点可正常工作。
通过服务端等待将冒烟任务推进到终态（
```
run-endpoint
```
提交，然后对任务ID调用
```
get-job-status wait:300000
```
），预期首次任务有1–5分钟以上的冷启动时间。使用服务端等待，而非让用户自行轮询——工具可以承担等待工作。如果继续等待无法获得更多信息，以有证据支撑的报告结束（worker状态、任务状态）。这才是完整的答案；在等待过程中静默停止是不可接受的。
使用你被授予的工具。永远不要让部署流程依赖于你没有的网页抓取或shell工具来查找worker的schema。通过
```
list-hub-repos includeConfig
```
和端点自身返回的错误来推导输入格式，并就地修正payload。
部署时必须设置所有必填环境变量（配置中无默认值的）：没有
```
MODEL_NAMES
```
的嵌入worker不会加载任何模型，所有任务都会失败。对于可选能力的token（例如worker文档中说明的token环境变量，默认值为空）：如果用户提供了，在创建时设置（绝不忽略用户提交的凭证）；如果没有，仍继续部署、运行、报告返回结果，然后说明该token是解锁该能力的关键。
除非用户要求预热容量，否则默认以缩容到零模式启动；严禁静默设置非零的worker最小数量；这会24小时持续计费。
大模型的规格计算必须展示算术过程，不能只给出最终总数：说明参数量及读取来源、所选精度下每个参数的字节数、权重总大小、KV缓存/开销预留，以及最终得出的GPU选型。
当用户要求特定的行为设置（预热/最小worker数量、空闲超时、FlashBoot、GPU固定）时，在创建/更新后必须回读端点，确认这些字段确实生效。未经过回读的写入就是未验证的写入，因为写入调用的响应本身不等于读取结果。REST PATCH是稀疏更新（省略的字段保持不变），但GPU选择除外：只发送
```
gpu.pools
```
而不发送
```
gpu.excludedTypes
```
会清除排除列表，因此必须同时发送两者，且固定GPU后也要确认扩缩容相关字段。回读结果应包含在最终答案中。
部署新端点时，冷启动优化有四个杠杆；对于已存在的端点，转到端点操作流程，此处不做任何更改。列出所有四个杠杆及其成本权衡：FlashBoot（worker启动更快，无固定成本，尽力而为）、最小/预热worker（无冷启动；该worker持续计费，因此根据价格读取结果报出$/小时 × 730小时/月的费用）、空闲超时（更长的时间窗口能在流量突发之间保持worker预热：冷启动更少，但计费的空闲秒数更多）、模型缓存（模型文件存储在网络卷、通过
```
runpodctl model add
```
存入Runpod模型仓库，或预打包进镜像；大模型冷启动的主要耗时是模型加载而非启动）（端点的模型引用需在控制台或通过runpodctl设置；该步骤交给用户完成）。然后使用不会产生固定费用的杠杆（开启FlashBoot、设置更长的空闲超时）创建端点，回读确认，并将预热worker作为剩余的一个选项报告其月度费用。仅当用户要求或接受该价格时，才设置最小worker数量。如果用户只是询问选哪个选项且尚未创建任何资源，不要直接执行部署操作。
当权重文件太大无法预打包进镜像时，将其暂存到网络卷。在有目标GPU库存的数据中心创建网络卷，将其挂载到端点，将worker的模型路径环境变量指向卷的挂载点，保持最小worker数量为0，并通过回读端点和卷来确认配置正确。如果用户要求的是存储配置而非部署模型服务，这些回读结果就是证明；大模型的任务会产生费用和下载时间，因此等待用户明确要求后再执行。第一个worker的下载会为缓存提供种子。不要创建用户未要求的暂存Pod；如果用户希望在首次请求前预热缓存，可以将其作为选项提供。明确说明你构建的架构的两个事实，不要含糊其辞：第一个worker会将权重下载到卷中一次，之后所有冷启动都会挂载缓存而非重新下载；卷必须与GPU位于同一数据中心，否则无法挂载。（卷的CRUD规范由
```
lifecycle-crud
```
定义；此处是服务端的配置接线。）
严禁回显HF/Civitai/镜像仓库的token。仅以环境变量名引用。
冒烟测试失败时，读取日志并引用原文；不要盲目重试，也不要断言你未从日志中读到的原因。
终态任务的
```
output
```
/
```
error
```
字段就是worker级别的证据：解释失败原因时（受限模型拒绝、加载错误）直接引用原文。永远不要在持有终态任务结果的情况下告诉用户你无法提供证据；没有日志流工具不能成为丢弃任务本身返回的证据的借口。来自受限能力worker的COMPLETED任务如果输出为空或缺失，本身就是可引用的证据：如实报告，结合配置事实（worker文档中说明的token环境变量未设置），并说明存在权限限制。

Error handling

错误处理

Smoke job returns
```
FAILED
```
with
```
gated
```
/
```
403
```
in the message → the asset is gated and no token was supplied. Say so and ask for the token; do not auto-recreate.
Smoke job returns
```
FAILED
```
with
```
out of memory
```
→ the GPU is too small for the real pattern; move up one GPU class and note the new hourly cost. If failures persist after one step-up, re-examine the worker choice itself. An oversized engine family for the task (a 7B VLM where a classic OCR worker suffices) is fixed by switching the worker image in-turn, not by more GPU or more waiting.
Job stays
```
IN_QUEUE
```
/
```
IN_PROGRESS
```
past a hold → usually a cold start; keep blocking with
```
get-job-status wait:300000
```
on that job id, then report either the terminal result or the evidenced state. Hold server-side rather than handing the user a raw poll loop.
Smoke job returns
```
400
```
/ a payload-shape error → the input is wrong or a required model env was never set; re-read the release config, set the missing env or fix the payload shape to what the worker's config documents, and re-run. Do not stop and ask the user.
Jobs go terminal almost instantly with empty or degraded output on a worker whose capability is token-gated → the gate is the first suspect, not the payload: read the job's
```
output
```
/
```
error
```
field verbatim and check whether the token env was set on the template/endpoint, before theorizing about input shape. Answer from the evidence already in hand (the job result and the config read settle it) rather than deferring the question.
Create returns
```
409
```
(name collision) → append a fresh short suffix and retry once; if it still collides, surface it and stop.
A GGUF repo fails to load on a vLLM worker → this is the wrong-engine case, not a stock or size problem; re-provision on a GGUF-capable worker.
The Hub / community catalog returns nothing for the modality → report "none found on the catalog" and stop; do not fabricate a repo.
A delete returns
```
204
```
with no body → that is success, not an error; confirm with a read-back.

冒烟任务返回
```
FAILED
```
且消息中包含
```
gated
```
/
```
403
```
→ 资源是受限的，且未提供token。说明情况并向用户索要token；不要自动重新创建。
冒烟任务返回
```
FAILED
```
且包含
```
out of memory
```
→ GPU显存太小，无法满足实际内存需求；提升一个GPU规格并说明新的小时费用。如果提升一次后仍失败，重新检查worker选择本身。对于任务来说过大的引擎系列（例如用7B VLM做经典OCR就能完成的任务），应通过切换worker镜像来解决，而非增加GPU或继续等待。
任务超过一次等待时长仍处于
```
IN_QUEUE
```
/
```
IN_PROGRESS
```
状态 → 通常是冷启动；继续对该任务ID调用
```
get-job-status wait:300000
```
阻塞等待，然后报告终态结果或有证据支撑的状态。使用服务端等待，而非让用户自行原始轮询。
冒烟任务返回
```
400
```
/ payload格式错误 → 输入有误或从未设置必填的模型环境变量；重新读取发布版配置，补全缺失的环境变量或根据worker配置文档修正payload格式，然后重新运行。不要停下来询问用户。
对于能力受token限制的worker，任务几乎瞬间进入终态但输出为空或质量下降 → 首先怀疑是权限限制，而非payload问题：在推测输入格式之前，先逐字读取任务的
```
output
```
/
```
error
```
字段，并检查模板/端点上是否设置了token环境变量。基于已有的证据回答（任务结果和配置读取就能解决），不要把问题推给用户。
创建返回
```
409
```
（名称冲突） → 添加一个新的短后缀并重试一次；如果仍冲突，上报问题并停止。
GGUF仓库在vLLM worker上加载失败 → 这是引擎选择错误的问题，而非库存或规格问题；切换到支持GGUF的worker重新部署。
Hub/社区目录中没有找到对应模态的资源 → 报告“目录中未找到”并停止；不要编造仓库。
删除返回
```
204
```
且无响应体 → 这是成功，而非错误；通过回读确认。

Tool binding

工具绑定

Capability	Tool
GPU catalog + price + stock	`list-gpu-types` / `get-gpu-type` ( `include:["AVAILABILITY"]` , `product:["SERVERLESS"]` )
Per-data-center GPU availability	the same read ( `dataCenters[]` ); `list-data-centers` ( `include:["GPU_AVAILABILITY"]` )
CUDA versions with capacity, per GPU type	the same read ( `cudaVersions` ); `get-capacity` for the matrix view
Data centers	`list-data-centers` , `get-data-center`
Browse Hub / community catalog	`list-hub-repos` ( `includeConfig:true` )
Deploy a Hub release directly	`deploy-hub-repo`
Pin a Hub release / read history	`list-endpoint-releases`
Create a template	`create-template`
Create an endpoint	`create-endpoint`
Create a network volume	`create-network-volume`
Multi-GPU / pin to one card	`set-endpoint-gpus` ( `gpuCount` , `pools` , exclusions); or `update-endpoint` with `gpu.pools` and `gpu.excludedTypes` together
Endpoint health / warmup	`endpoint-health`
List workers (per-worker detail)	`list-endpoint-workers`
Smoke test (block until terminal)	`run-endpoint` , then `get-job-status` ( `wait:300000` ); `runsync-endpoint` for a warm endpoint and a fast job
Read a worker's logs	`stream-worker-logs` ( `source=system` for the container lifecycle)
Registry credential (private image)	`create-registry` , `get-registry`
Inspect endpoint / template	`get-endpoint` , `get-template`
Teardown	`delete-endpoint` , `delete-template` , `delete-network-volume`

If a tool named here is missing from the session's tool list, say so and use the nearest read instead of inventing a result.

能力	工具
GPU目录 + 价格 + 库存	`list-gpu-types` / `get-gpu-type` （ `include:["AVAILABILITY"]` , `product:["SERVERLESS"]` ）
各数据中心GPU可用性	同一次读取（ `dataCenters[]` ）； `list-data-centers` （ `include:["GPU_AVAILABILITY"]` ）
各GPU型号有容量的CUDA版本	同一次读取（ `cudaVersions` ）； `get-capacity` 获取矩阵视图
数据中心	`list-data-centers` , `get-data-center`
浏览Hub / 社区目录	`list-hub-repos` （ `includeConfig:true` ）
直接部署Hub发布版	`deploy-hub-repo`
固定Hub发布版 / 读取历史	`list-endpoint-releases`
创建模板	`create-template`
创建端点	`create-endpoint`
创建网络卷	`create-network-volume`
多GPU / 固定单卡	`set-endpoint-gpus` （ `gpuCount` 、 `pools` 、排除列表）；或通过 `update-endpoint` 同时设置 `gpu.pools` 和 `gpu.excludedTypes`
端点健康 / 预热	`endpoint-health`
列出worker（单worker详情）	`list-endpoint-workers`
冒烟测试（阻塞直到终态）	`run-endpoint` ，然后调用 `get-job-status` （ `wait:300000` ）； `runsync-endpoint` 用于已预热端点和快速任务
读取worker日志	`stream-worker-logs` （容器生命周期使用 `source=system` ）
镜像仓库凭证（私有镜像）	`create-registry` , `get-registry`
检查端点 / 模板	`get-endpoint` , `get-template`
资源拆除	`delete-endpoint` , `delete-template` , `delete-network-volume`

如果会话的工具列表中缺少此处列出的某个工具，说明情况并使用最接近的读取工具替代，不要编造结果。

Report template — deployment summary

报告模板 — 部署总结

A good deploy summary tells the user: which worker/engine was chosen and why; the endpoint name and id; the GPU pool and its hourly price from the catalog read; the smoke-job proof (the output the job returned, quoted or referenced); the current worker state; what the endpoint costs standing (scale-to-zero or $/hr); and how to delete it later. When the wait ended without a terminal job, the evidence takes the proof's place: job id, submitted payload, worker states observed, and the recommended next step.

一份好的部署总结应告知用户：选择了哪个worker/引擎及其原因；端点名称和ID；GPU资源池及其来自目录读取的小时价格；冒烟测试证明（任务返回的输出，以原文引用或参考形式呈现）；当前worker状态；端点的固定成本（缩容到零或$/小时）；以及后续如何删除。如果等待结束后任务仍未进入终态，则用证据代替证明：任务ID、提交的payload、观察到的worker状态，以及建议的下一步操作。

Contract

约定

The cross-journey answer contract (definite facts from reads, commit-don't-hedge, mutations bind to this conversation's creations, a standalone final message, honest failure, granted tools only) is defined in the

runpod-mcp

skill and applies to every reply from this journey. If the

runpod-mcp

skill has not been loaded in this conversation, load it now, before the next tool call: its rules on which resources you may change, when a request is the go, and when to stop for the user are not repeated here.

跨流程的回答约定（基于读取的确定事实、明确承诺不含糊、仅修改本次对话创建的资源、独立完整的最终消息、诚实报告失败、仅使用被授予的工具）在

runpod-mcp

技能中定义，适用于本流程的所有回复。如果本次对话中尚未加载

runpod-mcp

技能，请在下次工具调用前加载：其关于可修改哪些资源、何时可以执行请求、何时需要停下来询问用户的规则，此处不再重复。

serverless-deploy

Original

Translation

Serverless deploy

Serverless部署

Required capabilities

所需能力

Phase 0 — identify the source and pick the engine (unprompted decision)

阶段0 — 识别来源并选择引擎（自主决策）

Phase 1 — size the hardware (read the catalog, cite
`discovery`
)

阶段1 — 确定硬件规格（读取目录，引用
`discovery`
结果）

Phase 2 — provision

阶段2 — 资源部署

Phase 3 — warm and smoke-test one real job

阶段3 — 预热并执行一次真实任务的冒烟测试

Phase 4 — hand back

阶段4 — 交付结果

Hard rules

硬性规则

Error handling

错误处理

Tool binding

工具绑定

Report template — deployment summary

报告模板 — 部署总结

Contract

约定

serverless-deploy

Original

Translation

Serverless deploy

Serverless部署

Required capabilities

所需能力

Phase 0 — identify the source and pick the engine (unprompted decision)

阶段0 — 识别来源并选择引擎（自主决策）

Phase 1 — size the hardware (read the catalog, cite discovery)

阶段1 — 确定硬件规格（读取目录，引用discovery结果）

Phase 2 — provision

阶段2 — 资源部署

Phase 3 — warm and smoke-test one real job

阶段3 — 预热并执行一次真实任务的冒烟测试

Phase 4 — hand back

阶段4 — 交付结果

Hard rules

硬性规则

Error handling

错误处理

Tool binding

工具绑定

Report template — deployment summary

报告模板 — 部署总结

Contract

约定

Phase 1 — size the hardware (read the catalog, cite
`discovery`
)

阶段1 — 确定硬件规格（读取目录，引用
`discovery`
结果）