runpod-flash
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseRunpod SDK apps (Flash)
Runpod SDK 应用(Flash)
For an explicit Dockerfile/template project request, use
runpod-build-template. Do not replace that
deliverable with Flash unless the user chooses its code-first workflow.
Write Python locally, run it on remote Runpod GPUs/CPUs with , then
to ship. Flash is part of the package (2.0): one SDK, one
CLI, one login. The standalone package is deprecated and gets no fixes
after October 29, 2026.
rp flash devrp flash deployrunpodrprunpod-flashrp --helprp flash <command> --helppip show runpod对于明确要求使用 Dockerfile/模板的项目,请使用
runpod-build-template。除非用户选择其代码优先工作流,
否则不要用 Flash 替代该交付物。
在本地编写 Python,通过 在远程 Runpod GPU/CPU 上运行,然后
使用 发布。Flash 是 包(2.0)的一部分:一个 SDK、一个
CLI、一个登录流程。独立的 包已弃用,2026 年 10 月 29 日之后
不再获得修复。
rp flash devrp flash deployrunpodrprunpod-flashrp --helprp flash <command> --helppip show runpodWhich SDK is this project on?
此项目使用的是哪个 SDK?
Check before writing code. The two APIs do not mix.
| You see | Project is on | Do |
|---|---|---|
| Runpod SDK 2.0 | This file + |
| Legacy runpod-flash | reference/runpod-flash-legacy.md to maintain it, reference/migrate-from-runpod-flash.md to move it |
| A new project | Runpod SDK 2.0 | Start below |
Write new code on the SDK. For an existing project, propose migrating and
check its Known differences first; don't migrate silently.
runpod-flashIf the SDK path fails on an SDK bug, is the fallback: a traceback inside
, a worker that crashes before your function runs, or the same code working on
. Upgrade first; the steps to check for a fix and fall back are in
reference/runpod-flash-legacy.md.
runpod-flashrunpod/apps/runpod-flashrunpodWorked examples first for anything multi-step. Flash appears in end-to-end paths:
03 variant B (whisper endpoint),
07 (network-volume handoff) and
08 (fine-tune → serve); the full index is
runpod/golden-paths/README.md. Runnable single-file apps
live in .
examples/Load on demand:
| Need | Read |
|---|---|
Install, | reference/setup-and-cli.md |
Decorator parameters, call styles, | reference/api.md |
| Worked patterns: choosing a model, warm model loading, CPU→GPU pipeline, fan-out, tasks, raw HTTP | reference/patterns.md |
Moving a | reference/migrate-from-runpod-flash.md |
Maintaining a | reference/runpod-flash-legacy.md |
Quick start: (or ) → (or )
→ → .
uv add "runpod>=2.0.0"pip install "runpod>=2.0.0"rp loginexport RUNPOD_API_KEY=...rp flash init my-appcd my-app && rp flash dev main.py编写代码前先检查。两个 API 无法混用。
| 你看到的内容 | 项目所用 SDK | 应对操作 |
|---|---|---|
| Runpod SDK 2.0 | 本文档 + |
| 旧版 runpod-flash | 使用 reference/runpod-flash-legacy.md 维护,使用 reference/migrate-from-runpod-flash.md 迁移 |
| 新项目 | Runpod SDK 2.0 | 从下文开始 |
新代码使用 SDK。对于现有的 项目,应先提出迁移建议并检查其
已知差异(Known differences);不要在未经确认的情况下静默迁移。
runpod-flash如果 SDK 路径因 SDK 缺陷而失败, 是备用方案:
内部出现 traceback、Worker 在函数执行前崩溃,或同一段代码在
中可以运行。先升级 ;检查修复版本和回退步骤的说明见
reference/runpod-flash-legacy.md。
runpod-flashrunpod/apps/runpod-flashrunpod对于任何多步骤任务,优先参考完整示例。 Flash 出现在以下端到端路径中:
03 变体 B(Whisper Endpoint)、
07(NetworkVolume 交接) 和
08(微调 → 服务);完整索引见
runpod/golden-paths/README.md。可运行的单文件应用
位于 中。
examples/按需加载:
| 需求 | 阅读 |
|---|---|
安装、 | reference/setup-and-cli.md |
Decorator 参数、调用方式、 | reference/api.md |
| 实用模式:选择模型、预加载模型、CPU→GPU 流水线、扇出、任务、原始 HTTP | reference/patterns.md |
将 | reference/migrate-from-runpod-flash.md |
原样维护 | reference/runpod-flash-legacy.md |
快速入门:(或 )→ (或 )
→ → 。
uv add "runpod>=2.0.0"pip install "runpod>=2.0.0"rp loginexport RUNPOD_API_KEY=...rp flash init my-appcd my-app && rp flash dev main.pyThe app model
应用模型
python
import runpod
app = runpod.App("my-app") # names the group of resources
@app.queue(gpu="4090", dependencies=["torch"]) # queue-based serverless endpoint
def infer(x: list[float]):
import torch
return {"sum": torch.tensor(x, device="cuda").sum().item()}
@runpod.local_entrypoint # runs on YOUR machine
def main():
print(infer.remote([1.0, 2.0, 3.0])) # runs on a Runpod worker| Decorator | Creates | Use for |
|---|---|---|
| Autoscaling queue-based endpoint | Request/response inference, batch jobs |
| Load-balanced endpoint serving routes ( | HTTP services, multi-route APIs, load-once model state |
| One ephemeral Pod per call, terminated when it returns | Training, fine-tuning, long jobs |
Call styles on a queue or task handle: (blocks), ,
(returns a , or a for a task), (queue generator
functions only), (runs on your machine). API classes are called by route: .
fn.remote(...)await fn.remote.aio(...)fn.spawn(...)JobTaskJobfn.stream(...)fn.local(...)MyApi.post("/path", body)python
import runpod
app = runpod.App("my-app") # names the group of resources
@app.queue(gpu="4090", dependencies=["torch"]) # queue-based serverless endpoint
def infer(x: list[float]):
import torch
return {"sum": torch.tensor(x, device="cuda").sum().item()}
@runpod.local_entrypoint # runs on YOUR machine
def main():
print(infer.remote([1.0, 2.0, 3.0])) # runs on a Runpod worker| Decorator | 创建的资源 | 适用场景 |
|---|---|---|
函数上的 | 自动扩缩的队列式 Endpoint | 请求/响应推理、批处理作业 |
类上的 | 承载路由的负载均衡 Endpoint( | HTTP 服务、多路由 API、只需加载一次的模型状态 |
函数上的 | 每次调用创建一个临时 Pod,返回后终止 | 训练、微调、长时作业 |
队列或任务的调用方式:(阻塞调用)、、
(返回 ,任务则返回 )、(仅适用于队列生成器
函数)、(在你的机器上运行)。API 类按路由调用:。
fn.remote(...)await fn.remote.aio(...)fn.spawn(...)JobTaskJobfn.stream(...)fn.local(...)MyApi.post("/path", body)Dev vs deploy
Dev 与 Deploy
- — iterate. Creates temporary endpoints (
rp flash dev main.py), runs thedev-<app>-<resource>-<hash>on your machine, streams the workers'@runpod.local_entrypointoutput to your terminal, and re-runs after every file save. Ctrl-C deletes the endpoints. There is no local HTTP server; the entrypoint is how you exercise the code.print - — run the entrypoint once, tear down, exit non-zero on failure. The smoke test for CI and for "does the migrated code work".
rp flash dev main.py --once - — ship. Builds an artifact (≤1,500 MB, torch excluded on built-in GPU images) and creates persistent endpoints in the
rp flash deployenvironment (defaultfor others). Re-run to update.--env NAMEpackages without deploying, a cheap offline check that discovery and dependency resolution work.--build-only
Dev and deploy run the same module source, so module-level imports, constants and helpers
behave identically in both. Deploy packages the project directory, but a dev session ships
only the module's source, so keep the app in one file when you use .
rp flash dev- — 迭代开发。 创建临时 Endpoint(
rp flash dev main.py), 在你的机器上运行dev-<app>-<resource>-<hash>,将 Worker 的@runpod.local_entrypoint输出流式传输到 终端,并在每次保存文件后重新运行。Ctrl-C 会删除这些 Endpoint。这里没有 本地 HTTP 服务器;应通过 entrypoint 验证代码。print - — 运行 entrypoint 一次,然后拆除资源;失败时以非零状态退出。 适用于 CI,以及检查“迁移后的代码是否正常工作”。
rp flash dev main.py --once - — 发布。 构建 artifact(≤1,500 MB;内置 GPU 镜像中不包含 torch), 并在
rp flash deploy环境中创建持久 Endpoint(其他环境使用default)。重新运行即可更新。--env NAME只打包而不部署,可离线快速检查资源发现和依赖解析是否正常。--build-only
Dev 和 Deploy 运行的是同一个模块源文件,因此模块级 import、常量和辅助函数在两者中的
行为完全一致。Deploy 会打包项目目录,但 Dev 会话仅发送模块的源代码,因此使用
时应将应用保持在一个文件中。
rp flash devAutonomous dev loop
自主开发循环
A dev session is long-running. Three rules:
- Run it in the background with stdin closed (). With no stdin it re-runs the entrypoint on every file save, so you never need to press Enter.
< /dev/null - Capture its output to a log file and read it. Worker s, tracebacks,
printand the per-run✗ remote execution failed: .../✓ done in ...lines land there.✗ failed in ... - Stop it with (SIGTERM) or SIGINT, so it deletes its endpoints, and confirm the log ends with
kill. A second signal stops waiting for a running entrypoint and goes straight to cleanup.cleaning up N deletedskips cleanup. When one run is enough, preferkill -9; it cleans up on its own.--once
bash
rp flash dev main.py < /dev/null > /tmp/rp-dev.log 2>&1 &
DEV=$!Dev 会话会持续运行。遵循三条规则:
- 在后台运行并关闭 stdin()。没有 stdin 时,它会在每次保存文件后重新运行 entrypoint,因此无需按 Enter。
< /dev/null - 将输出捕获到日志文件并读取。 Worker 的 、traceback、
print, 以及每次运行的✗ remote execution failed: .../✓ done in ...行都会写入该文件。✗ failed in ... - 使用 (SIGTERM)或 SIGINT 停止,以便删除 Endpoint;并确认日志以
kill结尾。第二个信号会停止等待正在运行的 entrypoint 并直接进入清理流程。cleaning up N deleted会跳过清理。如果只需运行一次,优先使用kill -9;它会自动完成清理。--once
bash
rp flash dev main.py < /dev/null > /tmp/rp-dev.log 2>&1 &
DEV=$!poll the log (bounded) for the run result: "✓ done in ..." or "✗ failed in ..."
poll the log (bounded) for the run result: "✓ done in ..." or "✗ failed in ..."
for i in $(seq 1 120); do grep -qE "✓ done in|✗ failed in" /tmp/rp-dev.log && break; sleep 3; done
tail -40 /tmp/rp-dev.log
for i in $(seq 1 120); do grep -qE "✓ done in|✗ failed in" /tmp/rp-dev.log && break; sleep 3; done
tail -40 /tmp/rp-dev.log
edit main.py and save -> the session re-scans, refreshes endpoints, re-runs main()
edit main.py and save -> the session re-scans, refreshes endpoints, re-runs main()
(each run appends a new result line; read the last one)
(each run appends a new result line; read the last one)
kill $DEV; wait $DEV; tail -3 /tmp/rp-dev.log # expect "cleaning up N deleted"
For a single check, `rp flash dev main.py --once` is simpler and cleans up on its own. If a
session died without cleanup, list leftovers with `runpodctl serverless list` and delete the
`dev-<app>-...` endpoints **this session created**, by ID.kill $DEV; wait $DEV; tail -3 /tmp/rp-dev.log # expect "cleaning up N deleted"
如果只需执行一次检查,`rp flash dev main.py --once` 更简单,并且会自动清理。如果会话意外终止且未完成清理,
可使用 `runpodctl serverless list` 列出残留资源,并按 ID 删除**此会话创建的**
`dev-<app>-...` Endpoint。Choosing the resource shape
选择资源形态
Default to plain Python on the Runpod runtime image: and
install on the worker, so no Dockerfile is needed, and even large
HuggingFace models stay in code (see reference/patterns.md).
dependencies=[...]system_dependencies=[...]- on any decorator runs your function on that image; it needs Python 3, not a Serverless handler. Use it for system libraries or a specific CUDA stack.
image="org/image:tag"names saved credentials for a private image (registry_auth=).rp registry add - A prebuilt inference server image (vLLM, worker-comfyui) that already implements a
Serverless handler is not an SDK app. Deploy it with runpodctl or runpod-mcp and call it
with .
runpod.Endpoint("ENDPOINT_ID") - An existing endpoint by ID: .
runpod.Endpoint("ENDPOINT_ID").run_sync({...}) - A resource in another deployed app: /
runpod.Queue(app="media", name="transcribe").runpod.Api(app="media", name="inference")
默认使用 Runpod runtime image 上的普通 Python: 和
会在 Worker 上安装,因此无需 Dockerfile;即使是大型
HuggingFace 模型,也可以直接保留在代码中(参见 reference/patterns.md)。
dependencies=[...]system_dependencies=[...]- 在任意 Decorator 上使用 ,可在该镜像上运行函数;它需要 Python 3,而不是 Serverless handler。适用于系统库或特定 CUDA 技术栈。
image="org/image:tag"指定私有镜像所使用的已保存凭据(registry_auth=)。rp registry add - 预构建推理服务器镜像(vLLM、worker-comfyui)如果已经实现 Serverless handler,
就不是 SDK 应用。使用 runpodctl 或 runpod-mcp 部署,并通过
调用。
runpod.Endpoint("ENDPOINT_ID") - 按 ID 使用现有 Endpoint:。
runpod.Endpoint("ENDPOINT_ID").run_sync({...}) - 使用另一个已部署应用中的资源:/
runpod.Queue(app="media", name="transcribe")。runpod.Api(app="media", name="inference")
Gotchas
注意事项
- Queue arguments and results must be JSON-serializable. Queue jobs are plain JSON in dev
and deploy alike. A non-JSON argument fails in dev with and against a deployed endpoint with Python's own JSON
arguments must be json-serializable. Pass URLs or base64, not objects. Whole-number floats can arrive as integers (TypeError→1.0), so cast numeric inputs explicitly (1).torch.tensor(x, dtype=torch.float32) - One file per app in dev. A dev session ships only the module's source, so a sibling local module imports in deploy but fails in dev. For dev sessions, keep helpers in the same file or in an installable package.
- Keep blocking work out of module scope. The SDK imports your module to discover
resources, locally and on every worker. Put heavy imports () and work inside functions; guard local-only code with
torch.runpod.is_local() - and
gpu=are exclusive (cpu=). GPU strings can be short (gpu and cpu are mutually exclusive,"4090") or a list of acceptable GPUs."H100" - Default (runpod-flash was
workers=(0, 3)). Set it explicitly when cost or concurrency matters.(0, 1)is seconds (default 60).idle_timeout - Auto GPU switching needs max ≥ 5 when you pass a list of GPUs.
workers - 10 MB payload limit on requests. Return large outputs via a volume or object storage URL.
- Volumes outlive everything. Dev sessions and delete endpoints, not the
rp flash undeploys they created, and those keep billing. Queue and API resources mount one volume atNetworkVolume; read/runpod-volumeinside the remote function, never at import time. Load model weights throughvol.path, not straight off the mount (reference/patterns.md).VolumeCache - Tasks bill until they return. A ed task that you never
.spawn()orwait()keeps its Pod running after your script exits.cancel() - Native CUDA libraries go in too, e.g. faster-whisper needs
dependencies+nvidia-cublas-cu12or it silently falls back to CPU.nvidia-cudnn-cu12 - Dev sessions and tasks need local Python 3.10–3.14. They ship code pickled by your
interpreter, so the worker matches your local version. Deploy falls back to 3.12 with a
warning on other versions; sets it explicitly.
rp flash deploy --python-version - A set wins over
RUNPOD_API_KEY. A stale exported key causes 401s while the saved login is fine. Check withrp login, thenrunpodctl useror export a good key.unset RUNPOD_API_KEY - The torch family comes from the GPU image; never pin it. Builds leave ,
torch,torchvisionandtorchaudioout of the artifact whether you list them or a dependency pulls them in (triton,peft,accelerate), plus the wheels only torch needs. GPU workers use the image's build (sentence-transformers), so a version pin on these packages is ignored. In an app that mixes CPU and GPU resources, a CPU resource whose dependencies need torch installs it at cold start; give ittorch 2.9.1+cu128room for that (the build warns when this applies).container_disk_gb - Dev sessions run the runtime image's own . Deploys vendor your local package, but
runpodworkers import therp flash devbaked into the built-in image. Anrunpodon aAttributeErrorname in a dev worker traceback means the image is behind your SDK;runpod.runs your installed version.rp flash deploy - Teardown by app and environment: removes one environment's endpoints;
rp flash undeploy --app APP --env default -yremoves every environment (withoutrp flash app delete APP --yesit prompts, and aborts with no terminal). Then delete volumes you no longer need.--yes - Queue and API names need 3+ characters to deploy. The name defaults to the function or
class name, and dev sessions don't check it, so passes
class SDand failsrp flash dev. Rename it or passrp flash deploy.name= - Every endpoint gets your account API key. Deploy and dev put in each endpoint's template env, whether or not its code calls another resource, so anyone who can read the template can read the key.
RUNPOD_API_KEY - Re-running the same app soon after a dev session can fail. Dev sessions leave their
templates behind, and the platform holds them for up to about 90 minutes, so a new
of the same app fails with "endpoint template names must be unique". Wait, or rename the
rp flash dev.runpod.App
- 队列参数和结果必须可进行 JSON 序列化。 无论在 Dev 还是 Deploy 中,队列作业都只是
普通 JSON。无法进行 JSON 序列化的参数会在 Dev 中触发
错误,在已部署的 Endpoint 上则触发 Python 原生的 JSON
arguments must be json-serializable。请传入 URL 或 base64,不要传入对象。整数值浮点数可能以整数形式到达 (TypeError→1.0),因此应显式转换数值输入类型(1)。torch.tensor(x, dtype=torch.float32) - Dev 中每个应用只能使用一个文件。 Dev 会话仅发送模块源代码,因此同级本地模块在 Deploy 中可以导入,却在 Dev 中失败。Dev 会话应将辅助代码放在同一文件中,或放入可安装的包中。
- 不要在模块作用域中执行阻塞工作。 SDK 会在本地及每个 Worker 上导入模块,以发现资源。
将重量级 import(如 )和工作放入函数内部;使用
torch保护仅限本地执行的代码。runpod.is_local() - 和
gpu=互斥(cpu=)。GPU 字符串可以是 短名称(gpu and cpu are mutually exclusive、"4090"),也可以是可选 GPU 的列表。"H100" - 是默认值(runpod-flash 为
workers=(0, 3))。当成本或并发能力很重要时, 请显式设置。(0, 1)的单位是秒,默认值为 60。idle_timeout - 传入 GPU 列表时,自动切换 GPU 要求 的最大值 ≥ 5。
workers - 请求的载荷上限为 10 MB。大型输出应通过 Volume 或对象存储 URL 返回。
- Volume 的生命周期长于其他资源。 Dev 会话和 会删除 Endpoint, 但不会删除它们创建的
rp flash undeploy,后者仍会持续计费。Queue 和 API 资源会在NetworkVolume挂载一个 Volume;应在远程函数内部读取/runpod-volume,切勿在 import 时读取。 通过vol.path加载模型权重,不要直接从挂载目录加载 (reference/patterns.md)。VolumeCache - Task 在返回前一直计费。 通过 创建且从未
.spawn()或wait()的任务, 会在脚本退出后继续运行其 Pod。cancel() - 原生 CUDA 库也应放入 ,例如 faster-whisper 需要
dependencies和nvidia-cublas-cu12,否则会静默回退到 CPU。nvidia-cudnn-cu12 - Dev 会话和 Task 需要本地 Python 3.10–3.14。 它们会发送由本地解释器序列化的代码,
因此 Worker 必须与本地版本一致。Deploy 在其他版本上会回退到 3.12 并显示警告;
可使用 显式指定版本。
rp flash deploy --python-version - 已设置的 优先于
RUNPOD_API_KEY。 导出的过期密钥会在保存的登录凭据 正常的情况下仍导致 401。使用rp login检查,然后runpodctl user或导出有效密钥。unset RUNPOD_API_KEY - torch 系列依赖来自 GPU 镜像;切勿固定其版本。 无论你是否显式列出,或通过某个依赖项
间接引入(如 、
peft、accelerate),构建过程都会从 artifact 中排除sentence-transformers、torch、torchvision和torchaudio,以及仅 torch 需要的 wheels。 GPU Worker 使用镜像中的构建版本(triton),因此这些包的版本固定会被忽略。 如果应用同时包含 CPU 和 GPU 资源,而某个 CPU 资源的依赖需要 torch,它会在冷启动时安装; 请为其预留足够的torch 2.9.1+cu128空间(出现这种情况时构建过程会给出警告)。container_disk_gb - Dev 会话使用 runtime image 自带的 。 Deploy 会打包本地安装的包, 但
runpod的 Worker 会导入内置镜像中的rp flash dev。如果 Dev Worker 的 traceback 中出现runpod名称相关的runpod.,说明镜像中的 SDK 版本落后于本地版本;AttributeError会使用已安装的版本。rp flash deploy - 按应用和环境拆除资源:删除某个环境中的 Endpoint;
rp flash undeploy --app APP --env default -y删除所有环境中的 Endpoint (不带rp flash app delete APP --yes时会提示确认,并且在没有终端的情况下会中止)。随后删除不再需要的 Volume。--yes - Queue 和 API 名称至少需要 3 个字符才能部署。 名称默认取函数名或类名,
而 Dev 会话不会检查名称,因此 可以通过
class SD,却会在rp flash dev时失败。请重命名,或传入rp flash deploy。name= - 每个 Endpoint 都会获得你的账户 API 密钥。 无论代码是否调用其他资源,
Dev 和 Deploy 都会将 放入每个 Endpoint 的模板环境变量中, 因此任何能够读取该模板的人都能读取该密钥。
RUNPOD_API_KEY - Dev 会话结束后很快重新运行同一个应用可能会失败。 Dev 会话会遗留模板,
平台最多保留约 90 分钟,因此再次对同一应用执行 可能因 “endpoint template names must be unique”而失败。请稍后重试,或重命名
rp flash dev。runpod.App
Resources
资源
- Setup & CLI: reference/setup-and-cli.md · API: reference/api.md · Patterns: reference/patterns.md · Examples: examples/
- SDK source and upstream examples: https://github.com/runpod/runpod-python ()
examples/apps/ - Docs: https://docs.runpod.io/sdk/overview · CLI: https://docs.runpod.io/sdk/cli · Migration: https://docs.runpod.io/sdk/migrate-from-flash
- 设置与 CLI:reference/setup-and-cli.md · API:reference/api.md · 模式:reference/patterns.md · 示例:examples/
- SDK 源码与上游示例:https://github.com/runpod/runpod-python ()
examples/apps/ - 文档:https://docs.runpod.io/sdk/overview · CLI:https://docs.runpod.io/sdk/cli · 迁移:https://docs.runpod.io/sdk/migrate-from-flash