roboflow-inference

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

For agents — source-of-truth: This skill is authored in
roboflow/computer-vision-skills
and shipped with the Roboflow plugin. If your client has loaded the plugin (you'll see
roboflow:<name>
skills in your available skills list), use those local skills — they're read fresh from disk every session. The same content served as MCP resources at
roboflow://skills/<name>/...
is a fallback for clients without the plugin and may lag this repo. Don't call
ReadMcpResourceTool
for
roboflow://skills/...
URIs when a local
roboflow:<name>
skill is available.

Tip: If you're connected to the Roboflow MCP server, prefer its inference tools over raw HTTP; auth is handled for OAuth and API-key connections. Use workflows_run
for a saved, published Workflow by URL slug (
workflow_id
); the workspace comes from the connection. For single-model calls use
models_infer
. Use
workflow_specs_validate
before saving or running a new specification, and
workflow_specs_run
to test an unpublished draft or run an authorized one-off.

对于 agent——权威来源： 此 skill 编写于
roboflow/computer-vision-skills
，并随 Roboflow plugin 一同提供。如果你的 client 已加载该 plugin（可用 skill 列表中会看到
roboflow:<name>
skills），请使用这些本地 skills——每个 session 都会从 disk 重新读取最新内容。以 MCP resources 形式通过
roboflow://skills/<name>/...
提供的相同内容，是不支持 plugin 的 client 的 fallback，并且可能落后于本 repo。当本地
roboflow:<name>
skill 可用时，不要为
roboflow://skills/...
URI 调用
ReadMcpResourceTool
。

提示： 如果你已连接到 Roboflow MCP server，请优先使用其 inference tools，而不是原始 HTTP；OAuth 和 API-key 连接的身份验证均由其处理。对于通过 URL slug（
workflow_id
）标识的已保存、已发布 Workflow，使用 workflows_run
；workspace 来自该连接。对于单模型调用，使用
models_infer
。保存或运行新 specification 前，使用
workflow_specs_validate
；测试未发布的 draft 或运行已授权的一次性 specification 时，使用
workflow_specs_run
。

Inference & Deployment

推理与部署

Prefer Workflows for integration code. Unlike a direct
models_infer
call, a workflow composes model + visualization + logic in one deployable unit, benefits from server-side optimizations, and keeps active learning and post-processing as a zero-friction addition — similar to the extensibility benefits of a microservice over a point-to-point call. Recommend a workflow by default; use
models_infer
when the user explicitly asks for direct inference or needs a quick one-off check. See workflows and workflow-templates.

Authoring Workflows. Save reusable Workflows on the Roboflow platform and run published versions by identifier. Choose the authoring path from the user's request and existing session context:
Mode A: Agent-driven (MCP, in-session). Use direct authoring tools for straightforward Workflows, grounding the design with
workflow_blocks_list
/
workflow_blocks_get_schema
. Use
agent_chat
for complex construction. Validate with
workflow_specs_validate
before saving or running. Edits saved by
agent_chat
are drafts; test them with
workflow_specs_run
before publishing when execution is authorized.
Mode B: Platform-driven (Roboflow app + in-app agent). Use when the user wants visual interaction or the client cannot complete the task. Send the Workflow's returned
app_url
, or use
roboflow-product-navigation
for the correct builder link. The user can review, edit, and test there.
workflows_run
executes the latest published version, not an agent's draft. Use
agent_workflow_publish
only when publishing is authorized. Draft testing does not require publishing. See workflows "Authoring & Deployment" for the full flow.

For live video (webcam, RTSP, file): the MCP
workflows_run
tool only handles single static images. For live video, present the user with three options (don't pick one silently): (A) WebRTC → serverless GPU, (B) WebRTC → local
inference server
, or (C) in-process
InferencePipeline
. They have different setup costs, dep sizes, and latency characteristics — surface a brief 1-line summary of each and let the user choose. See
roboflow://skills/roboflow-inference/workflows
("Video Stream" section) for full code and the comparison table.

集成代码应优先使用 Workflows。 与直接调用
models_infer
不同，Workflow 将 model + visualization + logic 组合为一个可部署单元，能够受益于 server 端优化，并可无缝加入 active learning 和 post-processing——其扩展性优势类似于 microservice 相比 point-to-point call 的优势。默认推荐 Workflow；仅当用户明确要求直接推理或需要快速临时检查时，使用
models_infer
。参见 workflows 和 workflow-templates。

编写 Workflows。 在 Roboflow platform 上保存可复用的 Workflow，并通过 identifier 运行已发布版本。根据用户请求和现有 session context 选择编写路径：
模式 A：Agent 驱动（MCP、session 内）。 对于 straightforward Workflows，使用 direct authoring tools，并通过
workflow_blocks_list
/
workflow_blocks_get_schema
获取设计依据。对于复杂构建，使用
agent_chat
。保存或运行前，使用
workflow_specs_validate
进行验证。由
agent_chat
保存的编辑内容属于 drafts；获得执行授权后，发布前应使用
workflow_specs_run
进行测试。
模式 B：平台驱动（Roboflow app + in-app agent）。 当用户需要可视化交互，或 client 无法完成任务时使用。将 Workflow 返回的
app_url
发送给用户，或使用
roboflow-product-navigation
获取正确的 builder 链接。用户可以在那里查看、编辑和测试 Workflow。
workflows_run
执行最新已发布的版本，而不是 agent 的 draft。仅在发布获得授权时使用
agent_workflow_publish
。测试 draft 不需要发布。完整流程参见 workflows 中的“编写与部署”。

对于实时视频（webcam、RTSP、file）： MCP
workflows_run
tool 仅支持单张静态图像。对于实时视频，请向用户提供 三个选项（不要擅自选择）：(A) WebRTC → serverless GPU，(B) WebRTC → 本地
inference server
，或 (C) 进程内
InferencePipeline
。它们具有不同的设置成本、依赖项大小和延迟特征——请简要提供每项的一行摘要，让用户自行选择。完整代码和对比表参见
roboflow://skills/roboflow-inference/workflows
（“Video Stream”部分）。

Deployment Options

部署选项

Option	Best For	Latency	Scaling	Cost Model	GPU
Serverless	Getting started, variable traffic	Low	Auto	Per-inference credit	Yes
Dedicated	Predictable workloads, low latency	Very low	Manual/autoscale	Per-hour credits	Optional
Self-hosted	Full control, edge	Hardware-dependent	Manual	Metered + infra cost	Optional
Batch Processing	Large offline datasets, videos	Async (minutes-hours)	Auto-provisioned	Per-job	Optional

选项	适用场景	延迟	扩缩容	成本模式	GPU
Serverless	快速入门、流量波动	低	自动	按次推理计 credit	是
Dedicated	负载可预测、低延迟	极低	手动/autoscale	按小时计 credit	可选
Self-hosted	完全控制、边缘部署	取决于硬件	手动	按量计费 + 基础设施成本	可选
Batch Processing	大型离线数据集、视频	异步（数分钟至数小时）	自动预置	按任务计费	可选

When to Use Which

何时使用

Serverless -- default choice. Zero setup, auto-scales, 20MB upload limit. Use
```
models_infer
```
or
```
workflows_run
```
MCP tools.
Dedicated -- need consistent latency, large models (Florence 2), or high throughput. Development and production tiers available. Subdomain:
```
<name>.roboflow.cloud
```
.
Self-hosted -- deploy Roboflow Inference via Docker on your own hardware (Jetson, cloud VMs, RPi). Same API surface as serverless -- just change
```
api_url
```
.
Batch Processing -- runs a Workflow on uploaded images/videos asynchronously. No real-time requirement. Results delivered as JSON.
Real-time video (webcam/RTSP/file) -- three deployment options; ask the user which one before writing code:
- (A) Serverless GPU + WebRTC — zero setup, just an API key; per-minute credits, plan-tiered (
```
webrtc-gpu-small/medium/large
```
  ).
- (B) Local inference server + WebRTC —
```
pip install inference-cli && inference server start
```
  (Docker recommended); lowest latency, isolates the heavy CV/model deps inside the server.
- (C)
  InferencePipeline
  in-process —
```
pip install inference
```
  in a venv (prefer
```
uv
```
  ); runs the workflow loop directly in the user's Python process, no separate server. Heavy deps (torch, opencv, onnxruntime) install locally.
All three have a slower first run (model download / warmup) before subsequent runs hit cached state — tell the user this so they don't think the script is hung.
- See
```
roboflow://skills/roboflow-inference/workflows
```
  ("Video Stream" section) for full code and a comparison table.

Serverless——默认选项。无需设置、自动扩缩容，上传大小限制为 20MB。使用
```
models_infer
```
或
```
workflows_run
```
MCP tools。
Dedicated——需要稳定延迟、大型模型（Florence 2）或高吞吐量时使用。提供开发版和生产版层级。子域名：
```
<name>.roboflow.cloud
```
。
Self-hosted——通过 Docker 将 Roboflow Inference 部署到自己的硬件上（Jetson、云端 VM、RPi）。其 API surface 与 serverless 相同，只需更改
```
api_url
```
。
Batch Processing——异步运行一个 Workflow，对上传的图像或视频进行处理。无需满足实时要求。结果以 JSON 形式交付。
实时视频（webcam/RTSP/file）——共有三种部署选项；编写代码前请先让用户选择：
- (A) Serverless GPU + WebRTC——无需设置，只需 API key；按分钟计费 credit，并按套餐层级提供（
```
webrtc-gpu-small/medium/large
```
  ）。
- (B) 本地 inference server + WebRTC——
```
pip install inference-cli && inference server start
```
  （推荐使用 Docker）；延迟最低，并将重量级 CV/model 依赖项隔离在 server 内。
- (C) 进程内
  InferencePipeline
  ——在 venv 中执行
```
pip install inference
```
  （推荐使用
```
uv
```
  ）；直接在用户的 Python 进程中运行 workflow loop，无需单独的 server。重量级依赖项（torch、opencv、onnxruntime）安装在本地。
这三种方式首次运行都会较慢（需要下载 model 或进行 warmup），之后才会进入 cached state——请告知用户这一点，避免其误以为脚本已经卡死。
- 完整代码和对比表参见
```
roboflow://skills/roboflow-inference/workflows
```
  （“Video Stream”部分）。

MCP Tools for Inference

用于推理的 MCP Tools

Tool	Purpose
`models_list`	List trained models for a project
`models_get`	Get details for a trained model
`models_infer`	Run single-model inference on one image via serverless API
`trainings_create`	Start training a model on a dataset version
`trainings_list`	Find runs and their `trainingId` for a project and version
`trainings_get`	Check a known run with `project_id` , `version_number` , and `training_id` ; for NAS model details use the paginated path in the training skill
`workflows_run`	Run the published Workflow by `workflow_id` (URL slug); workspace comes from the connection. Optional `parameters` .
`workflow_specs_validate`	Validate an inline workflow spec without running it — use before any inline run.
`workflow_specs_run`	Test an unpublished draft or run an authorized one-off from an inline specification

Tool	用途
`models_list`	列出 project 的已训练模型
`models_get`	获取已训练模型的详细信息
`models_infer`	通过 serverless API 对一张图像执行单模型推理
`trainings_create`	在 dataset version 上启动模型训练
`trainings_list`	查找 project 和 version 的训练运行及其 `trainingId`
`trainings_get`	使用 `project_id` 、 `version_number` 和 `training_id` 检查已知运行；如需 NAS 模型详情，请使用 training skill 中带分页的 path
`workflows_run`	通过 `workflow_id` （URL slug）运行已发布的 Workflow；workspace 来自该连接。可选传入 `parameters` 。
`workflow_specs_validate`	验证 inline workflow spec，但不运行——在任何 inline run 之前使用。
`workflow_specs_run`	从 inline specification 测试未发布的 draft，或运行已授权的一次性 specification

Local tooling: when MCP isn't enough

本地工具：MCP 不足以满足需求时

For most operations, prefer the Roboflow MCP tools above — they handle auth and need nothing installed locally. Reach for local Python packages only for the gaps: integration scripts (

inference-sdk

), Batch Processing / Data Staging (

inference-cli

), the self-hosted server (

inference-cli

), and asset scripts that need typed Python objects.

See

local-tooling

for what to install for which use case, the recommended

uv

-based env setup, conda / venv fallbacks, and common pitfalls.

对于大多数操作，请优先使用上述 Roboflow MCP tools——它们能够处理身份验证，且本地无需安装任何内容。仅在以下场景中使用本地 Python packages：集成脚本（

inference-sdk

）、Batch Processing / Data Staging（

inference-cli

）、self-hosted server（

inference-cli

），以及需要使用类型化 Python 对象的 asset scripts。

有关各使用场景的安装内容、推荐的基于

uv

的环境设置、conda / venv fallback 和常见问题，请参阅

local-tooling

。

Response Shapes by Task

不同任务的响应结构

For canonical response shapes (object detection, classification, segmentation, keypoint) with all fields including

class_id

detection_id

class_confidence

, see

roboflow://skills/roboflow-api-reference/inference

有关包含所有字段的标准响应结构（object detection、classification、segmentation、keypoint），包括

class_id

、

detection_id

、

class_confidence

，请参阅

roboflow://skills/roboflow-api-reference/inference

。

Large Response Handling

大型响应的处理

Instance segmentation
points
arrays are the main culprit for bloated responses. Each detection includes a polygon with potentially hundreds of coordinate pairs. A single image with many detections can return megabytes of JSON.

Mitigation strategies:

Use Workflows instead of direct inference -- add a polygon simplification or property extraction block to reduce output before it reaches the client
Filter classes -- use
```
class_filter
```
to only return classes you need
Raise confidence threshold -- fewer detections = smaller response
Post-process -- if consuming raw responses, drop or simplify the
```
points
```
array when you only need bounding boxes
Avoid returning raw segmentation results through LLM context -- extract only the fields you need (class counts, bounding boxes) and discard polygon data

Workflow image outputs are a second culprit. Visualization blocks (bounding box, polygon, mask, label, halo, …) emit rendered images as base64-encoded blobs inside the response — a 720p annotated frame is hundreds of KB of JSON-escaped string. When you call

workflows_run

workflow_specs_run

via MCP, this routinely overflows the tool-result token budget. Decode every image-shaped output (

{"type": "base64", "value": "..."}

) and write it to disk instead of carrying it through agent context. Don't hard-code field names — the output keys are whatever the workflow author declared via

JsonField

; iterate

output.keys()

and shape-check.

实例分割的
points
数组是响应膨胀的主要元凶。每次检测都包含一个多边形，该多边形可能具有数百对坐标。单张包含大量检测结果的图像可能返回数 MB 的 JSON。

缓解策略：

使用 Workflows 代替直接推理——添加 polygon simplification 或 property extraction block，在输出到达 client 前缩减数据量
筛选 classes——使用
```
class_filter
```
，仅返回所需 classes
提高置信度阈值——检测结果越少，响应越小
后处理——如果直接消费原始响应，而只需要 bounding boxes，请删除或简化
```
points
```
数组
避免通过 LLM context 返回原始分割结果——仅提取所需字段（如 class counts、bounding boxes），并丢弃 polygon 数据

Workflow 图像输出是另一个主要原因。 可视化 blocks（bounding box、polygon、mask、label、halo、…）会在 response 中输出渲染后的图像，并以 base64 编码 blob 的形式包含在内——一张 720p 标注帧会形成数百 KB 的 JSON 转义字符串。通过 MCP 调用

workflows_run

workflow_specs_run

时，这通常会超出 tool-result token 预算。请解码每个 image-shaped output（

{"type": "base64", "value": "..."}

）并将其写入 disk，不要将其带入 agent context。不要硬编码字段名称——output keys 是 workflow 作者通过

JsonField

声明的内容；请遍历

output.keys()

并执行 shape-check。

Batch Processing

For the full recipe (staging, jobs, monitoring, results, MCP tools), see the
roboflow-batch-processing
skill; this section is only the deployment comparison.

What it is. A Roboflow-managed cloud service that runs a Workflow over a batch of images or videos asynchronously, provisioning the infrastructure for you. "Ideal for asynchronously processing large amounts of data." — Roboflow docs.

Problem it solves. Bulk inference over thousands to millions of files without standing up your own GPUs, queues, or autoscaler. You hand Roboflow a Workflow plus a batch of inputs, pay per job, and get JSON results back when the job finishes.

Pick it when the data is stored (not live), per-file cost matters more than per-file latency, and minutes-to-hours per job is acceptable. Pick something else when you need real-time per-request results (use Serverless or Dedicated) or air-gapped/on-prem processing (use Self-hosted).

Surfaces: Roboflow web UI,

inference rf-cloud

CLI, and REST API.

有关完整流程（staging、jobs、monitoring、results、MCP tools），请参阅
roboflow-batch-processing
skill；本节仅介绍部署选项对比。

它是什么。 一项由 Roboflow 管理的 cloud service，可异步对一批图像或视频运行 Workflow，并为你自动预置基础设施。“非常适合异步处理大量数据。”——Roboflow docs

它解决的问题。 无需自行搭建 GPU、队列或 autoscaler，即可对数千乃至数百万个文件执行批量推理。你只需向 Roboflow 提交一个 Workflow 和一批输入，按 job 付费，并在 job 完成后获得 JSON 结果。

适用情况。 数据已存储而非实时产生、单文件成本比单文件延迟更重要，并且每个 job 耗时数分钟至数小时均可接受时，选择此方案。如果需要实时的逐请求结果（使用 Serverless 或 Dedicated），或者需要 air-gapped/on-prem 处理（使用 Self-hosted），请选择其他方案。

接入方式：Roboflow web UI、

inference rf-cloud

CLI 和 REST API。

Flow

流程

Have a saved Workflow in your workspace.
Provide inputs: an Asset Library selection (the platform stages it for you), or stage external files yourself as a Data Staging batch (local directory, JSONL of signed URLs, or cloud-storage path on S3 / GCS / Azure).
Submit a job referencing the Workflow + input batch; choose CPU or GPU.
Monitor — poll job status or register a webhook.
Export the output batch as JSON.

在 workspace 中拥有一个已保存的 Workflow。
提供输入：选择一个 Asset Library（platform 会为你完成 staging），或者自行将外部文件作为 Data Staging batch 进行 staging（本地目录、包含 signed URLs 的 JSONL，或 S3 / GCS / Azure 上的 cloud-storage path）。
提交一个引用 Workflow + input batch 的 job；选择 CPU 或 GPU。
监控——轮询 job 状态或注册 webhook。
将 output batch 导出为 JSON。

CLI

The

inference rf-cloud

CLI exposes two subcommand groups:

data-staging

(manage input/output batches) and

batch-processing

(submit and monitor jobs). Run any command with

--help

for the full option list.

Minimal end-to-end:

bash

undefined

inference rf-cloud

CLI 提供两个 subcommand group：

data-staging

（管理 input/output batch）和

batch-processing

（提交和监控 job）。运行任意命令并添加

--help

，即可查看完整的选项列表。

最小端到端示例：

bash

undefined

Stage images

inference rf-cloud data-staging create-batch-of-images
--images-dir ./my-images --batch-id my-batch

Submit

inference rf-cloud batch-processing process-images-with-workflow
--workflow-id my-workflow --batch-id my-batch

-> prints JOB_ID

Monitor

inference rf-cloud batch-processing show-job-details --job-id JOB_ID

Export results

inference rf-cloud data-staging export-batch
--target-dir ./results --batch-id OUTPUT_BATCH_ID


**Data Staging commands** — see [`batch-staging`](batch-staging.md) for nuances (data sources, JSONL reference format, multipart batches, webhook notifications):

| Command | Purpose |
|---|---|
| `data-staging list-batches` | List staging batches in the workspace |
| `data-staging create-batch-of-images` | Create an input batch from a local directory, signed-URL JSONL, or cloud-storage path |
| `data-staging create-batch-of-videos` | Same as above, but for video files |
| `data-staging show-batch-details` | Show metadata for a single batch |
| `data-staging list-batch-content` | List file URLs in a batch (filter by part, write JSONL) |
| `data-staging list-ingest-details` | Per-shard ingest status for debugging URL ingests |
| `data-staging export-batch` | Download all files from a batch (e.g. job outputs) to a local directory |

**Batch Processing (job) commands** — see [`batch-jobs`](batch-jobs.md) for nuances (compute configuration, workflow parameters, image-output persistence, aggregation format, video FPS, restarts, TRT compilation):

| Command | Purpose |
|---|---|
| `batch-processing list-jobs` | List jobs in the workspace |
| `batch-processing show-job-details` | Show stages and current status of a single job |
| `batch-processing process-images-with-workflow` | Submit an image-batch job |
| `batch-processing process-videos-with-workflow` | Submit a video-batch job |
| `batch-processing fetch-logs` | Fetch job logs (filter by severity, write JSONL) |
| `batch-processing abort-job` | Terminate a running job |
| `batch-processing restart-job` | Restart a failed job (optionally with new compute settings) |
| `batch-processing trt-compile` | Compile a model to TensorRT for one or more NVIDIA devices |

inference rf-cloud data-staging export-batch
--target-dir ./results --batch-id OUTPUT_BATCH_ID


**Data Staging commands**——有关数据源、JSONL reference 格式、multipart batch 和 webhook notification 等细节，请参阅 [`batch-staging`](batch-staging.md)：

| Command | 用途 |
|---|---|
| `data-staging list-batches` | 列出 workspace 中的 staging batch |
| `data-staging create-batch-of-images` | 从本地目录、signed-URL JSONL 或 cloud-storage path 创建 input batch |
| `data-staging create-batch-of-videos` | 与上述命令相同，但用于 video 文件 |
| `data-staging show-batch-details` | 显示单个 batch 的 metadata |
| `data-staging list-batch-content` | 列出 batch 中的 file URL（可按 part 筛选并写入 JSONL） |
| `data-staging list-ingest-details` | 查看每个 shard 的 ingest 状态，用于排查 URL ingest 问题 |
| `data-staging export-batch` | 将 batch 中的所有文件（例如 job outputs）下载到本地目录 |

**Batch Processing（job）commands**——有关 compute configuration、workflow parameters、image-output persistence、aggregation format、video FPS、restarts 和 TRT compilation 等细节，请参阅 [`batch-jobs`](batch-jobs.md)：

| Command | 用途 |
|---|---|
| `batch-processing list-jobs` | 列出 workspace 中的 job |
| `batch-processing show-job-details` | 显示单个 job 的 stages 和当前状态 |
| `batch-processing process-images-with-workflow` | 提交 image-batch job |
| `batch-processing process-videos-with-workflow` | 提交 video-batch job |
| `batch-processing fetch-logs` | 获取 job logs（可按 severity 筛选并写入 JSONL） |
| `batch-processing abort-job` | 终止正在运行的 job |
| `batch-processing restart-job` | 重启失败的 job（可选择使用新的 compute settings） |
| `batch-processing trt-compile` | 将 model 编译为 TensorRT，以供一个或多个 NVIDIA device 使用 |

Notes and constraints

说明与限制

Async only — minutes-to-hours latency depending on volume and hardware. Not for real-time.
Pricing: per job; GPU jobs cost more than CPU. See
```
roboflow-plans-and-pricing
```
.
Image-references ingest requires signed URLs from trusted sources; arbitrary public URLs are rejected — stage to a local directory or cloud-storage path instead.

Full reference: Roboflow Batch Processing docs.

仅支持异步——延迟取决于数据量和硬件，可能从数分钟到数小时不等。不适用于实时场景。
定价：按 job 收费；GPU job 的成本高于 CPU job。请参阅
```
roboflow-plans-and-pricing
```
。
Image-references ingest 要求使用来自可信来源的 signed URLs；任意 public URL 均会被拒绝——请改为 stage 到本地目录或 cloud-storage path。

完整参考：Roboflow Batch Processing docs。