Back to Details

deepstream-sop

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

DeepStream SOP Inference Microservice Skill

DeepStream SOP Inference Microservice Skill

This skill guides AI coding assistants in building, extending, and debugging the NVIDIA DeepStream SOP (Standard Operating Procedure) Inference Microservice — a GPU-accelerated pipeline for temporal action detection and VLM-based SOP compliance monitoring on industrial video feeds.
Reference repository: https://github.com/NVIDIA/sop-monitoring-blueprints/tree/main/microservices/sop-inference-bp Local reference code: 
sop-inference-bp/
directory (from a local clone of the repository)
本skill指导AI编码助手构建、扩展和调试 NVIDIA DeepStream SOP（标准操作流程）推理微服务—— 一款用于工业视频流的时序动作检测和基于VLM的SOP合规监控的GPU加速流水线。
参考仓库：https://github.com/NVIDIA/sop-monitoring-blueprints/tree/main/microservices/sop-inference-bp 本地参考代码：
sop-inference-bp/
目录（仓库本地克隆后的路径）

Models

模型

Model-agnostic at both inference stages — swap via env var (and Triton dir for GEBD).
StageRoleModel classDefaultSwap via
Stage 1 (CV)Per-frame boundary scoring → chunk segmentationGeneric Event Boundary Detection (GEBD)DDM (MCG-NJU/DDM) via Triton Python backendReplace 
triton_model_repo/<model>/
+ 
DDM_MODEL_PATH
(§ 5)
Stage 3 (VLM)Per-chunk action classificationVision-language model via vLLMCosmos Reason 1 7B (Reason 2 also supported)Set 
VLLM_MODEL_PATH
to a different HF ID or local path
"GEBD" = swappable Stage-1 slot; "DDM" = the default architecture (terms used interchangeably).
Chunking is selectable per request (§ 2): default 
ddm-net
uses GEBD; 
uniform
produces fixed-length chunks and bypasses Stage-1 GEBD (§ 3, § 6). DDM temporal window is configurable via 
FRAMES_PER_SIDE
/ 
SEQUENCE_BATCH
(§ 4, § 5), with optional TensorRT (§ 5).
两个推理阶段均支持模型无关性——通过环境变量（GEBD模型还需配合Triton目录）替换模型。
阶段作用模型类别默认模型替换方式
阶段1（计算机视觉）逐帧边界评分 → 视频块分割通用事件边界检测（GEBD）DDM（MCG-NJU/DDM），通过Triton Python后端部署替换
triton_model_repo/<model>/
目录 + 设置
DDM_MODEL_PATH
（第5节）
阶段3（视觉语言模型）视频块动作分类基于vLLM的视觉语言模型Cosmos Reason 1 7B（也支持Reason 2）将
VLLM_MODEL_PATH
设置为不同的HF模型ID或本地路径
"GEBD"指可替换的阶段1模块；"DDM"是默认架构（两个术语可互换使用）。
视频块分割方式可按请求选择（第2节）：默认的
ddm-net
使用GEBD；
uniform
方式生成固定长度的视频块，且绕过阶段1的GEBD（第3、6节）。DDM的时间窗口可通过
FRAMES_PER_SIDE
/
SEQUENCE_BATCH
配置（第4、5节），还可选择启用TensorRT（第5节）。

Architecture Overview

架构概述

Runs in a Docker container (
nvds-action-sop
) alongside a Kafka container. Full diagram: 
references/sop_architecture.svg
.
Data flow through the 4-stage 
SOPVideoProcessor
pipeline (per-request):
Input Sources                    Docker Container: nvds-action-sop
─────────────                    ──────────────────────────────────────────────────
Video Files ──┐                  FastAPI Server (port 8300)
RTSP Streams ─┤── base64/       ├─ /v1/chat/completions → SOPProcessManager
Basler Camera ┘   file/rtsp/       │
                  camera           │ ModelInitializer: VLM first, then DDM dummy pipeline
                                   │ 4 Thread Pools: cv(32), clip(32), vlm(64), vlm_req(64)
                                   │
                                   ▼ SOPVideoProcessor (per-request)
                                   ┌────────────────────────────────────────────────┐
                                   │ Stage 1: DeepStream Pipeline (GPU)             │
                                   │   Source → nvstreammux → tee1                  │
                                   │    ├─[inference] queue1 → nvdspreprocess       │
                                   │    │  → nvinferserver (Triton CAPI + DDM)      │
                                   │    │  → InferOutputTensorParser → score_queue  │
                                   │    ├─[frames]  queue3 → nvvideoconvert         │
                                   │    │  → capsfilter → appsink                   │
                                   │    │  → DecodedFrameRetriever → frame_queue    │
                                   │    └─[RTSP out] queue → convert → H.264 enc    │  (optional, § 18)
                                   │       → rtppay → udpsink → RTSPServer (§ 18)   │  opt-in only
                                   │              │ boundary scores                 │
                                   │              ▼                                 │
                                   │ Stage 2: Clip Post-Process                     │
                                   │   Boundary detection → chunk segmentation      │
                                   │              │ video frames + timestamps        │
                                   │              ▼                                 │
                                   │ Stage 3: VLM Inference                         │
                                   │   Embedded vLLM (Cosmos Reason 1/2)            │
                                   │   Frame sampling at VLM_FPS → classification   │
                                   │              │ action labels                    │
                                   │              ▼                                 │
                                   │ Stage 4: SOP Checker                           │
                                   │   Sequence validation → missing/misordered     │
                                   │              │ chunk results                    │
                                   │              ▼                                 │
                                   │         final_queue                            │
                                   └────────────────────────────────────────────────┘
                                          │
Output                                    ▼
──────                             ┌─────────────────┐
SSE Stream (chat.completion.chunk) │ Kafka Messages   │
Non-streaming (chat.completion)    │ (JSON/Protobuf)  │
Prometheus metrics (/v1/metrics)   └────────┬────────┘
                                            ▼
                                   Docker Container: kafka
                                   (apache/kafka:3.7.0)
服务运行在Docker容器（
nvds-action-sop
）中，与Kafka容器协同工作。完整架构图：
references/sop_architecture.svg
。
4阶段
SOPVideoProcessor
流水线的数据流（单请求维度）：
输入源                    Docker容器: nvds-action-sop
─────────────                    ──────────────────────────────────────────────────
视频文件 ──┐                  FastAPI服务器（端口8300）
RTSP流 ─┤── base64/       ├─ /v1/chat/completions → SOPProcessManager
Basler相机 ┘   file/rtsp/       │
                  camera           │ ModelInitializer：先预热VLM，再启动DDM虚拟流水线
                                   │ 4个线程池: cv(32), clip(32), vlm(64), vlm_req(64)
                                   │
                                   ▼ SOPVideoProcessor（单请求实例）
                                   ┌────────────────────────────────────────────────┐
                                   │ 阶段1: DeepStream流水线（GPU加速）             │
                                   │   源 → nvstreammux → tee1                  │
                                   │    ├─[推理] queue1 → nvdspreprocess       │
                                   │    │  → nvinferserver（Triton CAPI + DDM）      │
                                   │    │  → InferOutputTensorParser → score_queue  │
                                   │    ├─[帧数据]  queue3 → nvvideoconvert         │
                                   │    │  → capsfilter → appsink                   │
                                   │    │  → DecodedFrameRetriever → frame_queue    │
                                   │    └─[RTSP输出] queue → convert → H.264 enc    │  （可选，第18节）
                                   │       → rtppay → udpsink → RTSPServer（第18节）   │  仅按需启用
                                   │              │ 边界评分                 │
                                   │              ▼                                 │
                                   │ 阶段2: 视频块后处理                     │
                                   │   边界检测 → 视频块分段      │
                                   │              │ 视频帧 + 时间戳        │
                                   │              ▼                                 │
                                   │ 阶段3: VLM推理                         │
                                   │   内置vLLM（Cosmos Reason 1/2）            │
                                   │   按VLM_FPS采样帧数据 → 分类   │
                                   │              │ 动作标签                    │
                                   │              ▼                                 │
                                   │ 阶段4: SOP检查器                           │
                                   │   序列验证 → 检测缺失/乱序步骤     │
                                   │              │ 视频块结果                    │
                                   │              ▼                                 │
                                   │         final_queue                            │
                                   └────────────────────────────────────────────────┘
                                          │
输出                                    ▼
──────                             ┌─────────────────┐
SSE流（chat.completion.chunk） │ Kafka消息   │
非流式输出（chat.completion）    │ （JSON/Protobuf格式）  │
Prometheus指标（/v1/metrics）   └────────┬────────┘
                                            ▼
                                   Docker容器: kafka
                                   (apache/kafka:3.7.0)

Section Index

章节索引

Each section is a standalone file in 
references/
— load only what your task needs.
§FileResponsibility
1
skill_01_fastapi_endpoints.md
FastAPI endpoints, server init, Prometheus metrics
2
skill_02_pydantic_schemas.md
Request/response Pydantic models (
api_types.py
)
3
skill_03_deepstream_pipeline.md
DeepStream pyservicemaker pipeline, tensor parser, dummy pipeline
4
skill_04_config_templates.md
nvdspreprocess / nvinferserver config templates + rendering
5
skill_05_triton_ddm_model.md
Triton model repo, config.pbtxt, model.py, ddm_net.py
5b
skill_05b_custom_postprocess.md
C++ postprocess plugin, Makefile, IOptions API
6
skill_06_sop_process_manager.md
SOPProcessManager, SOPVideoProcessor, VLLMInference, Kafka
6b
skill_06b_sop_checker.md
SOP sequence and checker compliance: MissingNumberDetector, SopCheckerCache, SopCheckerRequest/Response
7
skill_07_sse_streaming.md
SSE generator, stream response formatting, dummy test mode
8
skill_08_basler_camera.md
Basler camera support, Pylon SDK, emulation, formats
9
skill_09_docker_build_deploy.md
Docker build, deploy, .env configuration
10
skill_10_test_suite.md
Test suite coverage, assertions, running tests
11
skill_11_env_variables.md
All environment variables reference
12
skill_12_evaluation_workflow.md
End-to-end eval workflow: static checks, build, launch, tests, API/camera/Kafka checks, report
13
skill_13_verification_curl.md
Verification steps and curl examples
14
skill_14_implementation_checklist.md
Implementation checklist: file copy list, generated files, Docker prereqs, verification
15
skill_15_latency_measurement.md
TTFC and C2C latency measurement for file input via SSE streaming
16
skill_16_message_schema.md
Kafka message schema selection (
JSON
default vs 
NvProtoSchema
) and extending messages with custom data
17
skill_17_camera_latency_measurement.md
Camera / live-stream chunk_e2e latency measurement using internal pipeline timestamps
18
skill_18_rtsp_streaming_output.md
OPT-IN RTSP streaming output: 
tee1
-tap re-stream, 
RTSPStreamingServer
, 
SW_ENCODER
toggle. Generate only when user explicitly requests RTSP
For end-to-end evaluation, read § 12 first; load build/test/curl/latency/camera/Kafka as needed.
§ 18 is opt-in — generate only when the user explicitly requests RTSP output; otherwise skip § 18 and the 
RTSP_*
rules below.
每个章节对应
references/
目录下的独立文件——仅加载任务所需的内容即可。
章节文件职责
1
skill_01_fastapi_endpoints.md
FastAPI接口、服务器初始化、Prometheus指标
2
skill_02_pydantic_schemas.md
请求/响应Pydantic模型（
api_types.py
）
3
skill_03_deepstream_pipeline.md
DeepStream pyservicemaker流水线、张量解析器、虚拟流水线
4
skill_04_config_templates.md
nvdspreprocess / nvinferserver配置模板及渲染逻辑
5
skill_05_triton_ddm_model.md
Triton模型仓库、config.pbtxt、model.py、ddm_net.py
5b
skill_05b_custom_postprocess.md
C++后处理插件、Makefile、IOptions API
6
skill_06_sop_process_manager.md
SOPProcessManager、SOPVideoProcessor、VLLMInference、Kafka集成
6b
skill_06b_sop_checker.md
SOP序列与合规检查：MissingNumberDetector、SopCheckerCache、SopCheckerRequest/Response
7
skill_07_sse_streaming.md
SSE生成器、流响应格式化、虚拟测试模式
8
skill_08_basler_camera.md
Basler相机支持、Pylon SDK、仿真、格式适配
9
skill_09_docker_build_deploy.md
Docker构建、部署、.env配置
10
skill_10_test_suite.md
测试套件覆盖范围、断言逻辑、测试运行方法
11
skill_11_env_variables.md
所有环境变量参考
12
skill_12_evaluation_workflow.md
端到端评估流程：静态检查、构建、启动、测试、API/相机/Kafka校验、报告生成
13
skill_13_verification_curl.md
验证步骤与curl示例
14
skill_14_implementation_checklist.md
实施检查清单：文件复制列表、生成文件、Docker前置条件、验证步骤
15
skill_15_latency_measurement.md
基于SSE流式传输的文件输入TTFC与C2C延迟测量
16
skill_16_message_schema.md
Kafka消息 schema选择（默认
JSON
vs 
NvProtoSchema
）及自定义数据扩展
17
skill_17_camera_latency_measurement.md
利用内部流水线时间戳测量相机/直播流的chunk_e2e延迟
18
skill_18_rtsp_streaming_output.md
按需启用RTSP流式输出：
tee1
分支重流、
RTSPStreamingServer
、
SW_ENCODER
开关。仅当用户明确请求RTSP时才生成相关内容
如需进行端到端评估，请先阅读第12节；按需加载构建/测试/curl/延迟/相机/Kafka相关章节。
第18节为按需内容——仅当用户明确请求RTSP输出时才生成相关内容；否则跳过第18节及以下
RTSP_*
规则。

Key Files Map

核心文件映射

The full source-to-target file mapping lives in 
skill_14_implementation_checklist.md
:
  • Files copied verbatim from 
    references/
    (non-trivial algorithms — cycle detection, qwen_vl_utils preprocessing, DeepStream 
    IOptions
    API, protobuf sources) with the rationale per file.
  • Files copied as adaptable templates (Dockerfile, compose.yaml, Triton config and 
    model.py
    , 
    ddm_net.py
    , Pylon emulation config, etc.).
  • Files generated from skill sections — each annotated with the Critical Rules below that the generation must follow exactly.
  • Docker build prerequisites and post-build verification checklist.
Config files (
nvds_preprocess_template.txt
, 
nvds_inference_template.txt
, 
vlm_prompts.txt
) are used as-is from 
configs/
.
When 
skill_06b
is loaded, read 
configs/actions.json
from the project root and run the § 6b-G generation workflow to produce 
nvds_action_detector/missing_number_detector.py
. If 
configs/actions.json
is absent or invalid, fall back to copying the reference file.
完整的源文件到目标文件映射请查看 
skill_14_implementation_checklist.md
：
  • 直接复制的文件：来自
    references/
    目录（包含非 trivial 算法——循环检测、qwen_vl_utils预处理、DeepStream 
    IOptions
    API、protobuf源文件），每个文件均附有复制理由。
  • 可适配的模板文件：Dockerfile、compose.yaml、Triton配置与
    model.py
    、
    ddm_net.py
    、Pylon仿真配置等。
  • 从skill章节生成的文件——每个文件均标注了生成时必须严格遵循的以下关键规则。
  • Docker构建前置条件与构建后验证清单。
配置文件（
nvds_preprocess_template.txt
、
nvds_inference_template.txt
、
vlm_prompts.txt
）直接使用
configs/
目录下的文件。
当加载
skill_06b
时，请读取项目根目录下的
configs/actions.json
并执行第6b-G生成流程，生成
nvds_action_detector/missing_number_detector.py
。若
configs/actions.json
不存在或无效，则回退到复制参考文件。

Critical Rules

关键规则

Each rule's full detail lives in the linked 
skill_NN_*.md
reference file.
TagRule summaryDetails in
MANAGER_INIT_IN_MAIN
SOPProcessManager
init in 
main()
before 
uvicorn.run()
— not inside 
lifespan()
skill_01_fastapi_endpoints.md
NAMED_KWARGS
create_video_processor()
uses named kwargs; camera args as separate kwargs
skill_06_sop_process_manager.md
LIVE_REQUIRES_STREAM_TRUE
stream: true
required for live inputs (RTSP / camera)
skill_08_basler_camera.md
VLM_DISABLED_DISABLES_SOP_CHECKER
DISABLE_VLM_INFERENCE=true
auto-disables SOP checker at import
skill_06_sop_process_manager.md
CHUNK_PARAMS_MAX_LENGTH
ChunkParams.max_length_sec
= 10s internal; 60s API default
skill_06_sop_process_manager.md
VLM_WARMUP_BEFORE_DDM
ModelInitializer
: VLM warmup FIRST, then CV dummy pipeline
skill_06_sop_process_manager.md
VLM_WARMUP_3_FRAMES
VLM warmup needs 3 frames (
torch.zeros
) — Qwen3VL hangs on < 3
skill_06_sop_process_manager.md
THREAD_POOL_SIZES
4 thread pools: 
cv
(32), 
clip
(32), 
vlm_inference
(64), 
vlm_request
(64)
skill_06_sop_process_manager.md
MEDIA_INFO_PYMEDIAINFO
Media info via 
pymediainfo
; live sources set fps=30/duration=inf directly
skill_06_sop_process_manager.md
CAMERA_EMULATION_PYLON_CAMEMU
PYLON_CAMEMU=1
for camera emulation (serial 
0815-0000
)
skill_08_basler_camera.md
DEEPSTREAM_LIB_HIDE
DeepStream lib hide trick: rename lib → lib.tmp during gst-plugin-pylon build
skill_08_basler_camera.md
VLM_REAL_GPU_FRAMES
VLM uses real GPU frames via 
DecodedFrameRetriever
; never 
torch.zeros
for inference
skill_06_sop_process_manager.md
BUFFER_RETRIEVER_STATIC_BASE
DecodedFrameRetriever
MUST inherit 
BufferRetriever
statically via 
super().__init__()
; runtime 
__class__.__bases__
mutation hangs 
pipeline.attach()
skill_06_sop_process_manager.md
FRAME_RETRIEVER_PRIORITY
create_inference_pipeline
: 
frame_retriever=
kwarg takes priority over 
frame_queue
skill_03_deepstream_pipeline.md
MUX_ORIGINAL_RESOLUTION
nvstreammux uses original resolution (not 224); pass 
mux_width/mux_height
from 
get_media_info()
(probe live RTSP for non-camera inputs; camera path unaffected)
skill_03_deepstream_pipeline.md
, 
skill_06_sop_process_manager.md
FILE_URI_NO_DOUBLE_PREFIX
create_inference_pipeline
file source: check 
file_path.startswith("file://")
before prepending — API passes 
file://
URLs directly
skill_03_deepstream_pipeline.md
CLEANUP_ON_DISCONNECT
Pipeline cleanup on client disconnect via 
trigger_stop_processors
in 
try/finally
skill_07_sse_streaming.md
UNIFIED_CLIP_POST_PROCESS
Unified 
clip_post_process()
for file + live; 
stop()
puts 
None
in 
_score_queue
skill_06_sop_process_manager.md
ABORT_INFLIGHT_VLM
Abort in-flight VLM requests on 
stop()
via 
llm.abort(req_id)
skill_06_sop_process_manager.md
LOGGER_EXPORT_GET_LOGGER
ds_logger.py
must export 
get_logger
skill_06_sop_process_manager.md
KAFKA_USE_CREATE_PRODUCER
Kafka: use 
create_producer()
from 
messager.py
; no 
Messager
class
skill_06_sop_process_manager.md
USER_PROMPT_PRIORITY
User request text takes priority over 
VLM_PROMPT_PATH
file; 
{"type":"text"}
in the request overrides the config-file prompt
skill_06_sop_process_manager.md
EVAL_USE_CONFIG_PROMPT
Eval/latency requests omit request text by default so the VLM uses 
VLM_PROMPT_PATH
skill_12_evaluation_workflow.md
, 
skill_13_verification_curl.md
, 
skill_15_latency_measurement.md
, 
skill_17_camera_latency_measurement.md
CHUNK_SCHEMA_FIELD_NAMES
Chunk schema: 
chunk_idx
, 
cv_boundary_score
, 
checker_result
; summary 
chunk_idx=-1
skill_06_sop_process_manager.md
SEQUENTIAL_FRAME_DRAIN
Drain 
decoded_frame_queue
(FIFO, shared across chunks) in a SINGLE thread and submit VLM per chunk incrementally; parallel drain steals frames → 0-frame chunks / wrong VLM input
skill_06_sop_process_manager.md
WALL_CLOCK_BEFORE_GPU
DecodedFrameRetriever.consume()
: capture 
wall_clock_entry = time.time()
BEFORE GPU dlpack; queue 3-tuple 
(timestamp, wall_clock_entry, tensor)
skill_06_sop_process_manager.md
, 
skill_17_camera_latency_measurement.md
CHUNK_E2E_PIPELINE_TIMESTAMPS
Write 
pipeline_chunk_end_timestamp
(last frame wall_clock) and 
pipeline_vlm_ready_timestamp
(
tm_e2e.now()
) into 
chunk_info
for camera latency (§ 17)
skill_06_sop_process_manager.md
, 
skill_17_camera_latency_measurement.md
VLM_INFERENCE_REQUIRED_KWARGS
Every 
VLLMInference.inference()
call must pass 
video_fps
, 
system_prompt
, 
max_completion_tokens
skill_06_sop_process_manager.md
UNIFORM_CHUNKING_BYPASSES_DDM
chunking_options.algorithm="uniform"
→ fixed-length chunks; 
create_inference_pipeline(uniform_chunk=True)
skips DDM but keeps 
tee1
fanout; Stage 2 uses 
uniform_clip_post_process
skill_02_pydantic_schemas.md
, 
skill_03_deepstream_pipeline.md
, 
skill_06_sop_process_manager.md
DDM_TEMPORAL_CONFIGURABLE
SLIDING_WINDOWS_SIZE = 2*FRAMES_PER_SIDE + SEQUENCE_BATCH
rendered into preprocess/nvinferserver (no hard-coded 18); Triton 
config.pbtxt
sequence dim 
-1
skill_04_config_templates.md
, 
skill_05_triton_ddm_model.md
DDM_TRT_OPTIONAL_PATH
DDM_TRT_OPTIMIZATION=true
runs DDM via TensorRT (per-thread contexts, fixed batch = SEQUENCE_BATCH); PyTorch fallback; never both. PyTorch is default
skill_05_triton_ddm_model.md
DDM_TRT_STREAM_ORDERING
DDMTensorRTEngine.infer()
: 
wait_stream(current)
→ 
execute_async_v3
→ 
torch.cuda.synchronize(device)
 (NOT per-stream). Per-stream sync leaves TRT aux-stream work in flight → gst-CV SIGSEGV (NVBug 6289256)
skill_05_triton_ddm_model.md
METADATA_LICENSE_FROM_FILE
/v1/metadata
reads 
licenseInfo
from 
DS_SOP_LICENSE_PATH
(default 
/opt/nvidia/nvds_sop/license.txt
); never hard-code license text
skill_01_fastapi_endpoints.md
CAMERA_EMULATION_FRAMES_RGB
Pylon emulation PNGs must be explicit 3-channel RGB (matches 
Emulation_0815-0000.pfs PixelFormat=RGB8Packed
); generate via 
nvvideoconvert ! videoconvert ! "video/x-raw,format=RGB" ! pngenc
skill_08_basler_camera.md
COMPOSE_ENV_PASSTHROUGH
docker compose
only substitutes 
${VAR}
references; every runtime env var must be explicitly listed under 
environment:
to reach the container.
skill_09_docker_build_deploy.md
The four 
RTSP_*
rules below apply only when the optional RTSP streaming-output feature (§ 18) is requested. They do not apply to the default build — skip them if the user did not ask for RTSP output.
| 
RTSP_OUTPUT_TAPS_TEE1
| RTSP output branch links from the existing 
tee1
(added after the main inference link) only when 
rtsp_port
is present. | 
skill_18_rtsp_streaming_output.md
| | 
RTSP_LEAKY_QUEUE_TINY
| RTSP branch queue must be 
leaky=2
+ tiny cap (
max-size-buffers=2
) to prevent backpressure and NVMM pool exhaustion. | 
skill_18_rtsp_streaming_output.md
| | 
RTSP_KEYINT_MAX_30
| RTSP H.264 encoder must set 
key-int-max=30
(and B-frames disabled) to allow downstream seeking. | 
skill_18_rtsp_streaming_output.md
| | 
RTSP_ENCODER_FALLBACK
| Select software/hardware H.264 encoder based on 
SW_ENCODER
with MJPEG fallback. | 
skill_18_rtsp_streaming_output.md
|
每条规则的详细说明请查看链接的
skill_NN_*.md
参考文件。
标签规则摘要详细说明位置
MANAGER_INIT_IN_MAIN
SOPProcessManager
需在
main()
中初始化，早于
uvicorn.run()
——不可在
lifespan()
内初始化
skill_01_fastapi_endpoints.md
NAMED_KWARGS
create_video_processor()
使用命名参数；相机参数作为独立的命名参数传递
skill_06_sop_process_manager.md
LIVE_REQUIRES_STREAM_TRUE
实时输入（RTSP / 相机）需设置
stream: true
skill_08_basler_camera.md
VLM_DISABLED_DISABLES_SOP_CHECKER
当
DISABLE_VLM_INFERENCE=true
时，会在导入阶段自动禁用SOP检查器
skill_06_sop_process_manager.md
CHUNK_PARAMS_MAX_LENGTH
ChunkParams.max_length_sec
内部限制为10秒；API默认值为60秒
skill_06_sop_process_manager.md
VLM_WARMUP_BEFORE_DDM
ModelInitializer
：先预热VLM，再启动计算机视觉虚拟流水线
skill_06_sop_process_manager.md
VLM_WARMUP_3_FRAMES
VLM预热需要3帧数据（
torch.zeros
）——Qwen3VL在少于3帧时会挂起
skill_06_sop_process_manager.md
THREAD_POOL_SIZES
4个线程池：
cv
(32), 
clip
(32), 
vlm_inference
(64), 
vlm_request
(64)
skill_06_sop_process_manager.md
MEDIA_INFO_PYMEDIAINFO
通过
pymediainfo
获取媒体信息；实时源直接设置fps=30/duration=inf
skill_06_sop_process_manager.md
CAMERA_EMULATION_PYLON_CAMEMU
设置
PYLON_CAMEMU=1
启用相机仿真（序列号
0815-0000
）
skill_08_basler_camera.md
DEEPSTREAM_LIB_HIDE
DeepStream库隐藏技巧：在构建gst-plugin-pylon期间，将库重命名为lib.tmp
skill_08_basler_camera.md
VLM_REAL_GPU_FRAMES
VLM通过
DecodedFrameRetriever
使用真实GPU帧数据；推理时绝不能使用
torch.zeros
skill_06_sop_process_manager.md
BUFFER_RETRIEVER_STATIC_BASE
DecodedFrameRetriever
必须通过
super().__init__()
静态继承
BufferRetriever
；运行时修改
__class__.__bases__
会导致
pipeline.attach()
挂起
skill_06_sop_process_manager.md
FRAME_RETRIEVER_PRIORITY
create_inference_pipeline
中：
frame_retriever=
参数优先级高于
frame_queue
skill_03_deepstream_pipeline.md
MUX_ORIGINAL_RESOLUTION
nvstreammux使用原始分辨率（而非224）；从
get_media_info()
传递
mux_width/mux_height
（非相机输入的RTSP流需探测分辨率；相机路径不受影响）
skill_03_deepstream_pipeline.md
, 
skill_06_sop_process_manager.md
FILE_URI_NO_DOUBLE_PREFIX
create_inference_pipeline
的文件源：在添加前缀前检查
file_path.startswith("file://")
——API会直接传递
file://
格式的URL
skill_03_deepstream_pipeline.md
CLEANUP_ON_DISCONNECT
在客户端断开连接时，通过
try/finally
中的
trigger_stop_processors
清理流水线
skill_07_sse_streaming.md
UNIFIED_CLIP_POST_PROCESS
文件+实时场景共用统一的
clip_post_process()
；
stop()
会在
_score_queue
中放入
None
skill_06_sop_process_manager.md
ABORT_INFLIGHT_VLM
在调用
stop()
时，通过
llm.abort(req_id)
终止正在进行的VLM请求
skill_06_sop_process_manager.md
LOGGER_EXPORT_GET_LOGGER
ds_logger.py
必须导出
get_logger
方法
skill_06_sop_process_manager.md
KAFKA_USE_CREATE_PRODUCER
Kafka集成：使用
messager.py
中的
create_producer()
；禁止使用
Messager
类
skill_06_sop_process_manager.md
USER_PROMPT_PRIORITY
用户请求文本优先级高于
VLM_PROMPT_PATH
文件；请求中的
{"type":"text"}
会覆盖配置文件中的提示词
skill_06_sop_process_manager.md
EVAL_USE_CONFIG_PROMPT
评估/延迟请求默认不包含请求文本，以便VLM使用
VLM_PROMPT_PATH
中的提示词
skill_12_evaluation_workflow.md
, 
skill_13_verification_curl.md
, 
skill_15_latency_measurement.md
, 
skill_17_camera_latency_measurement.md
CHUNK_SCHEMA_FIELD_NAMES
视频块schema字段：
chunk_idx
, 
cv_boundary_score
, 
checker_result
；摘要的
chunk_idx=-1
skill_06_sop_process_manager.md
SEQUENTIAL_FRAME_DRAIN
在单个线程中按顺序消费
decoded_frame_queue
（FIFO，跨视频块共享），并逐视频块提交VLM请求；并行消费会导致帧被抢占，进而产生0帧视频块/错误的VLM输入
skill_06_sop_process_manager.md
WALL_CLOCK_BEFORE_GPU
DecodedFrameRetriever.consume()
：在GPU dlpack操作前捕获
wall_clock_entry = time.time()
；将三元组
(timestamp, wall_clock_entry, tensor)
放入队列
skill_06_sop_process_manager.md
, 
skill_17_camera_latency_measurement.md
CHUNK_E2E_PIPELINE_TIMESTAMPS
将
pipeline_chunk_end_timestamp
（最后一帧的wall_clock）和
pipeline_vlm_ready_timestamp
（
tm_e2e.now()
）写入
chunk_info
，用于相机延迟测量（第17节）
skill_06_sop_process_manager.md
, 
skill_17_camera_latency_measurement.md
VLM_INFERENCE_REQUIRED_KWARGS
每次调用
VLLMInference.inference()
必须传递
video_fps
, 
system_prompt
, 
max_completion_tokens
参数
skill_06_sop_process_manager.md
UNIFORM_CHUNKING_BYPASSES_DDM
当
chunking_options.algorithm="uniform"
时，生成固定长度视频块；
create_inference_pipeline(uniform_chunk=True)
会跳过DDM但保留
tee1
分支；阶段2使用
uniform_clip_post_process
skill_02_pydantic_schemas.md
, 
skill_03_deepstream_pipeline.md
, 
skill_06_sop_process_manager.md
DDM_TEMPORAL_CONFIGURABLE
SLIDING_WINDOWS_SIZE = 2*FRAMES_PER_SIDE + SEQUENCE_BATCH
会被渲染到预处理/nvinferserver配置中（不可硬编码为18）；Triton 
config.pbtxt
中的序列维度设为
-1
skill_04_config_templates.md
, 
skill_05_triton_ddm_model.md
DDM_TRT_OPTIONAL_PATH
当
DDM_TRT_OPTIMIZATION=true
时，通过TensorRT运行DDM（单线程上下文，固定批次=SEQUENCE_BATCH）；默认使用PyTorch作为回退方案；不可同时启用两者
skill_05_triton_ddm_model.md
DDM_TRT_STREAM_ORDERING
DDMTensorRTEngine.infer()
：
wait_stream(current)
→ 
execute_async_v3
→ 
torch.cuda.synchronize(device)
（不可按流同步）。按流同步会导致TRT辅助流任务仍在运行，进而引发gst-CV SIGSEGV错误（NVBug 6289256）
skill_05_triton_ddm_model.md
METADATA_LICENSE_FROM_FILE
/v1/metadata
从
DS_SOP_LICENSE_PATH
（默认
/opt/nvidia/nvds_sop/license.txt
）读取
licenseInfo
；绝不能硬编码许可证文本
skill_01_fastapi_endpoints.md
CAMERA_EMULATION_FRAMES_RGB
Pylon仿真PNG必须是明确的3通道RGB格式（与
Emulation_0815-0000.pfs PixelFormat=RGB8Packed
匹配）；可通过
nvvideoconvert ! videoconvert ! "video/x-raw,format=RGB" ! pngenc
生成
skill_08_basler_camera.md
COMPOSE_ENV_PASSTHROUGH
docker compose
仅替换
${VAR}
引用；每个运行时环境变量必须在
environment:
下显式列出才能传递到容器中。
skill_09_docker_build_deploy.md
以下4条
RTSP_*
规则仅在**启用可选的RTSP流式输出功能（第18节）**时适用。默认构建无需遵循——若用户未请求RTSP输出，请跳过这些规则。
| 
RTSP_OUTPUT_TAPS_TEE1
| RTSP输出分支仅在
rtsp_port
存在时，从已有的
tee1
（添加在主推理分支之后）链接。 | 
skill_18_rtsp_streaming_output.md
| | 
RTSP_LEAKY_QUEUE_TINY
| RTSP分支队列必须设置
leaky=2
+ 极小容量（
max-size-buffers=2
），以防止背压和NVMM池耗尽。 | 
skill_18_rtsp_streaming_output.md
| | 
RTSP_KEYINT_MAX_30
| RTSP H.264编码器必须设置
key-int-max=30
（并禁用B帧），以支持下游的seek操作。 | 
skill_18_rtsp_streaming_output.md
| | 
RTSP_ENCODER_FALLBACK
| 根据
SW_ENCODER
选择软件/硬件H.264编码器，同时支持MJPEG回退。 | 
skill_18_rtsp_streaming_output.md
|
",