whiteboard-video-workflow
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseWhiteboard Video Workflow
Whiteboard Video Workflow
从 SRT 字幕文件到完整白板动画视频片段的自动化工作流。
Automated workflow for generating complete whiteboard animation video clips from SRT subtitle files.
输入参数
Input Parameters
| 参数 | 必填 | 说明 |
|---|---|---|
| 是 | SRT 字幕文件的绝对路径 |
| 否 | 输出根目录,默认为 SRT 文件所在目录 |
| Parameter | Required | Description |
|---|---|---|
| Yes | Absolute path of the SRT subtitle file |
| No | Output root directory, defaults to the directory where the SRT file is located |
工作流步骤
Workflow Steps
整个流程分为 10 步,必须严格按顺序执行。步骤 3、5、7 使用 subagent,其余步骤由主 agent 直接执行。
The entire process is divided into 10 steps, which must be executed in strict order. Steps 3, 5, and 7 use subagent, and the remaining steps are directly executed by the main agent.
步骤 0: 环境预检
Step 0: Environment Pre-check
运行本 skill 的 ,一次性检查所有依赖(Python 虚拟环境、RUNNINGHUB_API_KEY):
scripts/check_env.pybash
python3 <skill-dir>/scripts/check_env.py- 成功(退出码 0):从输出中捕获 ,记录该路径用于步骤 7,继续
PYTHON_PATH=<路径> - 失败(退出码 1):从输出的 JSON 中解析各项检查结果,向用户展示清晰易懂的错误说明和修复指引(见下方故障排查指引),终止工作流
ENV_RESULT=...
注意:脚本会自动检测并安装可修复的依赖,只将无法自动修复的问题报告给大模型。
Run the of this skill to check all dependencies at once (Python virtual environment, RUNNINGHUB_API_KEY):
scripts/check_env.pybash
python3 <skill-dir>/scripts/check_env.py- Success (exit code 0): Capture from the output, record this path for Step 7, continue
PYTHON_PATH=<path> - Failure (exit code 1): Parse each check result from the output JSON, show users clear and easy-to-understand error descriptions and repair guidance (see the troubleshooting guide below), terminate the workflow
ENV_RESULT=...
Note: The script will automatically detect and install repairable dependencies, and only report problems that cannot be automatically repaired to the large model.
故障排查指引
Troubleshooting Guide
当环境预检失败时,必须向用户清晰解释原因并给出具体修复步骤。根据失败项不同,给出对应说明:
RUNNINGHUB_API_KEY 未配置:
白板动画视频生成需要 RunningHub API 来调用 AI 模型先生成图片。你需要在 skill 目录下的文件中配置 API Key。.env修复方法:
- 在
文件中添加一行:<skill-dir>/.envRUNNINGHUB_API_KEY=你的API密钥- 如果文件不存在,先创建它
- API Key 可在 RunningHub 获取
- 配置完成后重新运行即可
Python 依赖安装失败:
白板动画依赖 OpenCV、NumPy、PyAV 等 Python 库来处理视频。自动安装未能成功。修复方法:
- 请确认你的 Python 版本为 3.9 或更高版本
- 然后重新运行本工作流,脚本会再次尝试自动安装
多项检查同时失败时,逐条列出每个失败项及对应修复方法,便于用户一次性解决所有问题。
When the environment pre-check fails, you must clearly explain the reason to the user and give specific repair steps. According to different failure items, give corresponding instructions:
RUNNINGHUB_API_KEY is not configured:
Whiteboard animation video generation requires RunningHub API to call AI models to generate images first. You need to configure the API Key in thefile under the skill directory..envFix method:
- Add a line to the
file:<skill-dir>/.envRUNNINGHUB_API_KEY=your API key- If the file does not exist, create it first
- The API Key can be obtained from RunningHub
- Rerun after the configuration is complete
Python dependency installation failed:
Whiteboard animation relies on Python libraries such as OpenCV, NumPy, and PyAV to process videos. Automatic installation failed.Fix method:
- Please confirm that your Python version is 3.9 or higher
- Then rerun this workflow, and the script will try to install automatically again
When multiple checks fail at the same time, list each failure item and corresponding repair method one by one, so that users can solve all problems at once.
步骤 1: 确定输出目录
Step 1: Determine Output Directory
- 如果用户未指定 ,则使用
outputDir所在目录作为输出根目录srtPath - 将 转换为绝对路径
outputDir
- If the user does not specify , use the directory where
outputDiris located as the output root directorysrtPath - Convert to an absolute path
outputDir
步骤 2: 创建输出目录结构
Step 2: Create Output Directory Structure
运行本 skill 的 :
scripts/workflow_helper.pybash
python3 <skill-dir>/scripts/workflow_helper.py init-dirs "<outputDir>"输出 JSON 含 、、 三个绝对路径,保存备用。
storyboardDirimageDirvideoDirRun the of this skill:
scripts/workflow_helper.pybash
python3 <skill-dir>/scripts/workflow_helper.py init-dirs "<outputDir>"The output JSON contains three absolute paths: , , and , which are saved for later use.
storyboardDirimageDirvideoDir步骤 3: 解析 SRT 生成分镜脚本(subagent)
Step 3: Parse SRT to Generate Storyboard Script (subagent)
启动一个 subagent,指令为:
使用 Read 工具读取文件,按照其中的工作流步骤执行。<将本 skill 目录替换为实际绝对路径>/references/storyboard-parser.md输入参数:
- srtPath =
<将srtPath替换为实际绝对路径>- projectRoot =
<将storyboardDir替换为实际绝对路径>- skill-dir =
(用于定位脚本)<将本 skill 目录替换为实际绝对路径>完成后返回 storyboard.json 的绝对路径和场景数量。注意:主 agent 必须将实际路径值填入指令中,不要传递变量名,subagent 无法访问主 agent 的上下文。
必须等待 subagent 完成并获取 storyboard.json 路径后才继续。
Start a subagent with the instruction:
Use the Read tool to read the file, and execute according to the workflow steps in it.<replace this skill directory with the actual absolute path>/references/storyboard-parser.mdInput parameters:
- srtPath =
<replace srtPath with the actual absolute path>- projectRoot =
<replace storyboardDir with the actual absolute path>- skill-dir =
(used to locate scripts)<replace this skill directory with the actual absolute path>Return the absolute path of storyboard.json and the number of scenes after completion.Note: The main agent must fill in the actual path value into the instruction, do not pass the variable name, the subagent cannot access the context of the main agent.
You must wait for the subagent to complete and obtain the storyboard.json path before continuing.
步骤 4: 解析 storyboard 生成图片提示词
Step 4: Parse Storyboard to Generate Image Prompts
运行本 skill 的 :
scripts/workflow_helper.pybash
python3 <skill-dir>/scripts/workflow_helper.py gen-prompts "<storyboardJsonPath>"输出一个 JSON 字符串数组,每个元素是一个带白板风格前缀的图片生成提示词,数组索引与 storyboard 中的 scenes 顺序一一对应。
同时从 storyboard.json 中提取每个 scene 的 值(毫秒),按顺序记录为数组备用。
durationRun the of this skill:
scripts/workflow_helper.pybash
python3 <skill-dir>/scripts/workflow_helper.py gen-prompts "<storyboardJsonPath>"Output a JSON string array, each element is an image generation prompt with a whiteboard style prefix, and the array index corresponds to the order of scenes in the storyboard one by one.
At the same time, extract the value (milliseconds) of each scene from storyboard.json, and record it as an array in order for later use.
duration步骤 5: 批量生成白板图片(subagent)
Step 5: Batch Generate Whiteboard Images (subagent)
启动一个 subagent,指令为:
使用 Read 工具读取文件,按照其中的工作流步骤执行。<将本 skill 目录替换为实际绝对路径>/references/image-generator.md使用批量模式,将以下 JSON 字符串数组作为 prompt 参数传入:<将步骤4输出的提示词JSON数组的实际内容粘贴于此>参数:
- skill-dir =
(用于定位脚本)<将本 skill 目录替换为实际绝对路径>- 输出目录 =
<将imageDir替换为实际绝对路径>- 宽高比 = "16:9"
注意:主 agent 必须将实际提示词内容和路径值填入指令中,不要传递变量名,subagent 无法访问主 agent 的上下文。重要: 返回所有生成图片的路径列表,顺序必须与提示词数组顺序一致。
必须等待 subagent 完成并获取所有图片路径后才继续。
Start a subagent with the instruction:
Use the Read tool to read the file, and execute according to the workflow steps in it.<replace this skill directory with the actual absolute path>/references/image-generator.mdUse batch mode, pass the following JSON string array as the prompt parameter:<paste the actual content of the prompt JSON array output in Step 4 here>Parameters:
- skill-dir =
(used to locate scripts)<replace this skill directory with the actual absolute path>- Output directory =
<replace imageDir with the actual absolute path>- Aspect ratio = "16:9"
Note: The main agent must fill in the actual prompt content and path value into the instruction, do not pass the variable name, the subagent cannot access the context of the main agent.Important: Return the list of paths of all generated images, and the order must be consistent with the order of the prompt array.
You must wait for the subagent to complete and obtain all image paths before continuing.
步骤 6: 校验图片顺序
Step 6: Verify Image Order
确认步骤 5 返回的图片路径数组长度与 storyboard 的 scenes 数量一致,顺序正确(第 i 张图片对应第 i 个 scene)。
Confirm that the length of the image path array returned in Step 5 is consistent with the number of scenes in the storyboard, and the order is correct (the i-th image corresponds to the i-th scene).
步骤 7: 批量生成白板动画视频片段(subagent)
Step 7: Batch Generate Whiteboard Animation Video Clips (subagent)
启动一个 subagent,指令为:
调用 whiteboard-animation skill,使用批量模式。跳过环境预检(主 agent 已确认环境就绪),直接运行批量生成脚本:<将PYTHON_PATH替换为步骤0获取的实际值> <将whiteboard-animation skill目录替换为实际路径>/scripts/batch_generate.py \ --images <imagePaths[0]> <imagePaths[1]> ... \ --durations <durations[0]> <durations[1]> ... \ --output-dir <videoDir绝对路径>主 agent 必须将和 skill 目录的实际绝对路径填入指令中,不要传递变量名,subagent 无法访问主 agent 的上下文。PYTHON_PATH参数:
:按分镜顺序排列的图片路径列表(空格分隔)--images :与图片一一对应的时长列表(单位:毫秒,直接使用 storyboard 中的 duration 值,无需转换)--durations :--output-dir<videoDir绝对路径>重要: 完成后,收集目录下所有生成的视频文件路径,按文件名时间戳排序,将完整的视频路径列表返回给主 agent。顺序必须与输入图片顺序一致。<videoDir>
必须等待 subagent 完成并获取所有视频路径后才继续。
Start a subagent with the instruction:
Call the whiteboard-animation skill, use batch mode. Skip environment pre-check (the main agent has confirmed that the environment is ready), and run the batch generation script directly:<replace PYTHON_PATH with the actual value obtained in Step 0> <replace the whiteboard-animation skill directory with the actual path>/scripts/batch_generate.py \ --images <imagePaths[0]> <imagePaths[1]> ... \ --durations <durations[0]> <durations[1]> ... \ --output-dir <absolute path of videoDir>The main agent must fill in the actual absolute path ofand the skill directory into the instruction, do not pass the variable name, the subagent cannot access the context of the main agent.PYTHON_PATHParameters:
: List of image paths arranged in storyboard order (separated by spaces)--images : List of durations corresponding to images one by one (unit: milliseconds, directly use the duration value in the storyboard, no conversion required)--durations :--output-dir<absolute path of videoDir>Important: After completion, collect all generated video file paths in thedirectory, sort them by file name timestamp, and return the complete video path list to the main agent. The order must be consistent with the order of input images.<videoDir>
You must wait for the subagent to complete and obtain all video paths before continuing.
步骤 8: 合并视频片段
Step 8: Merge Video Clips
运行本 skill 的 ,将所有视频片段按顺序合并为一个完整视频。必须使用步骤 0 获取的 (PyAV 依赖在虚拟环境中):
scripts/workflow_helper.pyPYTHON_PATHbash
<PYTHON_PATH> <skill-dir>/scripts/workflow_helper.py merge-videos "<outputDir>" <videoPath1> <videoPath2> ...- 第一个参数为输出目录(合并后的视频保存在此目录)
- 后续参数为按分镜顺序排列的视频片段路径
输出 JSON 含 (合并后的视频绝对路径)、、。
mergedVideototalSegmentssizeMBRun the of this skill to merge all video clips into a complete video in order. Must use the obtained in Step 0 (the PyAV dependency is in the virtual environment):
scripts/workflow_helper.pyPYTHON_PATHbash
<PYTHON_PATH> <skill-dir>/scripts/workflow_helper.py merge-videos "<outputDir>" <videoPath1> <videoPath2> ...- The first parameter is the output directory (the merged video is saved in this directory)
- The subsequent parameters are the paths of video clips arranged in storyboard order
The output JSON contains (absolute path of the merged video), , and .
mergedVideototalSegmentssizeMB步骤 9: 输出结果
Step 9: Output Result
输出最终结果,包含合并后的完整视频路径和所有片段信息:
输出格式示例:
json
{
"mergedVideo": "/path/to/output/白板视频_20260329_120000.mp4",
"videoSegments": [
"/path/to/video/vid_20260329_120000_h264.mp4",
"/path/to/video/vid_20260329_120010_h264.mp4"
],
"totalSegments": 2,
"sizeMB": 15.3,
"outputDir": "/path/to/output"
}Output the final result, including the merged complete video path and all segment information:
Output format example:
json
{
"mergedVideo": "/path/to/output/白板视频_20260329_120000.mp4",
"videoSegments": [
"/path/to/video/vid_20260329_120000_h264.mp4",
"/path/to/video/vid_20260329_120010_h264.mp4"
],
"totalSegments": 2,
"sizeMB": 15.3,
"outputDir": "/path/to/output"
}关键约束
Key Constraints
- 步骤 0 必须在任何工作开始前执行,脚本会自动检测并安装可修复的依赖,只将无法修复的问题报告给大模型
check_env.py - 步骤 3、5、7 必须使用 subagent 执行,主 agent 等待结果
- 步骤 0 获取的 必须传递给步骤 7 的 subagent 使用,也用于步骤 8 的视频合并(PyAV 依赖在虚拟环境中),避免重复环境检查
PYTHON_PATH - 步骤 7 使用 whiteboard-animation 的批量模式(batch_generate.py),由 subagent 一次传入所有图片和时长,脚本内部串行生成,subagent 完成后返回所有视频路径
- 图片和视频的顺序必须与 storyboard 的 scenes 顺序严格对应
- duration 贯穿全链路使用毫秒,从 storyboard 到 batch_generate.py 再到 generate_whiteboard.py 统一为毫秒,避免浮点转换丢失精度
- 步骤 8 使用 PyAV(Python 库)合并视频片段,通过 H.264 重新编码输出统一格式的最终视频
- Step 0 must be executed before any work starts. The script will automatically detect and install repairable dependencies, and only report unreparable problems to the large model
check_env.py - Steps 3, 5, and 7 must be executed by subagent, and the main agent waits for the result
- The obtained in Step 0 must be passed to the subagent of Step 7 for use, and also used for video merging in Step 8 (the PyAV dependency is in the virtual environment) to avoid repeated environment checks
PYTHON_PATH - Step 7 uses the batch mode of whiteboard-animation (batch_generate.py), the subagent passes in all images and durations at one time, the script generates serially internally, and returns all video paths after the subagent completes
- The order of images and videos must strictly correspond to the order of scenes in the storyboard
- Duration is used in milliseconds throughout the entire link, from storyboard to batch_generate.py to generate_whiteboard.py, unified as milliseconds to avoid loss of precision in floating point conversion
- Step 8 uses PyAV (Python library) to merge video clips, and re-encodes through H.264 to output the final video in a unified format
Resources
Resources
references/
references/
- - SRT 分镜解析工作流指令,由步骤 3 的 subagent 读取执行
storyboard-parser.md - - Banana2 图片生成工作流指令,由步骤 5 的 subagent 读取执行
image-generator.md
- - SRT storyboard parsing workflow instruction, read and executed by the subagent in Step 3
storyboard-parser.md - - Banana2 image generation workflow instruction, read and executed by the subagent in Step 5
image-generator.md
scripts/
scripts/
- - 一次性环境预检,检查 Python 虚拟环境、API Key,自动安装可修复的依赖
check_env.py - - 提供
workflow_helper.py(创建目录结构)、init-dirs(解析 storyboard 生成提示词)、gen-prompts(合并视频片段)三个子命令merge-videos - - 解析 SRT + groups.json 生成 storyboard.json
generate-storyboard.py - - Banana2 模型文生图,支持单张和批量并发模式
generate-image.py - - 白板风格提示词模板
banana_prompt_template.py
- - One-time environment pre-check, check Python virtual environment, API Key, automatically install repairable dependencies
check_env.py - - Provides three subcommands:
workflow_helper.py(create directory structure),init-dirs(parse storyboard to generate prompts),gen-prompts(merge video clips)merge-videos - - Parse SRT + groups.json to generate storyboard.json
generate-storyboard.py - - Banana2 model text-to-image, supports single and batch concurrent modes
generate-image.py - - Whiteboard style prompt template
banana_prompt_template.py