whiteboard-video-workflow

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Whiteboard Video Workflow

Whiteboard Video Workflow

从 SRT 字幕文件到完整白板动画视频片段的自动化工作流。
Automated workflow for generating complete whiteboard animation video clips from SRT subtitle files.

输入参数

Input Parameters

参数必填说明
srtPath
SRT 字幕文件的绝对路径
outputDir
输出根目录,默认为 SRT 文件所在目录
ParameterRequiredDescription
srtPath
YesAbsolute path of the SRT subtitle file
outputDir
NoOutput root directory, defaults to the directory where the SRT file is located

工作流步骤

Workflow Steps

整个流程分为 10 步,必须严格按顺序执行。步骤 3、5、7 使用 subagent,其余步骤由主 agent 直接执行。
The entire process is divided into 10 steps, which must be executed in strict order. Steps 3, 5, and 7 use subagent, and the remaining steps are directly executed by the main agent.

步骤 0: 环境预检

Step 0: Environment Pre-check

运行本 skill 的
scripts/check_env.py
,一次性检查所有依赖(Python 虚拟环境、RUNNINGHUB_API_KEY):
bash
python3 <skill-dir>/scripts/check_env.py
  • 成功(退出码 0):从输出中捕获
    PYTHON_PATH=<路径>
    记录该路径用于步骤 7,继续
  • 失败(退出码 1):从输出的
    ENV_RESULT=...
    JSON 中解析各项检查结果,向用户展示清晰易懂的错误说明和修复指引(见下方故障排查指引),终止工作流
注意:脚本会自动检测并安装可修复的依赖,只将无法自动修复的问题报告给大模型。
Run the
scripts/check_env.py
of this skill to check all dependencies at once (Python virtual environment, RUNNINGHUB_API_KEY):
bash
python3 <skill-dir>/scripts/check_env.py
  • Success (exit code 0): Capture
    PYTHON_PATH=<path>
    from the output, record this path for Step 7, continue
  • Failure (exit code 1): Parse each check result from the output
    ENV_RESULT=...
    JSON, show users clear and easy-to-understand error descriptions and repair guidance (see the troubleshooting guide below), terminate the workflow
Note: The script will automatically detect and install repairable dependencies, and only report problems that cannot be automatically repaired to the large model.

故障排查指引

Troubleshooting Guide

当环境预检失败时,必须向用户清晰解释原因并给出具体修复步骤。根据失败项不同,给出对应说明:
RUNNINGHUB_API_KEY 未配置:
白板动画视频生成需要 RunningHub API 来调用 AI 模型先生成图片。你需要在 skill 目录下的
.env
文件中配置 API Key。
修复方法:
  1. <skill-dir>/.env
    文件中添加一行:
    RUNNINGHUB_API_KEY=你的API密钥
  2. 如果文件不存在,先创建它
  3. API Key 可在 RunningHub 获取
  4. 配置完成后重新运行即可
Python 依赖安装失败:
白板动画依赖 OpenCV、NumPy、PyAV 等 Python 库来处理视频。自动安装未能成功。
修复方法:
  1. 请确认你的 Python 版本为 3.9 或更高版本
  2. 然后重新运行本工作流,脚本会再次尝试自动安装
多项检查同时失败时,逐条列出每个失败项及对应修复方法,便于用户一次性解决所有问题。
When the environment pre-check fails, you must clearly explain the reason to the user and give specific repair steps. According to different failure items, give corresponding instructions:
RUNNINGHUB_API_KEY is not configured:
Whiteboard animation video generation requires RunningHub API to call AI models to generate images first. You need to configure the API Key in the
.env
file under the skill directory.
Fix method:
  1. Add a line to the
    <skill-dir>/.env
    file:
    RUNNINGHUB_API_KEY=your API key
  2. If the file does not exist, create it first
  3. The API Key can be obtained from RunningHub
  4. Rerun after the configuration is complete
Python dependency installation failed:
Whiteboard animation relies on Python libraries such as OpenCV, NumPy, and PyAV to process videos. Automatic installation failed.
Fix method:
  1. Please confirm that your Python version is 3.9 or higher
  2. Then rerun this workflow, and the script will try to install automatically again
When multiple checks fail at the same time, list each failure item and corresponding repair method one by one, so that users can solve all problems at once.

步骤 1: 确定输出目录

Step 1: Determine Output Directory

  • 如果用户未指定
    outputDir
    ,则使用
    srtPath
    所在目录作为输出根目录
  • outputDir
    转换为绝对路径
  • If the user does not specify
    outputDir
    , use the directory where
    srtPath
    is located as the output root directory
  • Convert
    outputDir
    to an absolute path

步骤 2: 创建输出目录结构

Step 2: Create Output Directory Structure

运行本 skill 的
scripts/workflow_helper.py
bash
python3 <skill-dir>/scripts/workflow_helper.py init-dirs "<outputDir>"
输出 JSON 含
storyboardDir
imageDir
videoDir
三个绝对路径,保存备用。
Run the
scripts/workflow_helper.py
of this skill:
bash
python3 <skill-dir>/scripts/workflow_helper.py init-dirs "<outputDir>"
The output JSON contains three absolute paths:
storyboardDir
,
imageDir
, and
videoDir
, which are saved for later use.

步骤 3: 解析 SRT 生成分镜脚本(subagent)

Step 3: Parse SRT to Generate Storyboard Script (subagent)

启动一个 subagent,指令为:
使用 Read 工具读取文件
<将本 skill 目录替换为实际绝对路径>/references/storyboard-parser.md
,按照其中的工作流步骤执行。
输入参数:
  • srtPath =
    <将srtPath替换为实际绝对路径>
  • projectRoot =
    <将storyboardDir替换为实际绝对路径>
  • skill-dir =
    <将本 skill 目录替换为实际绝对路径>
    (用于定位脚本)
完成后返回 storyboard.json 的绝对路径和场景数量。
注意:主 agent 必须将实际路径值填入指令中,不要传递变量名,subagent 无法访问主 agent 的上下文。
必须等待 subagent 完成并获取 storyboard.json 路径后才继续。
Start a subagent with the instruction:
Use the Read tool to read the file
<replace this skill directory with the actual absolute path>/references/storyboard-parser.md
, and execute according to the workflow steps in it.
Input parameters:
  • srtPath =
    <replace srtPath with the actual absolute path>
  • projectRoot =
    <replace storyboardDir with the actual absolute path>
  • skill-dir =
    <replace this skill directory with the actual absolute path>
    (used to locate scripts)
Return the absolute path of storyboard.json and the number of scenes after completion.
Note: The main agent must fill in the actual path value into the instruction, do not pass the variable name, the subagent cannot access the context of the main agent.
You must wait for the subagent to complete and obtain the storyboard.json path before continuing.

步骤 4: 解析 storyboard 生成图片提示词

Step 4: Parse Storyboard to Generate Image Prompts

运行本 skill 的
scripts/workflow_helper.py
bash
python3 <skill-dir>/scripts/workflow_helper.py gen-prompts "<storyboardJsonPath>"
输出一个 JSON 字符串数组,每个元素是一个带白板风格前缀的图片生成提示词,数组索引与 storyboard 中的 scenes 顺序一一对应。
同时从 storyboard.json 中提取每个 scene 的
duration
值(毫秒),按顺序记录为数组备用。
Run the
scripts/workflow_helper.py
of this skill:
bash
python3 <skill-dir>/scripts/workflow_helper.py gen-prompts "<storyboardJsonPath>"
Output a JSON string array, each element is an image generation prompt with a whiteboard style prefix, and the array index corresponds to the order of scenes in the storyboard one by one.
At the same time, extract the
duration
value (milliseconds) of each scene from storyboard.json, and record it as an array in order for later use.

步骤 5: 批量生成白板图片(subagent)

Step 5: Batch Generate Whiteboard Images (subagent)

启动一个 subagent,指令为:
使用 Read 工具读取文件
<将本 skill 目录替换为实际绝对路径>/references/image-generator.md
,按照其中的工作流步骤执行。
使用批量模式,将以下 JSON 字符串数组作为 prompt 参数传入:
<将步骤4输出的提示词JSON数组的实际内容粘贴于此>
参数:
  • skill-dir =
    <将本 skill 目录替换为实际绝对路径>
    (用于定位脚本)
  • 输出目录 =
    <将imageDir替换为实际绝对路径>
  • 宽高比 = "16:9"
注意:主 agent 必须将实际提示词内容和路径值填入指令中,不要传递变量名,subagent 无法访问主 agent 的上下文。
重要: 返回所有生成图片的路径列表,顺序必须与提示词数组顺序一致。
必须等待 subagent 完成并获取所有图片路径后才继续。
Start a subagent with the instruction:
Use the Read tool to read the file
<replace this skill directory with the actual absolute path>/references/image-generator.md
, and execute according to the workflow steps in it.
Use batch mode, pass the following JSON string array as the prompt parameter:
<paste the actual content of the prompt JSON array output in Step 4 here>
Parameters:
  • skill-dir =
    <replace this skill directory with the actual absolute path>
    (used to locate scripts)
  • Output directory =
    <replace imageDir with the actual absolute path>
  • Aspect ratio = "16:9"
Note: The main agent must fill in the actual prompt content and path value into the instruction, do not pass the variable name, the subagent cannot access the context of the main agent.
Important: Return the list of paths of all generated images, and the order must be consistent with the order of the prompt array.
You must wait for the subagent to complete and obtain all image paths before continuing.

步骤 6: 校验图片顺序

Step 6: Verify Image Order

确认步骤 5 返回的图片路径数组长度与 storyboard 的 scenes 数量一致,顺序正确(第 i 张图片对应第 i 个 scene)。
Confirm that the length of the image path array returned in Step 5 is consistent with the number of scenes in the storyboard, and the order is correct (the i-th image corresponds to the i-th scene).

步骤 7: 批量生成白板动画视频片段(subagent)

Step 7: Batch Generate Whiteboard Animation Video Clips (subagent)

启动一个 subagent,指令为:
调用 whiteboard-animation skill,使用批量模式。跳过环境预检(主 agent 已确认环境就绪),直接运行批量生成脚本:
<将PYTHON_PATH替换为步骤0获取的实际值> <将whiteboard-animation skill目录替换为实际路径>/scripts/batch_generate.py \
  --images <imagePaths[0]> <imagePaths[1]> ... \
  --durations <durations[0]> <durations[1]> ... \
  --output-dir <videoDir绝对路径>
主 agent 必须将
PYTHON_PATH
和 skill 目录的实际绝对路径填入指令中,不要传递变量名,subagent 无法访问主 agent 的上下文。
参数:
  • --images
    :按分镜顺序排列的图片路径列表(空格分隔)
  • --durations
    :与图片一一对应的时长列表(单位:毫秒,直接使用 storyboard 中的 duration 值,无需转换)
  • --output-dir
    <videoDir绝对路径>
重要: 完成后,收集
<videoDir>
目录下所有生成的视频文件路径,按文件名时间戳排序,将完整的视频路径列表返回给主 agent。顺序必须与输入图片顺序一致。
必须等待 subagent 完成并获取所有视频路径后才继续。
Start a subagent with the instruction:
Call the whiteboard-animation skill, use batch mode. Skip environment pre-check (the main agent has confirmed that the environment is ready), and run the batch generation script directly:
<replace PYTHON_PATH with the actual value obtained in Step 0> <replace the whiteboard-animation skill directory with the actual path>/scripts/batch_generate.py \
  --images <imagePaths[0]> <imagePaths[1]> ... \
  --durations <durations[0]> <durations[1]> ... \
  --output-dir <absolute path of videoDir>
The main agent must fill in the actual absolute path of
PYTHON_PATH
and the skill directory into the instruction, do not pass the variable name, the subagent cannot access the context of the main agent.
Parameters:
  • --images
    : List of image paths arranged in storyboard order (separated by spaces)
  • --durations
    : List of durations corresponding to images one by one (unit: milliseconds, directly use the duration value in the storyboard, no conversion required)
  • --output-dir
    :
    <absolute path of videoDir>
Important: After completion, collect all generated video file paths in the
<videoDir>
directory, sort them by file name timestamp, and return the complete video path list to the main agent. The order must be consistent with the order of input images.
You must wait for the subagent to complete and obtain all video paths before continuing.

步骤 8: 合并视频片段

Step 8: Merge Video Clips

运行本 skill 的
scripts/workflow_helper.py
,将所有视频片段按顺序合并为一个完整视频。必须使用步骤 0 获取的
PYTHON_PATH
(PyAV 依赖在虚拟环境中):
bash
<PYTHON_PATH> <skill-dir>/scripts/workflow_helper.py merge-videos "<outputDir>" <videoPath1> <videoPath2> ...
  • 第一个参数为输出目录(合并后的视频保存在此目录)
  • 后续参数为按分镜顺序排列的视频片段路径
输出 JSON 含
mergedVideo
(合并后的视频绝对路径)、
totalSegments
sizeMB
Run the
scripts/workflow_helper.py
of this skill to merge all video clips into a complete video in order. Must use the
PYTHON_PATH
obtained in Step 0
(the PyAV dependency is in the virtual environment):
bash
<PYTHON_PATH> <skill-dir>/scripts/workflow_helper.py merge-videos "<outputDir>" <videoPath1> <videoPath2> ...
  • The first parameter is the output directory (the merged video is saved in this directory)
  • The subsequent parameters are the paths of video clips arranged in storyboard order
The output JSON contains
mergedVideo
(absolute path of the merged video),
totalSegments
, and
sizeMB
.

步骤 9: 输出结果

Step 9: Output Result

输出最终结果,包含合并后的完整视频路径和所有片段信息:
输出格式示例:
json
{
  "mergedVideo": "/path/to/output/白板视频_20260329_120000.mp4",
  "videoSegments": [
    "/path/to/video/vid_20260329_120000_h264.mp4",
    "/path/to/video/vid_20260329_120010_h264.mp4"
  ],
  "totalSegments": 2,
  "sizeMB": 15.3,
  "outputDir": "/path/to/output"
}
Output the final result, including the merged complete video path and all segment information:
Output format example:
json
{
  "mergedVideo": "/path/to/output/白板视频_20260329_120000.mp4",
  "videoSegments": [
    "/path/to/video/vid_20260329_120000_h264.mp4",
    "/path/to/video/vid_20260329_120010_h264.mp4"
  ],
  "totalSegments": 2,
  "sizeMB": 15.3,
  "outputDir": "/path/to/output"
}

关键约束

Key Constraints

  • 步骤 0 必须在任何工作开始前执行,
    check_env.py
    脚本会自动检测并安装可修复的依赖,只将无法修复的问题报告给大模型
  • 步骤 3、5、7 必须使用 subagent 执行,主 agent 等待结果
  • 步骤 0 获取的
    PYTHON_PATH
    必须传递给步骤 7 的 subagent 使用,也用于步骤 8 的视频合并(PyAV 依赖在虚拟环境中),避免重复环境检查
  • 步骤 7 使用 whiteboard-animation 的批量模式(batch_generate.py),由 subagent 一次传入所有图片和时长,脚本内部串行生成,subagent 完成后返回所有视频路径
  • 图片和视频的顺序必须与 storyboard 的 scenes 顺序严格对应
  • duration 贯穿全链路使用毫秒,从 storyboard 到 batch_generate.py 再到 generate_whiteboard.py 统一为毫秒,避免浮点转换丢失精度
  • 步骤 8 使用 PyAV(Python 库)合并视频片段,通过 H.264 重新编码输出统一格式的最终视频
  • Step 0 must be executed before any work starts. The
    check_env.py
    script will automatically detect and install repairable dependencies, and only report unreparable problems to the large model
  • Steps 3, 5, and 7 must be executed by subagent, and the main agent waits for the result
  • The
    PYTHON_PATH
    obtained in Step 0 must be passed to the subagent of Step 7 for use, and also used for video merging in Step 8 (the PyAV dependency is in the virtual environment) to avoid repeated environment checks
  • Step 7 uses the batch mode of whiteboard-animation (batch_generate.py), the subagent passes in all images and durations at one time, the script generates serially internally, and returns all video paths after the subagent completes
  • The order of images and videos must strictly correspond to the order of scenes in the storyboard
  • Duration is used in milliseconds throughout the entire link, from storyboard to batch_generate.py to generate_whiteboard.py, unified as milliseconds to avoid loss of precision in floating point conversion
  • Step 8 uses PyAV (Python library) to merge video clips, and re-encodes through H.264 to output the final video in a unified format

Resources

Resources

references/

references/

  • storyboard-parser.md
    - SRT 分镜解析工作流指令,由步骤 3 的 subagent 读取执行
  • image-generator.md
    - Banana2 图片生成工作流指令,由步骤 5 的 subagent 读取执行
  • storyboard-parser.md
    - SRT storyboard parsing workflow instruction, read and executed by the subagent in Step 3
  • image-generator.md
    - Banana2 image generation workflow instruction, read and executed by the subagent in Step 5

scripts/

scripts/

  • check_env.py
    - 一次性环境预检,检查 Python 虚拟环境、API Key,自动安装可修复的依赖
  • workflow_helper.py
    - 提供
    init-dirs
    (创建目录结构)、
    gen-prompts
    (解析 storyboard 生成提示词)、
    merge-videos
    (合并视频片段)三个子命令
  • generate-storyboard.py
    - 解析 SRT + groups.json 生成 storyboard.json
  • generate-image.py
    - Banana2 模型文生图,支持单张和批量并发模式
  • banana_prompt_template.py
    - 白板风格提示词模板
  • check_env.py
    - One-time environment pre-check, check Python virtual environment, API Key, automatically install repairable dependencies
  • workflow_helper.py
    - Provides three subcommands:
    init-dirs
    (create directory structure),
    gen-prompts
    (parse storyboard to generate prompts),
    merge-videos
    (merge video clips)
  • generate-storyboard.py
    - Parse SRT + groups.json to generate storyboard.json
  • generate-image.py
    - Banana2 model text-to-image, supports single and batch concurrent modes
  • banana_prompt_template.py
    - Whiteboard style prompt template