video-generation

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Video Generation Skill

视频生成Skill

Overview

概述

This skill generates high-quality videos using structured prompts and a Python script. The workflow includes creating JSON-formatted prompts and executing video generation with optional reference image.

本Skill使用结构化提示词和Python脚本生成高质量视频。工作流程包括创建JSON格式的提示词，以及结合可选的参考图片执行视频生成。

Core Capabilities

核心功能

Create structured JSON prompts for AIGC video generation
Support reference image as guidance or the first/last frame of the video
Generate videos through automated Python script execution

为AIGC视频生成创建结构化JSON提示词
支持将参考图片作为引导或视频的首帧/末帧
通过自动执行Python脚本生成视频

Workflow

工作流程

Step 1: Understand Requirements

步骤1：理解需求

When a user requests video generation, identify:

Subject/content: What should be in the image
Style preferences: Art style, mood, color palette
Technical specs: Aspect ratio, composition, lighting
Reference image: Any image to guide generation
You don't need to check the folder under
```
/mnt/user-data
```

当用户请求生成视频时，明确以下信息：

主题/内容：视频中应包含的元素
风格偏好：艺术风格、氛围、配色方案
技术规格：宽高比、构图、光线
参考图片：用于引导生成的图片
无需检查
```
/mnt/user-data
```
下的文件夹

Step 2: Create Structured Prompt

步骤2：创建结构化提示词

Generate a structured JSON file in

/mnt/user-data/workspace/

with naming pattern:

{descriptive-name}.json

在

/mnt/user-data/workspace/

目录下生成一个结构化JSON文件，命名格式为：

{描述性名称}.json

Step 3: Create Reference Image (Optional when image-generation skill is available)

步骤3：创建参考图片（当有图片生成Skill时可选）

Generate reference image for the video generation.

If only 1 image is provided, use it as the guided frame of the video

为视频生成创建参考图片。

若仅提供1张图片，将其用作视频的引导帧

Step 3: Execute Generation

步骤3：执行生成

Call the Python script:

bash

python /mnt/skills/public/video-generation/scripts/generate.py \
  --prompt-file /mnt/user-data/workspace/prompt-file.json \
  --reference-images /path/to/ref1.jpg \
  --output-file /mnt/user-data/outputs/generated-video.mp4 \
  --aspect-ratio 16:9

Parameters:

```
--prompt-file
```
: Absolute path to JSON prompt file (required)
```
--reference-images
```
: Absolute paths to reference image (optional)
```
--output-file
```
: Absolute path to output image file (required)
```
--aspect-ratio
```
: Aspect ratio of the generated image (optional, default: 16:9)

[!NOTE] Do NOT read the python file, instead just call it with the parameters.

调用Python脚本：

bash

python /mnt/skills/public/video-generation/scripts/generate.py \
  --prompt-file /mnt/user-data/workspace/prompt-file.json \
  --reference-images /path/to/ref1.jpg \
  --output-file /mnt/user-data/outputs/generated-video.mp4 \
  --aspect-ratio 16:9

参数说明：

```
--prompt-file
```
：JSON提示词文件的绝对路径（必填）
```
--reference-images
```
：参考图片的绝对路径（可选）
```
--output-file
```
：输出视频文件的绝对路径（必填）
```
--aspect-ratio
```
：生成视频的宽高比（可选，默认值：16:9）

[!提示] 请勿读取Python文件，只需传入参数调用它即可。

Video Generation Example

视频生成示例

User request: "Generate a short video clip depicting the opening scene from "The Chronicles of Narnia: The Lion, the Witch and the Wardrobe"

Step 1: Search for the opening scene of "The Chronicles of Narnia: The Lion, the Witch and the Wardrobe" online

Step 2: Create a JSON prompt file with the following content:

json

{
  "title": "The Chronicles of Narnia - Train Station Farewell",
  "background": {
    "description": "World War II evacuation scene at a crowded London train station. Steam and smoke fill the air as children are being sent to the countryside to escape the Blitz.",
    "era": "1940s wartime Britain",
    "location": "London railway station platform"
  },
  "characters": ["Mrs. Pevensie", "Lucy Pevensie"],
  "camera": {
    "type": "Close-up two-shot",
    "movement": "Static with subtle handheld movement",
    "angle": "Profile view, intimate framing",
    "focus": "Both faces in focus, background soft bokeh"
  },
  "dialogue": [
    {
      "character": "Mrs. Pevensie",
      "text": "You must be brave for me, darling. I'll come for you... I promise."
    },
    {
      "character": "Lucy Pevensie",
      "text": "I will be, mother. I promise."
    }
  ],
  "audio": [
    {
      "type": "Train whistle blows (signaling departure)",
      "volume": 1
    },
    {
      "type": "Strings swell emotionally, then fade",
      "volume": 0.5
    },
    {
      "type": "Ambient sound of the train station",
      "volume": 0.5
    }
  ]
}

Step 3: Use the image-generation skill to generate the reference image

Load the image-generation skill and generate a single reference image

narnia-farewell-scene-01.jpg

according to the skill.

Step 4: Use the generate.py script to generate the video

bash

python /mnt/skills/public/video-generation/scripts/generate.py \
  --prompt-file /mnt/user-data/workspace/narnia-farewell-scene.json \
  --reference-images /mnt/user-data/outputs/narnia-farewell-scene-01.jpg \
  --output-file /mnt/user-data/outputs/narnia-farewell-scene-01.mp4 \
  --aspect-ratio 16:9

Do NOT read the python file, just call it with the parameters.

用户请求：“生成一段描绘《纳尼亚传奇：狮子、女巫和魔衣橱》开场场景的短视频片段”

步骤1：在线搜索《纳尼亚传奇：狮子、女巫和魔衣橱》的开场场景

步骤2：创建一个包含以下内容的JSON提示词文件：

json

{
  "title": "The Chronicles of Narnia - Train Station Farewell",
  "background": {
    "description": "World War II evacuation scene at a crowded London train station. Steam and smoke fill the air as children are being sent to the countryside to escape the Blitz.",
    "era": "1940s wartime Britain",
    "location": "London railway station platform"
  },
  "characters": ["Mrs. Pevensie", "Lucy Pevensie"],
  "camera": {
    "type": "Close-up two-shot",
    "movement": "Static with subtle handheld movement",
    "angle": "Profile view, intimate framing",
    "focus": "Both faces in focus, background soft bokeh"
  },
  "dialogue": [
    {
      "character": "Mrs. Pevensie",
      "text": "You must be brave for me, darling. I'll come for you... I promise."
    },
    {
      "character": "Lucy Pevensie",
      "text": "I will be, mother. I promise."
    }
  ],
  "audio": [
    {
      "type": "Train whistle blows (signaling departure)",
      "volume": 1
    },
    {
      "type": "Strings swell emotionally, then fade",
      "volume": 0.5
    },
    {
      "type": "Ambient sound of the train station",
      "volume": 0.5
    }
  ]
}

步骤3：使用图片生成Skill生成参考图片

加载图片生成Skill，并根据该Skill生成一张参考图片

narnia-farewell-scene-01.jpg

。

步骤4：使用generate.py脚本生成视频

bash

python /mnt/skills/public/video-generation/scripts/generate.py \
  --prompt-file /mnt/user-data/workspace/narnia-farewell-scene.json \
  --reference-images /mnt/user-data/outputs/narnia-farewell-scene-01.jpg \
  --output-file /mnt/user-data/outputs/narnia-farewell-scene-01.mp4 \
  --aspect-ratio 16:9

请勿读取Python文件，只需传入参数调用它即可。

Output Handling

输出处理

After generation:

Videos are typically saved in
```
/mnt/user-data/outputs/
```
Share generated videos (come first) with user as well as generated image if applicable, using
```
present_files
```
tool
Provide brief description of the generation result
Offer to iterate if adjustments needed

生成完成后：

视频通常会保存到
```
/mnt/user-data/outputs/
```
目录下
使用
```
present_files
```
工具将生成的视频（优先）以及生成的图片（如有）分享给用户
提供生成结果的简要描述
若需要调整，可提供迭代优化服务

Notes

注意事项

Always use English for prompts regardless of user's language
JSON format ensures structured, parsable prompts
Reference image enhance generation quality significantly
Iterative refinement is normal for optimal results

无论用户使用何种语言，提示词始终使用英文
JSON格式确保提示词结构化、可解析
参考图片可显著提升生成质量
为获得最佳结果，迭代优化是正常流程