seedance-video-creator

Original🇨🇳 Chinese
Translated
1 scripts

Seedance 2.0: An integrated tool for professional storyboard prompt generation and video creation. It is triggered when users want to create storyboard videos, generate videos via Seedance/Jimeng, need professional storyboard prompts and directly generate videos. It supports multi-image reference, storyboard guidance, API-based video generation, and automatic download.

2installs
Added on

NPX Install

npx skill4agent add wwwzhouhui/skills_collection seedance-video-creator

Tags

Translated version includes tags in frontmatter

SKILL.md Content (Chinese)

View Translation Comparison →

Seedance 2.0 Storyboard Video Creation Tool

Turn creative ideas into professional storyboard prompts, and directly generate and automatically download videos via the Jimeng API.

Prerequisites

Before using this tool, ensure the following services are available:
  1. jimeng-free-api-all service is running (default port 8000)
    bash
    # Docker Deployment
    docker run -it -d --init --name jimeng-free-api-all \
      -p 8000:8000 -e TZ=Asia/Shanghai \
      wwwzhouhui569/jimeng-free-api-all:latest
  2. Obtain SessionID:
  3. Environment Variables (optional, can also be specified during interaction):
    • JIMENG_API_URL
      : API address (default
      http://127.0.0.1:8000
      )
    • JIMENG_SESSION_ID
      : Jimeng SessionID

Core Capabilities

Seedance 2.0 Features

Multimodal Input (up to 12 files):
  • Images: ≤9 (first frame/last frame/character/scene reference)
  • Videos: ≤3, total duration ≤15s (camera movement/action/effect reference)
  • Audio: ≤3, total duration ≤15s (background music/sound effects/dialogue reference)
  • Text: Natural language description
Video Parameters:
  • Model:
    seedance-2.0
    /
    seedance-2.0-pro
  • Aspect Ratio:
    1:1
    /
    4:3
    /
    3:4
    /
    16:9
    /
    9:16
  • Resolution:
    480p
    /
    720p
    /
    1080p
  • Duration:
    4
    /
    5
    /
    10
    seconds
Interaction Method: Use
@1
,
@2
(or
@图1
,
@image1
) to specify material usage

Workflow

Mode 1: Full Guidance Mode (Recommended for Beginners)

Suitable for users who are unsure about the desired effect, complete creation through 5-step guidance.

Mode 2: Quick Generation Mode

Suitable for users with clear ideas, directly provide prompts and materials to generate videos.

Mode 3: Prompt-only Mode

Only generate storyboard prompts without calling the API (for manual use on the Jimeng platform).

Phase 1: Storyboard Prompt Generation

Step 1: Understand User's Idea

Ask the user in a friendly manner:
  1. What story do you want to tell? (One-sentence summary of the core content)
  2. Video duration? (4/5/10 seconds, default 10 seconds)
  3. Do you have reference materials? (Image/video path)
  4. Do you need to directly generate a video? (Yes → proceed to Phase 2 / No → only output prompts)

Step 2: Dig into Details

Based on the user's initial response, gradually guide to improve the following dimensions:

1. Content Narrative

  • The beginning, development, climax and ending of the story
  • Key plots and characters
  • Dialogue/voiceover content

2. Visual Style

  • Overall style: Realistic/Animation/Ink wash/Sci-fi/Retro/Cinematic
  • Tone and atmosphere: Bright/Dim/Warm color/Cool color/Black and white
  • Aspect ratio: Vertical (9:16)/Horizontal (16:9)/Cinematic widescreen (2.35:1)/Standard (4:3)

3. Lens Language

  • Shot changes: Long shot → Full shot → Medium shot → Close-up → Extreme close-up
  • Camera movement: Push/Pull/Pan/Truck/Follow/Orbit/Rise/Fall/Hitchcock zoom
  • Transition methods: Cut/Dissolve/Match cut/Effect transition

4. Action and Rhythm

  • Main action: Walking/Running/Fighting/Dancing/Facial expression changes
  • Action rhythm: Slow/Urgent/With tension and relaxation
  • Whether music beat matching is needed

5. Sound Design

  • Background music style: Passionate/Warm/Suspenseful/Cheerful/Epic
  • Sound effect requirements: Ambient sound/Action sound effect/Special sound effect
  • Dialogue/voiceover: Line content and tone

Step 3: Build Storyboard Structure

Break down the story into a sequence of shots on the timeline.
Example Structure (10-second video):
0-2 seconds: Opening shot, establish the scene
2-4 seconds: Development, introduce the main subject/conflict
4-7 seconds: Climax, core action/emotional outburst
7-9 seconds: Twist/Transition
9-10 seconds: Ending/Closing shot

Step 4: Generate Professional Prompts

Generate prompts that comply with Seedance 2.0 syntax based on the collected information.
Prompt Structure Template:
[Overall Description] Style + Duration + Aspect Ratio + Overall Atmosphere

[Storyboard Description]
0-X seconds: [Camera movement], [Frame content], [Main action], [Lighting/Effects]
X-Y seconds: [Camera movement], [Frame content], [Main action], [Lighting/Effects]
...

[Sound Instructions] Background music style/Sound effects/Dialogue

[Reference Material Instructions] (if any)
@1 As first frame/character reference
@2 Reference camera movement/action
Prompt Writing Principles:
  1. Clear timeline, mark the time range for each segment
  2. Explicit lens language (push/pull/pan/truck/follow/orbit, etc.)
  3. Specific action descriptions, avoid vague terms
  4. Use
    @1
    ,
    @2
    format when referencing multiple images (corresponding to the upload order of images)
  5. Keep the total length within 500 words, as Seedance's performance declines with overly long prompts

Step 5: Optimization and Confirmation

  • Check for violations of platform rules (realistic face restrictions)
  • Provide material suggestions
  • Proceed to Phase 2 after confirming user satisfaction

Phase 2: Video Generation and Download

Step 1: Confirm Generation Parameters

Before calling the API, confirm the following parameters with the user:
ParameterDescriptionDefault Value
API Addressjimeng-free-api-all service address
http://127.0.0.1:8000
SessionIDsessionid of the Jimeng platformEnvironment variable or provided by user
Modelseedance-2.0 or seedance-2.0-pro
seedance-2.0
Aspect RatioVideo aspect ratio
16:9
Resolution480p/720p/1080p
720p
Duration4/5/10 seconds
10
Reference ImagesList of local image pathsNone

Step 2: Call API to Generate Video

Use different calling methods depending on whether there are reference images:

Method A: Text-only Generation (No Reference Images)

bash
curl -s -X POST "${API_URL}/v1/videos/generations" \
  -H "Authorization: Bearer ${SESSION_ID}" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"${MODEL}\",
    \"prompt\": \"${PROMPT}\",
    \"ratio\": \"${RATIO}\",
    \"resolution\": \"${RESOLUTION}\",
    \"duration\": ${DURATION}
  }"

Method B: Multi-image Reference Generation (Local Images)

bash
curl -s -X POST "${API_URL}/v1/videos/generations" \
  -H "Authorization: Bearer ${SESSION_ID}" \
  -F "model=${MODEL}" \
  -F "prompt=${PROMPT}" \
  -F "ratio=${RATIO}" \
  -F "resolution=${RESOLUTION}" \
  -F "duration=${DURATION}" \
  -F "files=@/path/to/image1.jpg" \
  -F "files=@/path/to/image2.jpg"

Method C: Multi-image Reference Generation (Image URLs)

bash
curl -s -X POST "${API_URL}/v1/videos/generations" \
  -H "Authorization: Bearer ${SESSION_ID}" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"${MODEL}\",
    \"prompt\": \"${PROMPT}\",
    \"ratio\": \"${RATIO}\",
    \"resolution\": \"${RESOLUTION}\",
    \"duration\": ${DURATION},
    \"file_paths\": [\"https://example.com/img1.jpg\", \"https://example.com/img2.jpg\"]
  }"
Important:
  • The API is synchronously blocking and will automatically poll until video generation is complete (usually 60-120 seconds)
  • It is recommended to set curl timeout to 300 seconds:
    --max-time 300
  • @1
    ,
    @2
    in the prompt correspond to the upload order of images in the
    files
    parameter

Step 3: Parse Results

The API returns an OpenAI-compatible format:
json
{
  "created": 1700000000,
  "data": [
    {
      "url": "https://v3-dreamnia.jimeng.com/...",
      "revised_prompt": "Original prompt"
    }
  ]
}
Extract
data[0].url
from the response as the video download address.

Step 4: Download Video

bash
# Extract video URL and download
VIDEO_URL=$(echo "${RESPONSE}" | jq -r '.data[0].url')
OUTPUT_FILE="seedance_$(date +%Y%m%d_%H%M%S).mp4"
curl -L -o "${OUTPUT_FILE}" "${VIDEO_URL}"
echo "Video downloaded to: $(pwd)/${OUTPUT_FILE}"

Step 5: Present Results

Report to the user:
  1. Video generation success/failure
  2. Video file path and size
  3. Prompt used (convenient for users to fine-tune and retry)

Error Handling

API Error Codes

Status CodeMeaningHandling Method
200SuccessExtract data[0].url
401SessionID expiredPrompt user to re-obtain
400Parameter errorCheck model/ratio/duration
429Insufficient creditsPrompt user to wait or recharge
500Server errorRetry or check service status

Common Issues

IssueSolution
Connection refusedCheck if the jimeng-free-api-all container is running
Generation timeoutIncrease curl --max-time, or reduce resolution
Content filteredModify the prompt to avoid sensitive content
Video URL expiredThe URL has a time limit, download immediately after generation
Insufficient creditsDaily free credits are limited (about 66 credits), plan usage reasonably

Complete Example Dialogue

Example: Beach Dance Video

User: Help me generate a video of a girl dancing on the beach
Guidance:
  1. What style of dance? Modern dance/Classical dance/Hip-hop/Free dance?
  2. What time of day is it at the beach? Sunrise/Daytime/Sunset/Night?
  3. Overall atmosphere? Cheerful/Sad/Free/Romantic?
  4. Duration in seconds? Do you have reference images?
Generated Prompt:
Cinematic realistic style, 10 seconds, 16:9 widescreen, warm atmosphere of the golden hour at sunset

0-2 seconds: Long shot slowly pushing in, sunset on the horizon, girl's silhouette standing on the beach, skirt blown by the sea breeze
2-5 seconds: Medium shot with orbit camera, girl starts spinning and dancing, hair and skirt flying, backlighting from the sunset forms a rim light
5-8 seconds: Close-up follow shot, girl dances facing the camera, expression free and joyful, waves gently hitting the beach as background
8-10 seconds: Long shot pulling out, girl freezes in the afterglow of the sunset, frame fades to dark

Background sound effects: Sea waves + soft piano background music
API Call:
bash
curl -s --max-time 300 -X POST "http://127.0.0.1:8000/v1/videos/generations" \
  -H "Authorization: Bearer ${SESSION_ID}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "seedance-2.0",
    "prompt": "Cinematic realistic style, 10 seconds, 16:9 widescreen, warm atmosphere of the golden hour at sunset\n\n0-2 seconds: Long shot slowly pushing in, sunset on the horizon, girl's silhouette standing on the beach, skirt blown by the sea breeze\n2-5 seconds: Medium shot with orbit camera, girl starts spinning and dancing, hair and skirt flying, backlighting from the sunset forms a rim light\n5-8 seconds: Close-up follow shot, girl dances facing the camera, expression free and joyful, waves gently hitting the beach as background\n8-10 seconds: Long shot pulling out, girl freezes in the afterglow of the sunset, frame fades to dark\n\nBackground sound effects: Sea waves + soft piano background music",
    "ratio": "16:9",
    "resolution": "720p",
    "duration": 10
  }'

Example: Multi-image Reference Video

User: Generate a video of two people dancing using these two images
  • Image 1: /home/user/dancer1.jpg
  • Image 2: /home/user/dancer2.jpg
Generated Prompt:
Cinematic realistic style, 10 seconds, 4:3 aspect ratio, stage lighting atmosphere

0-2 seconds: Medium shot, @1 and @2 stand face to face, lights from above, preparation pose
2-5 seconds: Full shot, the two start a duet, camera slowly orbits, movements are elegant and coordinated
5-8 seconds: Medium close-up follow shot, dance steps speed up, spinning movements, light and shadow interweave
8-10 seconds: Long shot pulling out, the two freeze in the ending pose of the dance, frame fades to dark
API Call:
bash
curl -s --max-time 300 -X POST "http://127.0.0.1:8000/v1/videos/generations" \
  -H "Authorization: Bearer ${SESSION_ID}" \
  -F "model=seedance-2.0" \
  -F "prompt=Cinematic realistic style, 10 seconds, 4:3 aspect ratio, stage lighting atmosphere..." \
  -F "ratio=4:3" \
  -F "resolution=720p" \
  -F "duration=10" \
  -F "files=@/home/user/dancer1.jpg" \
  -F "files=@/home/user/dancer2.jpg"

Quick Reference for Storyboard Templates

Template 1: Narrative Story Type

[Style] Cinematic realistic/Animation/Ink wash/Sci-fi style
[Duration] 10 seconds
[Aspect Ratio] 16:9

0-2 seconds: [Camera movement], [Scene establishment], [Main subject introduction]
2-4 seconds: [Camera movement], [Plot development], [Action description]
4-7 seconds: [Camera movement], [Climax/Conflict], [Emotional outburst]
7-9 seconds: [Camera movement], [Twist/Transition]
9-10 seconds: [Camera movement], [Ending/Closing shot]

[Sound] Background music style + Sound effects + Dialogue
[Reference] @1 as first frame, @2 as character reference

Template 2: Product Display Type

[Style] Commercial advertisement/Minimalist/High-end/Technological
[Duration] 10 seconds

0-2 seconds: Eye-catching opening, product close-up or suspense setup
2-4 seconds: Full shot of product, orbit/push-pull camera movement
4-7 seconds: Close-up of product details, material/craft display
7-9 seconds: Usage scenario, product application in real environment
9-10 seconds: Brand closing, slogan display

[Sound] Grand/Cheerful and fashionable/Technological background music
[Reference] @1 product appearance, @2 material reference

Template 3: Character Action Type

[Style] Based on character setting (Martial arts/Sci-fi/Modern/Fantasy)
[Duration] 10 seconds

0-2 seconds: Character appearance, freeze frame or slow display of posture
2-4 seconds: Action start, preparation pose
4-8 seconds: Core action display (Fighting/Dancing/Stunt)
8-9 seconds: Action conclusion, pose freeze
9-10 seconds: Effect/atmosphere enhancement, frame closing

[Sound] Action sound effects + Ambient background music
[Reference] @1 character image, @2 action reference

Template 4: Scenery Travel Shot Type

[Style] Cinematic documentary/Healing/Epic
[Duration] 10 seconds

0-2 seconds: Wide establishing shot, showing the full environment
2-4 seconds: Medium shot pushing in, introducing characters or details
4-7 seconds: Multi-angle switching, showing different aspects of the environment
7-9 seconds: Close-up of details, light and shadow changes
9-10 seconds: Return to wide shot or artistic closing

[Sound] Ambient sound + Ambient background music
[Reference] @1-@5 scene references

Template 5: Video Extension/Continuation

Extend @Video 1 by X seconds (select X seconds as the generated length)

Continue the style and main subject of the previous video:
0-X seconds: [New content description], seamlessly connected with the previous video

[Requirements] Maintain character consistency, smooth and coherent movements

Template 6: Plot Subversion/Editing

Edit based on @Video 1:

[Retain] Camera movement/partial action/scene of the original video
[Modify] [Specific modification points]
[Subvert] [Plot reversal description]

[Requirements] Maintain lens coherence, only modify at specified positions

Quick Reference for Lens Language

ChineseEffectApplicable Scenario
推镜头Emphasize the subject, create tensionEmotional outburst, detail display
拉镜头Show the scene, create a sense of distanceOpening establishment, closing conclusion
摇镜头Display the environment left/right/up/downEnvironment introduction, character relationship
移镜头Parallel movement, follow the subjectWalking, chasing
跟镜头Follow the subject to moveAction follow-shot
环绕镜头360-degree display of the subjectProduct display, character appearance
升降镜头From top to bottom or bottom to topFull scene display, momentum display
希区柯克变焦Background compression/stretching effectSuspense, psychological changes
手持晃动Documentary feel/tensionDocumentary, chasing
一镜到底No cuts, continuous shootingImmersion, long narrative

Quick Reference for Atmosphere Keywords

Light and Shadow: Backlight, Side light, Top light, Rembrandt light, Silhouette, Rim light, Volumetric light, Tyndall effect Tone: Warm tone, Cool tone, High saturation, Low saturation, Black and white, Cyberpunk, Retro film Texture: Cinematic, Documentary style, Advertisement texture, MV style, Oil painting feel, Ink wash feel Emotion: Warm, Tense, Suspenseful, Cheerful, Sad, Epic, Healing, Thrilling

Special Scene Handling

1. Character Consistency Requirement

  • It is recommended that users upload character reference images as
    files
  • Specify character reference with
    @1
    in the prompt
  • Note: Realistic human face materials are not supported temporarily

2. Camera Movement/Action Replication

  • It is recommended that users provide reference videos
  • Explain in the prompt "fully reference all camera movement effects of @Video 1"

3. Video Extension

  • Clearly state "extend @Video 1 by Xs"
  • Set the duration parameter to the length of the new part

4. Plot Subversion/Editing

  • When modifying based on an existing video, clearly describe the changes

5. Music Beat Matching

  • Mark the timeline to align with the strong beats of the music
  • It is recommended to provide a reference video to show the rhythm

Notes

  • Materials containing realistic human faces are not supported temporarily
  • Video references will consume more generation credits
  • The total limit for mixed input is 12 files
  • Prioritize uploading materials that have the greatest impact on the frame
  • API calls are synchronously blocking, generating a 10-second video usually takes 60-120 seconds
  • The video download URL has a time limit, download immediately after generation
  • Daily free credits are limited (about 66 credits), plan usage reasonably

Output Format

The final output should include:
  1. Understanding Confirmation: Confirmed story content
  2. Storyboard Prompt: Complete prompt that can be used directly
  3. Generation Parameters: model/ratio/resolution/duration
  4. Generation Result: Video file path (if API was called)
  5. Usage Tips: Instructions for manual use on the Jimeng platform