synthetic-screen-recording

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Synthetic Screen Recording (Remotion TerminalScene)

合成屏幕录制(Remotion TerminalScene)

Decision this skill answers: When the user wants a screen-recording-looking demo of a terminal, CLI tool, or coding workflow — do I capture the real desktop (OS screen recording via
screen_recorder
, Windows-MCP, Cap, or Playwright), or do I synthesize it in Remotion with the
TerminalScene
component?
Heuristic: If the agent can author the exact command/output sequence in advance, synthesize. Only capture live when the real behavior is unpredictable, needs a real app UI, or the user explicitly asked for a real recording.
本技能解决的决策问题: 当用户需要一个看起来像屏幕录制的终端、CLI工具或编码流程演示时,我应该捕获真实桌面(通过
screen_recorder
、Windows-MCP、Cap或Playwright进行系统屏幕录制),还是使用Remotion的
TerminalScene
组件合成录制内容
判断准则: 如果Agent可以预先编写好精确的命令/输出序列,就选择合成录制。只有当真实行为不可预测、需要真实应用UI,或者用户明确要求真实录制时,才进行实时捕获。

Why this exists

为什么会有这个技能

v3 of the OpenMontage showcase tried to use Windows-MCP +
screen_recorder
to drive a Git-Bash window for the install walkthrough. It stalled on window positioning, focus races, and taskbar privacy concerns. We pivoted to pure Remotion rendering — a React component named
TerminalScene
that draws a fake terminal and types commands character-by-character. The output is visually indistinguishable from a real screen recording (same traffic-light window chrome, blinking cursor, scrolling output) but deterministic, privacy-safe, pixel-perfect at 1080p, and pace-controllable to the frame.
That component + pattern is the capability this skill makes discoverable.
OpenMontage展示项目的v3版本曾尝试使用Windows-MCP +
screen_recorder
来驱动Git-Bash窗口完成安装流程演示,但在窗口定位、焦点竞争和任务栏隐私问题上遇到了阻碍。于是我们转向纯Remotion渲染——一个名为
TerminalScene
的React组件,它可以绘制一个模拟终端,并逐字符输入命令。输出效果与真实屏幕录制视觉上几乎无差别(拥有相同的红绿灯窗口边框、闪烁光标、滚动输出),但具有确定性、隐私安全、1080p像素级完美,并且可以精确控制到每一帧的节奏。
本技能的作用就是让大家了解这个组件和对应的使用模式。

When to use synthetic (TerminalScene)

何时使用合成录制(TerminalScene)

YES, synthesize when:
  • The demo is a terminal / CLI / coding session where commands and outputs are predictable
  • The user wants a polished tutorial feel (clean typography, floating pills, cursor blink)
  • Install walkthroughs, setup demos, API key config,
    make
    targets,
    git clone
    flows
  • You need tight sync with narration — every command must land on a specific beat
  • You want the result reproducible (re-render gets identical pixels)
  • The user's actual desktop has private apps/windows visible you'd otherwise have to crop
NO, capture a real screen when:
  • The demo is a real app UI that can't be faked (Figma, Photoshop, a web app with live state, a browser flow)
  • The user explicitly asked for a recording of their actual screen
  • The behavior depends on timing you can't script (streaming LLM output, real network latency)
  • There's a visual quirk (a cursor effect, a plugin pop-up) that only appears in the live environment
For a browser demo
playwright-recording
skill, not this one. For a real desktop
screen_recorder
tool or Cap via
cap_recorder
.
适合使用合成录制的场景:
  • 演示内容是终端/CLI/编码会话,且命令和输出可预测
  • 用户想要精致的教程风格(清晰排版、浮动提示框、光标闪烁)
  • 安装流程演示、设置演示、API密钥配置、
    make
    目标、
    git clone
    流程
  • 需要与旁白严格同步——每个命令必须在特定节拍出现
  • 希望结果可重现(重新渲染得到完全相同的像素)
  • 用户的实际桌面有隐私应用/窗口,否则需要裁剪处理
适合捕获真实屏幕的场景:
  • 演示内容是无法模拟的真实应用UI(Figma、Photoshop、带有实时状态的Web应用、浏览器流程)
  • 用户明确要求录制他们的实际屏幕
  • 行为依赖无法脚本化的时序(流式LLM输出、真实网络延迟)
  • 存在仅在实时环境中才会出现的视觉特性(光标效果、插件弹窗)
浏览器演示 → 使用
playwright-recording
技能,而非本技能。 真实桌面录制 → 使用
screen_recorder
工具或通过
cap_recorder
调用Cap。

The component —
TerminalScene

组件——
TerminalScene

Located at:
remotion-composer/src/components/TerminalScene.tsx
Exported from:
remotion-composer/src/components/index.ts
Wired in dispatch:
remotion-composer/src/Explainer.tsx
(
if (cut.type === "terminal_scene")
)
Props:
ts
interface TerminalSceneProps {
  title?: string;           // shown in the window title bar
  steps: TerminalStep[];    // the timeline
  prompt?: string;          // "$", ">", etc.
  accentColor?: string;     // pill + prompt glow
  backgroundColor?: string;
}
Step kinds:
ts
{ kind: "cmd",   text: string, typeSpeed?: number, holdSeconds?: number }
{ kind: "out",   text: string, holdSeconds?: number }
{ kind: "pause", seconds: number }
{ kind: "pill",  text: string, color?: string, durationSeconds?: number }
  • cmd
    — prints the prompt, types the text character-by-character (
    typeSpeed
    is seconds per character, default 0.035), then holds for
    holdSeconds
    (default 0.3)
  • out
    — a line of program output, reveals instantly with a short fade-in
  • pause
    — dead time. Terminal holds on last visible state. USE THIS TO SYNC WITH NARRATION.
  • pill
    — non-blocking floating badge (top-right). Spring-in, hold, spring-out. Does NOT advance the cursor — the next step runs in parallel.
位置:
remotion-composer/src/components/TerminalScene.tsx
导出位置:
remotion-composer/src/components/index.ts
调度位置:
remotion-composer/src/Explainer.tsx
if (cut.type === "terminal_scene")
处)
Props:
ts
interface TerminalSceneProps {
  title?: string;           // 显示在窗口标题栏中
  steps: TerminalStep[];    // 时间线
  prompt?: string;          // "$", ">", 等
  accentColor?: string;     // 提示框 + 提示符高亮色
  backgroundColor?: string;
}
步骤类型:
ts
{ kind: "cmd",   text: string, typeSpeed?: number, holdSeconds?: number }
{ kind: "out",   text: string, holdSeconds?: number }
{ kind: "pause", seconds: number }
{ kind: "pill",  text: string, color?: string, durationSeconds?: number }
  • cmd
    — 显示提示符,逐字符输入文本(
    typeSpeed
    为每个字符的输入时长,默认0.035秒),然后保持
    holdSeconds
    时长(默认0.3秒)
  • out
    — 一行程序输出,通过短时间淡入立即显示
  • pause
    — 停顿时间。终端保持最后可见状态。使用此步骤与旁白同步。
  • pill
    — 非阻塞式浮动徽章(右上角)。弹入、保持、弹出。不会推进光标——下一步会并行运行。

Authoring pattern

创作模式

Author a new scene by adding a cut to
build_composition.py
(or your equivalent props builder):
python
install_steps = [
    {"kind": "pause", "seconds": 7.0},                 # wait for intro narration
    {"kind": "cmd", "text": "git clone https://github.com/calesthio/OpenMontage.git",
     "typeSpeed": 0.045, "holdSeconds": 0.3},
    {"kind": "out", "text": "Cloning into 'OpenMontage'..."},
    {"kind": "out", "text": "remote: Enumerating objects: 2847, done."},
    {"kind": "pill", "text": "repo cloned", "color": "#34D399", "durationSeconds": 2.6},
    {"kind": "pause", "seconds": 3.8},                 # bridge to next narration cue
    # ...
]

cuts.append({
    "id": "install-terminal",
    "type": "terminal_scene",
    "terminalTitle": "bash — OpenMontage setup",
    "prompt": "$",
    "accentColor": "#22D3EE",
    "steps": install_steps,
    "in_seconds": 50.0,
    "out_seconds": 110.0,
})
通过向
build_composition.py
(或等效的Props构建器)添加一个片段来创作新场景:
python
install_steps = [
    {"kind": "pause", "seconds": 7.0},                 // 等待开场旁白
    {"kind": "cmd", "text": "git clone https://github.com/calesthio/OpenMontage.git",
     "typeSpeed": 0.045, "holdSeconds": 0.3},
    {"kind": "out", "text": "Cloning into 'OpenMontage'..."},
    {"kind": "out", "text": "remote: Enumerating objects: 2847, done."},
    {"kind": "pill", "text": "repo cloned", "color": "#34D399", "durationSeconds": 2.6},
    {"kind": "pause", "seconds": 3.8},                 // 过渡到下一个旁白提示点
    // ...
]

cuts.append({
    "id": "install-terminal",
    "type": "terminal_scene",
    "terminalTitle": "bash — OpenMontage setup",
    "prompt": "$",
    "accentColor": "#22D3EE",
    "steps": install_steps,
    "in_seconds": 50.0,
    "out_seconds": 110.0,
})

THE RULE: pace with narration, never ahead

规则:与旁白同步节奏,绝不超前

The #1 failure mode: steps run continuously and burn through all content in the first 40% of the scene, leaving the terminal frozen for the remaining 60%. This is what killed the v3 first pass — the capability menu rendered at t=80s but narration didn't announce it until t=92s.
Do this instead:
  1. Know your narration cues — for each scene, write down the exact video-time each narration segment starts.
  2. Start with a pause that reaches the first narration cue before any command types.
  3. Time each command to land with its narration line
    cmd
    should start typing the moment narration says its line, not before.
  4. Put pauses between command groups that bridge to the next narration cue.
  5. End with a closer hold — a pause long enough that the final state is readable after narration ends.
Sanity-check your steps before rendering — every minute of Remotion render is precious. Sum the step durations and verify they equal scene duration:
python
import math
def trace(steps, scene_start, fps=30):
    t = 0.0
    for s in steps:
        k = s["kind"]
        if k == "cmd":
            tf = math.ceil(len(s["text"]) * s.get("typeSpeed", 0.035) * fps)
            t += tf / fps + s.get("holdSeconds", 0.3)
        elif k == "out":
            t += max(2, math.ceil(0.08 * fps)) / fps + s.get("holdSeconds", 0.15)
        elif k == "pause":
            t += s["seconds"]
        # "pill" is non-blocking — does NOT advance cursor
        print(f"  {t + scene_start:6.2f}s  {k}: {s.get('text', '')[:40]}")
trace(install_steps, 50)
Look at the output column. Each narration cue's video-time must appear adjacent to the command/output it announces. If a command lands 10s before or after its cue, adjust pauses.
See
lib/verify_scene_pacing.py
for a reusable version of this script.
最常见的失败模式: 步骤持续运行,在场景前40%的时间内就完成所有内容,导致终端在剩余60%的时间内处于冻结状态。这正是v3版本首次尝试失败的原因——功能菜单在第80秒渲染完成,但旁白直到第92秒才提及它。
正确做法:
  1. 了解旁白提示点 — 针对每个场景,写下每个旁白片段开始的精确视频时间。
  2. 以停顿开头 — 在输入任何命令前,先添加一个停顿,直到第一个旁白提示点出现。
  3. 让每个命令与对应旁白同步
    cmd
    应该在旁白提及该命令的瞬间开始输入,而不是提前。
  4. 在命令组之间添加停顿 — 过渡到下一个旁白提示点。
  5. 结尾保持停顿 — 添加足够长的停顿,确保旁白结束后最终状态仍可阅读。
渲染前检查步骤节奏 — Remotion的每一分钟渲染都很宝贵。计算所有步骤的总时长,验证是否等于场景时长:
python
import math
def trace(steps, scene_start, fps=30):
    t = 0.0
    for s in steps:
        k = s["kind"]
        if k == "cmd":
            tf = math.ceil(len(s["text"]) * s.get("typeSpeed", 0.035) * fps)
            t += tf / fps + s.get("holdSeconds", 0.3)
        elif k == "out":
            t += max(2, math.ceil(0.08 * fps)) / fps + s.get("holdSeconds", 0.15)
        elif k == "pause":
            t += s["seconds"]
        # "pill" 是非阻塞的——不会推进光标
        print(f"  {t + scene_start:6.2f}s  {k}: {s.get('text', '')[:40]}")
trace(install_steps, 50)
查看输出列。每个旁白提示点的视频时间必须与它所介绍的命令/输出相邻。如果命令比提示点早或晚10秒出现,请调整停顿时长。
可复用的脚本版本请查看
lib/verify_scene_pacing.py

Design rules (inherited from the v3 retune)

设计规则(继承自v3版本调整)

  • Intro pause — every terminal scene opens with at least 2s of empty-terminal-with-blinking-cursor before anything types. The viewer needs to register the window.
  • Pill timing — a pill should fire at the exact moment its named event completes on screen (e.g.,
    repo cloned
    immediately after the last
    Receiving objects
    line). Pills are your substitute for real-world UI notifications.
  • Command hold after typing — keep
    holdSeconds
    ≥ 0.3 on every
    cmd
    so viewers register the completed command before the first output scrolls in.
  • Output cadence — space
    holdSeconds
    on output lines between 0.4 and 1.0. Output that flies too fast feels like a bug; output that crawls feels boring.
  • Auto-scroll works — the terminal holds the most recent 18 lines. Don't worry about off-screen content.
  • Cursor blinks only on the latest command line while typing + a ~0.2s tail after typing completes.
  • 开场停顿 — 每个终端场景在开始输入内容前,至少要有2秒的空终端+闪烁光标时间。观众需要时间识别窗口。
  • 提示框时机 — 提示框应在屏幕上对应事件完成的瞬间弹出(例如,
    repo cloned
    应在最后一行
    Receiving objects
    输出后立即出现)。提示框是真实世界UI通知的替代方案。
  • 命令输入后的保持时长 — 每个
    cmd
    holdSeconds
    应≥0.3秒,以便观众在第一行输出滚动前识别完整命令。
  • 输出节奏 — 输出行的
    holdSeconds
    应设置在0.4到1.0之间。输出过快会感觉像bug;过慢则会显得乏味。
  • 自动滚动生效 — 终端保留最近的18行内容。无需担心屏幕外的内容。
  • 光标仅在输入最新命令行时闪烁 + 输入完成后约0.2秒的延迟

ProviderChip
(companion component)

ProviderChip
(配套组件)

The
.agents/skills/synthetic-screen-recording
pattern also owns
ProviderChip
— a rotating badge overlay that cycles through a list of provider names at a fixed cadence. Used in the v3 showcase to cycle through all 11 AI video-gen providers during the "generated motion" section.
python
overlays.append({
    "type": "provider_chip",
    "providers": ["Veo 3.1", "Seedance 2.0", "Kling 2.5", ...],
    "cycleSeconds": 2.5,
    "position": "bottom-right",
    "accentColor": "#22D3EE",
    "label": "generated with",
    "in_seconds": 195.0,
    "out_seconds": 222.5,
})
Wired in dispatch at:
remotion-composer/src/Explainer.tsx
overlay renderer (
overlay.type === "provider_chip"
).
.agents/skills/synthetic-screen-recording
模式还包含
ProviderChip
——一个旋转徽章覆盖层,以固定节奏循环显示一系列提供商名称。在v3展示项目中,用于在“生成动画”部分循环展示全部11个AI视频生成提供商。
python
overlays.append({
    "type": "provider_chip",
    "providers": ["Veo 3.1", "Seedance 2.0", "Kling 2.5", ...],
    "cycleSeconds": 2.5,
    "position": "bottom-right",
    "accentColor": "#22D3EE",
    "label": "generated with",
    "in_seconds": 195.0,
    "out_seconds": 222.5,
})
调度位置:
remotion-composer/src/Explainer.tsx
的覆盖层渲染器(
overlay.type === "provider_chip"
处)。

Adding new synthetic-UI components

添加新的合成UI组件

The pattern generalizes. When you need to fake another UI surface (Claude Code chat bubbles, a Jira ticket view, a GitHub PR diff, a Slack message, a VS Code status bar):
  1. Copy
    TerminalScene.tsx
    as a template.
  2. Define a
    steps
    interface for the relevant timeline primitives.
  3. Render each step by interpolating
    frame
    against cumulative start/end times.
  4. Wire it into
    Explainer.tsx
    's
    SceneRenderer
    dispatch with a new
    cut.type
    .
  5. Add the type to the
    Cut
    interface in
    Explainer.tsx
    and to
    components/index.ts
    .
  6. Add a section to this skill documenting it.
  7. Update
    remotion-composer/SCENE_TYPES.md
    with the new cut type.
该模式可以推广。当你需要模拟其他UI界面(Claude Code聊天气泡、Jira工单视图、GitHub PR差异、Slack消息、VS Code状态栏)时:
  1. 复制
    TerminalScene.tsx
    作为模板。
  2. 为相关时间线原语定义
    steps
    接口。
  3. 通过将
    frame
    与累积开始/结束时间插值来渲染每个步骤。
  4. Explainer.tsx
    SceneRenderer
    调度中添加新的
    cut.type
    ,将其接入。
  5. Explainer.tsx
    Cut
    接口和
    components/index.ts
    中添加该类型。
  6. 在本技能中添加一个部分对其进行文档说明。
  7. 更新
    remotion-composer/SCENE_TYPES.md
    ,添加新的片段类型。

Related skills

相关技能

  • .agents/skills/remotion
    — general Remotion authoring (hooks, springs, sequences)
  • .agents/skills/playwright-recording
    — real browser-flow capture for web apps
  • tools/capture/screen_recorder
    — ffmpeg-based desktop capture
  • tools/capture/cap_recorder
    — Cap.so polished desktop capture
  • skills/pipelines/screen-demo/asset-director.md
    — chooses between synthetic and real for a screen-demo project
  • .agents/skills/remotion
    — 通用Remotion创作(hooks、弹簧动画、序列)
  • .agents/skills/playwright-recording
    — 针对Web应用的真实浏览器流程捕获
  • tools/capture/screen_recorder
    — 基于ffmpeg的桌面捕获
  • tools/capture/cap_recorder
    — Cap.so的精致桌面捕获
  • skills/pipelines/screen-demo/asset-director.md
    — 为屏幕演示项目选择合成或真实录制

Provenance

来源

Introduced: OpenMontage showcase v3 render (2026-04-16). Original motivation: the v3 setup walkthrough section needed a 60-second install demo where every command aligned to Chirp 3 HD narration cues, and Windows-MCP-driven real capture was too flaky in practice. See
projects/openmontage-showcase/build_composition.py
for the reference implementation.
引入时间:OpenMontage展示项目v3版本渲染(2026-04-16)。最初动机:v3版本的安装流程演示部分需要一个60秒的安装演示,其中每个命令都要与Chirp 3 HD旁白提示点对齐,而Windows-MCP驱动的真实捕获在实践中过于不稳定。参考实现请查看
projects/openmontage-showcase/build_composition.py