framefields

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

framefields

framefields

Overview

概述

framefields
is a code-first video SDK. A video is a TypeScript
Composition
(a tree of layers with keyframed animation) that renders natively on WebGPU (Dawn) in Node.js, with no browser, Puppeteer, or Chromium. The engine is the
framefields
npm package; everything below runs in the user's own project.
This skill takes a project from nothing to a verified first render. For anything past that, load the specialist skill:
TaskSkill
Scenes, layout, typography, animation, beat grids, film structure
framefields-compose
Post-processing, color grading, VFX, on-device vision (tracking, cutouts, pose)
framefields-effects
Frame inspection, frame grids, pixel tests, video export and QA
framefields-render

framefields
是一款代码优先的视频SDK。视频本质上是一个TypeScript
Composition
(带有关键帧动画的图层树),可在Node.js中通过WebGPU(Dawn)原生渲染,无需浏览器、Puppeteer或Chromium。其引擎是npm包
framefields
;以下所有内容都在用户自己的项目中运行。
本技能可引导从零开始搭建项目并完成首次可验证的渲染。超出此范围的需求,请加载对应的专项技能:
任务技能
场景、布局、排版、动画、节拍网格、影片结构
framefields-compose
后期处理、调色、视觉特效、端侧视觉(追踪、抠图、姿态估计)
framefields-effects
帧检查、帧网格、像素测试、视频导出与质量检测
framefields-render

1. Check the environment

1. 环境检查

  • Node.js 22 or later. Run
    node --version
    . Older versions fail at import.
  • A WebGPU-capable GPU. macOS uses Metal; Linux and Windows use Vulkan. Headless CI machines without a GPU can't render.
  • An ES module project. The project's
    package.json
    needs
    "type": "module"
    .
  • Node.js 22 或更高版本。 运行
    node --version
    查看版本。更低版本会在导入时失败。
  • 支持WebGPU的GPU。 macOS使用Metal;Linux和Windows使用Vulkan。无GPU的无头CI机器无法进行渲染。
  • ES module项目。 项目的
    package.json
    中需要包含
    "type": "module"
    。

2. Install

2. 安装

Use the package manager the project already uses (look for
pnpm-lock.yaml
,
yarn.lock
,
bun.lock
, or
package-lock.json
).
Write
package.json
before installing.
npm 11, pnpm 10, and bun block dependency install scripts unless the project approves them, and framefields needs exactly one:
skia-canvas
downloads its native binary in its install script. Without it,
import "framefields"
fails with
Cannot find module '.../skia.node'
. The other packages that ship install scripts work without them, so deny them to keep installs quiet and skip downloads framefields never uses.
For a new project, create
package.json
with all three blocks. In an existing project, merge them into its
package.json
, keeping any entries it already has:
json
{
  "name": "my-film",
  "private": true,
  "type": "module",
  "allowScripts": {
    "skia-canvas": true,
    "sharp": false,
    "webgpu": false,
    "onnxruntime-node": false,
    "node-av": false,
    "esbuild": false
  },
  "pnpm": {
    "onlyBuiltDependencies": ["skia-canvas"],
    "ignoredBuiltDependencies": ["sharp", "webgpu", "onnxruntime-node", "node-av", "esbuild"]
  },
  "trustedDependencies": ["skia-canvas"]
}
allowScripts
is read by npm,
pnpm
by pnpm, and
trustedDependencies
by bun; each tool ignores the others' fields. Yarn runs install scripts by default and needs none of them. Then install:
bash
npm install framefields
npm install --save-dev tsx typescript @types/node
The install must finish without an
install-scripts
warning. If framefields was installed before the approval was in place, add it and rerun the skipped script with
npm rebuild skia-canvas
(
pnpm rebuild skia-canvas
). Never approve every script wholesale (
npm install-scripts approve --all
).
A minimal
tsconfig.json
:
json
{
  "compilerOptions": {
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "target": "ES2022",
    "strict": true,
    "skipLibCheck": true,
    "noEmit": true,
    "types": ["node"]
  },
  "include": ["src/**/*"]
}
使用项目已有的包管理器(可通过查找
pnpm-lock.yaml
、
yarn.lock
、
bun.lock
或
package-lock.json
判断)。
安装前先编写
package.json
。
npm 11、pnpm 10和bun默认会阻止依赖的安装脚本执行,除非项目明确批准,而framefields只需要一个安装脚本:
skia-canvas
会在其安装脚本中下载其原生二进制文件。如果没有这个脚本,
import "framefields"
会报错
Cannot find module '.../skia.node'
。其他附带安装脚本的包不需要执行脚本也能正常工作,因此请禁止它们的脚本执行,以保持安装过程简洁,并跳过framefields不会用到的下载内容。
对于新项目,创建包含以下三个配置块的
package.json
。对于现有项目,请将这些配置合并到其
package.json
中,保留原有已有的条目:
json
{
  "name": "my-film",
  "private": true,
  "type": "module",
  "allowScripts": {
    "skia-canvas": true,
    "sharp": false,
    "webgpu": false,
    "onnxruntime-node": false,
    "node-av": false,
    "esbuild": false
  },
  "pnpm": {
    "onlyBuiltDependencies": ["skia-canvas"],
    "ignoredBuiltDependencies": ["sharp", "webgpu", "onnxruntime-node", "node-av", "esbuild"]
  },
  "trustedDependencies": ["skia-canvas"]
}
allowScripts
由npm读取,
pnpm
配置块由pnpm读取,
trustedDependencies
由bun读取;每个工具都会忽略其他工具的配置字段。Yarn默认会运行安装脚本,因此不需要这些配置。然后执行安装:
bash
npm install framefields
npm install --save-dev tsx typescript @types/node
安装完成时不能出现
install-scripts
相关警告。如果在批准配置生效前就已经安装了framefields,请添加配置后运行
npm rebuild skia-canvas
(pnpm则用
pnpm rebuild skia-canvas
)重新执行被跳过的脚本。切勿批量批准所有脚本(
npm install-scripts approve --all
)。
最简
tsconfig.json
配置示例:
json
{
  "compilerOptions": {
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "target": "ES2022",
    "strict": true,
    "skipLibCheck": true,
    "noEmit": true,
    "types": ["node"]
  },
  "include": ["src/**/*"]
}

3. Scaffold the project

3. 项目脚手架搭建

Keep the composition separate from the script that renders it, so tests and the render script build the same film:
src/
  theme.ts    # canvas size, fps, colors, fonts: the only place for magic numbers
  film.ts     # buildFilm(): assembles the Composition
  render.ts   # renders frames for review, a live preview, or the full video
assets/
  fonts/      # .ttf / .otf files the film uses
output/       # renders (add to .gitignore)
Text needs a real font file. Ask the user which font to use.
typescript
// src/theme.ts
import { FontManager } from "framefields";

export const W = 1920;
export const H = 1080;
export const FPS = 30;
export const INK = "#0E0D0C";
export const PAPER = "#EEE9E0";
export const SANS = "Inter";

export async function registerFonts(): Promise<void> {
  await FontManager.register({ family: SANS, source: "assets/fonts/Inter.ttf" });
}
typescript
// src/film.ts
import { Composition, Layer, LayerAnimation } from "framefields";
import { FPS, H, INK, PAPER, registerFonts, SANS, W } from "./theme.js";

const DURATION = 3 * FPS; // frames

export async function buildFilm(): Promise<Composition> {
  // Fonts must be registered before any text layer is built.
  await registerFonts();

  const film = new Composition({
    width: W,
    height: H,
    fps: FPS,
    durationMs: (DURATION / FPS) * 1000,
    backgroundColor: INK,
  });

  film.add(
    Layer.text("Hello, framefields", {
      id: "title",
      position: "absolute",
      x: 0,
      y: H / 2 - 60,
      width: W,
      height: 120,
      fontFamily: SANS,
      fontSize: 96,
      fontWeight: 700,
      fill: PAPER,
      align: "center",
    }).animate(
      LayerAnimation.create()
        .fadeIn(0, 20, "power2.out")
        .fromTo("y", H / 2 - 20, H / 2 - 60, { start: 0, end: 30, ease: "expo.out" }),
    ),
  );

  return film;
}
typescript
// src/render.ts
import fs from "node:fs/promises";
import path from "node:path";
import { HeadlessMediaRenderer, startPreview } from "framefields";
import { buildFilm } from "./film.js";

const OUT = path.resolve(import.meta.dirname, "../output");
const [mode = "video", ...args] = process.argv.slice(2);

if (mode === "preview") {
  // The page runs film.ts itself and renders it with WebGPU in the browser.
  const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" });
  console.log(`Preview at ${session.url}`);
  await session.closed; // until its tab closes or the next preview replaces it
  process.exit(0);
}

const film = await buildFilm();
await fs.mkdir(OUT, { recursive: true });

if (mode === "frames") {
  const renderer = new HeadlessMediaRenderer();
  for (const frame of args.map(Number)) {
    const png = await film.renderFrame({ frame, renderer });
    await fs.writeFile(path.join(OUT, `f${String(frame).padStart(4, "0")}.png`), png);
  }
} else if (mode === "grid") {
  const png = await film.renderFrameGrid({
    frames: args.length ? args.map(Number) : [0, 10, 20, 30, 45, 60, 75, 89],
    columns: 4,
    cellWidth: 480,
    showLabels: true,
  });
  await fs.writeFile(path.join(OUT, "grid.png"), png);
} else {
  const result = await film.renderVideo({
    outputPath: path.join(OUT, "output.mp4"),
    quality: "high",
  });
  await result.cleanup?.();
  console.log(`Exported ${result.filePath}`);
}

// Native GPU and encoder threads keep Node alive; always exit explicitly.
process.exit(0);
Add scripts to
package.json
:
json
"scripts": {
  "typecheck": "tsc --noEmit -p .",
  "frames": "tsx src/render.ts frames",
  "grid": "tsx src/render.ts grid",
  "preview": "tsx src/render.ts preview",
  "render": "tsx src/render.ts"
}
请将composition与渲染它的脚本分开存放,这样测试和渲染脚本使用的是同一份影片代码:
src/
  theme.ts    # canvas size, fps, colors, fonts: the only place for magic numbers
  film.ts     # buildFilm(): assembles the Composition
  render.ts   # renders frames for review, a live preview, or the full video
assets/
  fonts/      # .ttf / .otf files the film uses
output/       # renders (add to .gitignore)
文本需要使用真实的字体文件。请询问用户要使用哪种字体。
typescript
// src/theme.ts
import { FontManager } from "framefields";

export const W = 1920;
export const H = 1080;
export const FPS = 30;
export const INK = "#0E0D0C";
export const PAPER = "#EEE9E0";
export const SANS = "Inter";

export async function registerFonts(): Promise<void> {
  await FontManager.register({ family: SANS, source: "assets/fonts/Inter.ttf" });
}
typescript
// src/film.ts
import { Composition, Layer, LayerAnimation } from "framefields";
import { FPS, H, INK, PAPER, registerFonts, SANS, W } from "./theme.js";

const DURATION = 3 * FPS; // frames

export async function buildFilm(): Promise<Composition> {
  // Fonts must be registered before any text layer is built.
  await registerFonts();

  const film = new Composition({
    width: W,
    height: H,
    fps: FPS,
    durationMs: (DURATION / FPS) * 1000,
    backgroundColor: INK,
  });

  film.add(
    Layer.text("Hello, framefields", {
      id: "title",
      position: "absolute",
      x: 0,
      y: H / 2 - 60,
      width: W,
      height: 120,
      fontFamily: SANS,
      fontSize: 96,
      fontWeight: 700,
      fill: PAPER,
      align: "center",
    }).animate(
      LayerAnimation.create()
        .fadeIn(0, 20, "power2.out")
        .fromTo("y", H / 2 - 20, H / 2 - 60, { start: 0, end: 30, ease: "expo.out" }),
    ),
  );

  return film;
}
typescript
// src/render.ts
import fs from "node:fs/promises";
import path from "node:path";
import { HeadlessMediaRenderer, startPreview } from "framefields";
import { buildFilm } from "./film.js";

const OUT = path.resolve(import.meta.dirname, "../output");
const [mode = "video", ...args] = process.argv.slice(2);

if (mode === "preview") {
  // The page runs film.ts itself and renders it with WebGPU in the browser.
  const session = await startPreview({ entry: new URL("./film.ts", import.meta.url), export: "buildFilm" });
  console.log(`Preview at ${session.url}`);
  await session.closed; // until its tab closes or the next preview replaces it
  process.exit(0);
}

const film = await buildFilm();
await fs.mkdir(OUT, { recursive: true });

if (mode === "frames") {
  const renderer = new HeadlessMediaRenderer();
  for (const frame of args.map(Number)) {
    const png = await film.renderFrame({ frame, renderer });
    await fs.writeFile(path.join(OUT, `f${String(frame).padStart(4, "0")}.png`), png);
  }
} else if (mode === "grid") {
  const png = await film.renderFrameGrid({
    frames: args.length ? args.map(Number) : [0, 10, 20, 30, 45, 60, 75, 89],
    columns: 4,
    cellWidth: 480,
    showLabels: true,
  });
  await fs.writeFile(path.join(OUT, "grid.png"), png);
} else {
  const result = await film.renderVideo({
    outputPath: path.join(OUT, "output.mp4"),
    quality: "high",
  });
  await result.cleanup?.();
  console.log(`Exported ${result.filePath}`);
}

// Native GPU and encoder threads keep Node alive; always exit explicitly.
process.exit(0);
向
package.json
中添加脚本:
json
"scripts": {
  "typecheck": "tsc --noEmit -p .",
  "frames": "tsx src/render.ts frames",
  "grid": "tsx src/render.ts grid",
  "preview": "tsx src/render.ts preview",
  "render": "tsx src/render.ts"
}

4. Verify, cheapest step first

4. 验证,优先执行低成本步骤

Never go straight to a full video render. Each step catches problems in seconds that a video render takes minutes to show:
  1. npm run typecheck
  2. npm run frames -- 0 30 60
    and look at the PNGs in
    output/
    . A black frame usually means a missing font, a layer outside the canvas, or opacity stuck at 0.
  3. npm run grid
    and check the motion reads smoothly across the contact sheet.
  4. npm run preview
    when a person is watching: they play the film with sound before you export (see below).
  5. npm run render
    once frames and motion look right.
Show the user the frame PNGs or the grid before the full render.
framefields-render
covers pixel-probe tests, render QA (
renderVideo({ qa })
), and diagnosing blank frames.
切勿直接进行完整视频渲染。以下每个步骤都能在数秒内发现问题,而这些问题如果等到视频渲染时才暴露,需要花费数分钟:
  1. npm run typecheck
  2. 执行
    npm run frames -- 0 30 60
    ,查看
    output/
    目录下的PNG文件。出现黑帧通常意味着字体缺失、图层超出画布范围,或者不透明度被设为0。
  3. 执行
    npm run grid
    ,检查联系表上的动画过渡是否流畅。
  4. 当有用户在场时执行
    npm run preview
    :他们可以在导出前播放带声音的影片(见下文)。
  5. 确认帧和动画都正常后,执行
    npm run render
    。
在完整渲染前,先向用户展示帧PNG或帧网格图。
framefields-render
技能涵盖像素探测测试、渲染质量检测(
renderVideo({ qa })
)以及黑帧问题排查。

Live preview

实时预览

npm run preview
serves a localhost player (picture, sound, timeline) and prints its URL. The page runs
film.ts
itself and renders with WebGPU in real time, so the user sees the real film; a slow device skips frames to keep up with the sound.
  • Run it as a background command and read its output: it says when the page opened, prints errors from the page (a failed build, a frame that threw, a missing font), and says why it stopped. It stops 5 s after its last tab closes; just run it again.
  • It doesn't open anything. If your app has a built-in browser (Claude Code, Codex), open the URL there; otherwise give the user the link.
    startPreview(..., { open: true })
    opens their default browser.
  • Reference assets from the file's URL. The page runs
    film.ts
    itself, so asset paths must resolve in the browser too: build them from the module's own URL,
    path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../assets/fonts")
    , rather than
    process.cwd()
    or a bare relative path. The preview serves any file inside the project (the folder with
    package.json
    ); pass
    root
    (one or more paths) to
    startPreview
    for fonts or media kept elsewhere.
  • After changing the film, run it again. It takes over the same URL, and the open tab reloads by itself at the same moment.
  • Link to a moment with
    #t=12.5
    (seconds). In a browser you can script,
    window.framefieldsPreview
    has
    seek(seconds)
    ,
    play()
    ,
    pause()
    and
    state()
    ;
    seek
    resolves once the frame is drawn, so screenshot after it.
  • The user can pin notes on the picture. Pausing and clicking (or dragging an area) adds a note at that spot and moment. Each one prints in the preview's output and is saved in
    .framefields/preview-notes/
    :
    notes.json
    , plus
    note-<id>.jpg
    , the frame with the spot marked. When the user says they left notes, read
    notes.json
    and look at each open note's picture before you edit. Then run the preview again so they can check the changes and tick them off.
    framefields-render
    has the details.
npm run preview
会启动一个本地播放器(包含画面、声音、时间轴)并打印其访问URL。页面会自行运行
film.ts
并通过WebGPU实时渲染,因此用户看到的是真实的影片效果;设备性能不足时会跳帧以保证声音同步。
  • 作为后台命令运行并查看输出:它会提示页面何时打开,打印页面产生的错误(构建失败、某帧抛出异常、字体缺失等),并说明停止运行的原因。最后一个标签页关闭5秒后它会自动停止;重新运行即可。
  • 它不会自动打开任何页面。如果你的应用有内置浏览器(Claude Code、Codex),请在其中打开该URL;否则将链接提供给用户。使用
    startPreview(..., { open: true })
    可以打开用户的默认浏览器。
  • 从文件的URL引用资源。 页面会自行运行
    film.ts
    ,因此资源路径必须也能在浏览器中解析:请基于模块自身的URL构建路径,例如
    path.resolve(path.dirname(fileURLToPath(import.meta.url)), "../assets/fonts")
    ,而不要使用
    process.cwd()
    或单纯的相对路径。预览服务会提供项目(包含
    package.json
    的文件夹)内的所有文件;如果字体或媒体文件存放在其他位置,请向
    startPreview
    传入
    root
    参数(一个或多个路径)。
  • 修改影片后,重新运行命令即可。它会复用同一个URL,已打开的标签页会在同一时间自动刷新。
  • 使用
    #t=12.5
    (单位为秒)链接到特定时间点。在可执行脚本的浏览器中,
    window.framefieldsPreview
    提供
    seek(seconds)
    、
    play()
    、
    pause()
    和
    state()
    方法;
    seek
    会在帧绘制完成后resolve,因此可以在其后执行截图操作。
  • 用户可以在画面上添加标注。 暂停后点击(或拖拽选中区域)即可在对应位置和时间点添加标注。每条标注都会打印在预览的输出中,并保存在
    .framefields/preview-notes/
    目录下:包括
    notes.json
    ,以及标注了对应位置的帧图片
    note-<id>.jpg
    。当用户说他们留下了标注时,请先读取
    notes.json
    并查看每条未处理标注的截图,然后再进行编辑。编辑完成后重新运行预览,以便用户检查修改并标记已完成。详细信息请参考
    framefields-render
    技能。

5. Grow the film

5. 扩展影片功能

Once the first render works, move to the specialist skills:
framefields-compose
for splitting the film into scene modules on a beat grid,
framefields-effects
for grading, grain, and vision-driven effects, and
framefields-render
for tests and delivery checks. Keep
theme.ts
as the single source of sizes, colors, fonts, and timing as the project grows.
首次渲染成功后,可转向使用专项技能:使用
framefields-compose
将影片拆分为基于节拍网格的场景模块,使用
framefields-effects
实现调色、颗粒感和视觉驱动特效,使用
framefields-render
进行测试和交付检查。随着项目扩展,请保持
theme.ts
作为尺寸、颜色、字体和时序的唯一数据源。