motion-canvas-agent

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Motion Canvas Agent Plugin

Motion Canvas Agent 插件

Full programmatic control of a running Motion Canvas editor via HTTP. Seek to any frame, capture screenshots, inspect the scene graph, change resolution/fps/background, and trigger rendering — all without browser automation.

通过HTTP对运行中的Motion Canvas编辑器进行全面的程序化控制。无需浏览器自动化即可实现跳转至任意帧、捕获截图、检查场景图、修改分辨率/帧率/背景以及触发渲染等操作。

Architecture

架构

Agent (curl/script)  ──HTTP──▶  Vite Server (agent-plugin.ts)  ──HMR WS──▶  Browser (agent-client.ts)
                                      │                                           │
                                      ▼                                           ▼
                                screenshots/                               Player + Renderer
                                (PNG files)                                + ProjectMetadata

Agent (curl/脚本)  ──HTTP──▶  Vite Server (agent-plugin.ts)  ──HMR WS──▶  浏览器 (agent-client.ts)
                                      │                                           │
                                      ▼                                           ▼
                                截图/                               Player + Renderer
                                (PNG 文件)                                + ProjectMetadata

Setup

安装步骤

1. Copy plugin files into your project

1. 将插件文件复制到你的项目中

```
assets/agent-plugin.ts
```
→ project root
```
assets/agent-client.ts
```
→
```
src/
```

```
assets/agent-plugin.ts
```
→ 项目根目录
```
assets/agent-client.ts
```
→
```
src/
```
目录

2. vite.config.ts

import {defineConfig} from 'vite';
import motionCanvas from '@motion-canvas/vite-plugin';
import {agentPlugin} from './agent-plugin';

export default defineConfig({
  plugins: [
    motionCanvas(),
    agentPlugin({screenshotDir: './screenshots'}),
  ],
});

import {defineConfig} from 'vite';
import motionCanvas from '@motion-canvas/vite-plugin';
import {agentPlugin} from './agent-plugin';

export default defineConfig({
  plugins: [
    motionCanvas(),
    agentPlugin({screenshotDir: './screenshots'}),
  ],
});

3. project.ts

import {makeProject} from '@motion-canvas/core';
import {agentClient} from './agent-client';

export default makeProject({
  plugins: [agentClient()],
  scenes: [...],
});

import {makeProject} from '@motion-canvas/core';
import {agentClient} from './agent-client';

export default makeProject({
  plugins: [agentClient()],
  scenes: [...],
});

4. Start + open browser

4. 启动服务并打开浏览器

bash

npm start

bash

npm start

Open http://localhost:9000 in a browser

在浏览器中打开 http://localhost:9000

undefined

undefined

HTTP API Reference

HTTP API 参考文档

Base URL:

http://localhost:9000/__agent

基础URL：

http://localhost:9000/__agent

Playback Control

播放控制

Method	Endpoint	Body	Response
`GET`	`/status`	—	`{connected, frame, duration, fps, paused, sceneName, errors}`
`POST`	`/seek`	`{frame: 120}`	`{ok, frame}`
`POST`	`/next-frame`	—	`{ok, frame}`
`POST`	`/prev-frame`	—	`{ok, frame}`
`POST`	`/play`	—	`{ok}`
`POST`	`/pause`	—	`{ok}`

方法	端点	请求体	响应
`GET`	`/status`	—	`{connected, frame, duration, fps, paused, sceneName, errors}`
`POST`	`/seek`	`{frame: 120}`	`{ok, frame}`
`POST`	`/next-frame`	—	`{ok, frame}`
`POST`	`/prev-frame`	—	`{ok, frame}`
`POST`	`/play`	—	`{ok}`
`POST`	`/pause`	—	`{ok}`

Screenshots

截图

Method	Endpoint	Body	Response
`POST`	`/screenshot`	`{name: "my-shot"}`	`{ok, path, frame}`
`POST`	`/screenshot-base64`	—	`{ok, data, frame}`

方法	端点	请求体	响应
`POST`	`/screenshot`	`{name: "my-shot"}`	`{ok, path, frame}`
`POST`	`/screenshot-base64`	—	`{ok, data, frame}`

Settings

设置

Method	Endpoint	Body	Response
`GET`	`/settings`	—	`{background, range, size, previewFps, previewScale, renderingFps, renderingScale}`
`POST`	`/settings/background`	`{color: "#0F172A"}`	`{ok}`
`POST`	`/settings/range`	`{start: 0, end: 300}`	`{ok}`
`POST`	`/settings/size`	`{width: 1920, height: 1080}`	`{ok}`
`POST`	`/settings/preview-fps`	`{fps: 30}`	`{ok}`
`POST`	`/settings/preview-scale`	`{scale: 0.5}`	`{ok}`
`POST`	`/settings/rendering-fps`	`{fps: 60}`	`{ok}`
`POST`	`/settings/rendering-scale`	`{scale: 2}`	`{ok}`

方法	端点	请求体	响应
`GET`	`/settings`	—	`{background, range, size, previewFps, previewScale, renderingFps, renderingScale}`
`POST`	`/settings/background`	`{color: "#0F172A"}`	`{ok}`
`POST`	`/settings/range`	`{start: 0, end: 300}`	`{ok}`
`POST`	`/settings/size`	`{width: 1920, height: 1080}`	`{ok}`
`POST`	`/settings/preview-fps`	`{fps: 30}`	`{ok}`
`POST`	`/settings/preview-scale`	`{scale: 0.5}`	`{ok}`
`POST`	`/settings/rendering-fps`	`{fps: 60}`	`{ok}`
`POST`	`/settings/rendering-scale`	`{scale: 2}`	`{ok}`

Rendering

渲染

Method	Endpoint	Body	Response
`POST`	`/render`	`{fps?, range?}`	`{ok, message}`
`POST`	`/render/abort`	—	`{ok, message}`

方法	端点	请求体	响应
`POST`	`/render`	`{fps?, range?}`	`{ok, message}`
`POST`	`/render/abort`	—	`{ok, message}`

Scene Info

场景信息

Method	Endpoint	Body	Response
`GET`	`/scenes`	—	`{scenes: [{name}], currentScene}`
`GET`	`/scene-graph`	—	`{scene, graph}` (recursive node tree)
`GET`	`/threads`	—	`{scene, slides: [{name, id}]}`

方法	端点	请求体	响应
`GET`	`/scenes`	—	`{scenes: [{name}], currentScene}`
`GET`	`/scene-graph`	—	`{scene, graph}` (递归节点树)
`GET`	`/threads`	—	`{scene, slides: [{name, id}]}`

Errors

错误处理

Method	Endpoint	Body	Response
`GET`	`/errors`	—	`{errors: [{message, stack}]}`
`POST`	`/clear-errors`	—	`{ok}`

方法	端点	请求体	响应
`GET`	`/errors`	—	`{errors: [{message, stack}]}`
`POST`	`/clear-errors`	—	`{ok}`

Scene Graph Format

场景图格式

GET /__agent/scene-graph

returns a recursive tree:

json

{
  "scene": "my-scene",
  "graph": {
    "type": "View2D",
    "key": "my-scene/View2D[1]",
    "position": {"x": 960, "y": 540},
    "size": {"width": 1920, "height": 1080},
    "children": [
      {
        "type": "Rect",
        "position": {"x": 0, "y": 0},
        "size": {"width": 400, "height": 300},
        "fill": "#1E293B",
        "children": [
          {
            "type": "Txt",
            "text": "Hello World"
          }
        ]
      }
    ]
  }
}

Each node includes:

type

key

position

size

opacity

(if not 1),

fill

text

(if applicable), and

children

GET /__agent/scene-graph

返回递归树结构：

json

{
  "scene": "my-scene",
  "graph": {
    "type": "View2D",
    "key": "my-scene/View2D[1]",
    "position": {"x": 960, "y": 540},
    "size": {"width": 1920, "height": 1080},
    "children": [
      {
        "type": "Rect",
        "position": {"x": 0, "y": 0},
        "size": {"width": 400, "height": 300},
        "fill": "#1E293B",
        "children": [
          {
            "type": "Txt",
            "text": "Hello World"
          }
        ]
      }
    ]
  }
}

每个节点包含：

type

、

key

、

position

、

size

、

opacity

（如果不为1）、

fill

、

text

（如有）以及

children

。

AI Agent Workflow

AI Agent 工作流

bash

undefined

bash

undefined

1. Check connection

1. 检查连接状态

curl -s localhost:9000/__agent/status

2. Set resolution

2. 设置分辨率

curl -s -X POST localhost:9000/__agent/settings/size
-H "Content-Type: application/json" -d '{"width": 1920, "height": 1080}'

3. Seek to a frame

3. 跳转至指定帧

curl -s -X POST localhost:9000/__agent/seek
-H "Content-Type: application/json" -d '{"frame": 60}'

4. Screenshot

4. 截图

curl -s -X POST localhost:9000/__agent/screenshot
-H "Content-Type: application/json" -d '{"name": "check-1"}'

→ Read ./screenshots/check-1.png

→ 读取 ./screenshots/check-1.png

5. Inspect scene graph

5. 检查场景图

curl -s localhost:9000/__agent/scene-graph

6. Check which scene we're in

6. 查看当前所在场景

curl -s localhost:9000/__agent/scenes

7. Check errors

7. 检查错误信息

curl -s localhost:9000/__agent/errors

8. Change background

8. 修改背景颜色

curl -s -X POST localhost:9000/__agent/settings/background
-H "Content-Type: application/json" -d '{"color": "#000000"}'

9. Trigger render

9. 触发渲染

curl -s -X POST localhost:9000/__agent/render

undefined

curl -s -X POST localhost:9000/__agent/render

undefined

Security

安全说明

Localhost only: The agent API runs on the Vite dev server which binds to localhost by default. Do not expose it to the network (avoid
```
--host 0.0.0.0
```
in production)
Screenshot paths: The screenshot name is sanitized — path separators and dots are stripped, and the resolved path is verified to stay inside the configured
```
screenshotDir
```
Scene graph text: Text content extracted from the scene graph is sanitized (control characters stripped, truncated to 100 chars). Treat scene graph data as untrusted when processing it

仅本地访问：Agent API运行在Vite开发服务器上，默认绑定localhost。请勿将其暴露至公网（生产环境中避免使用
```
--host 0.0.0.0
```
）
截图路径：截图名称会被清洗——路径分隔符和点会被移除，且解析后的路径会被验证确保在配置的
```
screenshotDir
```
目录内
场景图文本：从场景图提取的文本内容会被清洗（移除控制字符，截断至100字符）。处理场景图数据时请将其视为不可信内容

How It Works

工作原理

Runtime plugin (
```
agentClient()
```
) uses
```
Plugin.player()
```
,
```
Plugin.renderer()
```
, and
```
Plugin.project()
```
callbacks to capture the Player, Renderer, and Project instances
Player provides seek, frame stepping, playback control, current scene, duration
ProjectMetadata provides
```
shared
```
(background, range, size),
```
preview
```
(fps, scale), and
```
rendering
```
(fps, scale, exporter) settings via
```
.get()
```
/
```
.set()
```
Renderer provides
```
render(settings)
```
and
```
abort()
```
for export
Scene graph is traversed via
```
scene.getView().peekChildren()
```
recursively, serializing type, position, size, fill, text, opacity
Communication uses Vite's HMR WebSocket

运行时插件 (
```
agentClient()
```
) 使用
```
Plugin.player()
```
、
```
Plugin.renderer()
```
和
```
Plugin.project()
```
回调来捕获Player、Renderer和Project实例
Player 提供跳转帧、逐帧步进、播放控制、当前场景、时长等功能
ProjectMetadata 通过
```
.get()
```
/
```
.set()
```
方法提供
```
shared
```
（背景、范围、尺寸）、
```
preview
```
（帧率、缩放比例）和
```
rendering
```
（帧率、缩放比例、导出器）设置
Renderer 提供
```
render(settings)
```
和
```
abort()
```
方法用于导出
场景图 通过
```
scene.getView().peekChildren()
```
递归遍历，序列化类型、位置、尺寸、填充色、文本、透明度等信息
通信使用Vite的HMR WebSocket

Files

文件说明

File	Copy to	Purpose
`assets/agent-plugin.ts`	Project root	Vite plugin — HTTP endpoints + WS relay
`assets/agent-client.ts`	`src/`	Runtime plugin — Player/Renderer/Project control + canvas capture

文件	复制路径	用途
`assets/agent-plugin.ts`	项目根目录	Vite插件——HTTP端点 + WS中继
`assets/agent-client.ts`	`src/` 目录	运行时插件——Player/Renderer/Project控制 + 画布捕获