vision-tools

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

vision-tools

视觉工具集

Five local CLIs that give a text-only agent eyes. They read one shared vision config (

VISION_API_KEY

VISION_BASE_URL

VISION_MODEL

LANG

) — no extra credentials.

Pick the tool by the question you are answering:

Question	Tool
"What does this image show / say?"	`glance`
"Where is X?" — a thing you can name	`ground`
"Where are all the Xs?" — every instance of a kind	`detect`
"What is its exact shape, size, offset?"	`trace`
"Cut this box out as its own image file"	`crop`
"OCR this long screenshot / scrolling page / chat history"	`scripts/long_screenshot_ocr.py`
"Extract the icon/logo foreground as transparent PNG — manual region or auto (cropped+scaled screenshots)"	`scripts/extract_fg.py`
"Turn this HTML file into a screenshot"	`scripts/html_shot.py`
"Which colours dominate a region, and which palette value fits it?"	`scripts/dominant_colors.py`
A relation none of them return — a gap, a distance between two located things	code over the pixels (Pillow)

glance

answers what something is;

ground

and

detect

answer where. You give

ground

a description of a particular thing; you give

detect

a kind and it enumerates the instances.

Both give real coordinates, but they are not pixel-exact: the box arrives on a 0-1000 grid and is scaled to your image, so the last pixel or few are not reliable. That is accurate enough to crop with, to click, to compare positions against. When a number has to be exact,

trace

derives it from the actual pixels — offsets, sizes, shapes.

这是一套为纯文本Agent提供视觉能力的本地CLI工具集。它们共享同一个视觉配置（

VISION_API_KEY

VISION_BASE_URL

VISION_MODEL

LANG

）——无需额外凭证。

根据你要解决的问题选择对应的工具：

问题	工具
"这张图像展示了什么/包含什么文字？"	`glance`
"X在哪里？"——某个可命名的对象	`ground`
"所有的X都在哪里？"——某类对象的全部实例	`detect`
"它的确切形状、尺寸、偏移量是多少？"	`trace`
"把这个区域裁剪成独立的图像文件"	`crop`
"OCR识别长截图/滚动页面/聊天记录"	`scripts/long_screenshot_ocr.py`
"提取图标/Logo前景为透明PNG——手动指定区域或自动处理（裁剪+缩放后的截图）"	`scripts/extract_fg.py`
"将HTML文件转为截图"	`scripts/html_shot.py`
"某个区域的主色调是什么，哪个调色板颜色最匹配？"	`scripts/dominant_colors.py`
以上工具都无法返回的关系——比如两个定位对象之间的间隙、距离	基于像素编写代码（使用Pillow）

glance

回答对象是什么；

ground

和

detect

回答位置在哪里。你需要给

ground

提供特定对象的描述；给

detect

提供对象类别，它会枚举所有实例。

两者都会返回真实坐标，但并非像素级精确：坐标基于0-1000网格生成后缩放至你的图像尺寸，因此最后几个像素可能不可靠。但这个精度足以用于裁剪、点击、位置对比。如果需要精确数值，

trace

会从实际像素中提取——包括偏移量、尺寸、形状。

Use the provided tools before hand-rolled pixels

优先使用内置工具而非自定义像素代码

Everything this toolkit ships a tool for, call the tool — do not rewrite it with Pillow in the middle of a task. The CLIs exist so the same pixel work is not hand-coded differently every time:

cut a box out of an image →
```
crop
```
, not
```
Image.open(...).crop(...)
```
sample a region's palette →
```
scripts/dominant_colors.py
```
compare two images →
```
scripts/pixel_diff.py
```
vectorize to SVG →
```
trace
```
locate / inventory elements →
```
ground
```
/
```
detect
```
describe / OCR an image →
```
glance
```
safely split, OCR, and merge a long screenshot →
```
scripts/long_screenshot_ocr.py
```
HTML file to a screenshot →
```
scripts/html_shot.py
```

Hand-written Pillow is only for what none of them return: a relation between two things you already located (a gap, a distance), a resize or overlay, drawing. If you catch yourself writing

.crop()

.convert()

, or histogram code where one of the tools above fits, replace it with the tool call — same coordinates, same box format, and the output feeds the next tool directly.

该工具集提供的所有功能，都请直接调用对应工具——不要在任务中用Pillow重写实现。CLI工具的存在就是为了避免重复编写不同的像素处理代码：

从图像中裁剪区域 → 使用
```
crop
```
，而非
```
Image.open(...).crop(...)
```
采样区域调色板 → 使用
```
scripts/dominant_colors.py
```
对比两张图像 → 使用
```
scripts/pixel_diff.py
```
矢量化为SVG → 使用
```
trace
```
定位/盘点元素 → 使用
```
ground
```
/
```
detect
```
描述/OCR识别图像 → 使用
```
glance
```
安全拆分、OCR识别并合并长截图 → 使用
```
scripts/long_screenshot_ocr.py
```
HTML文件转截图 → 使用
```
scripts/html_shot.py
```

只有当以上工具都无法实现时，才需要手写Pillow代码：比如计算两个已定位对象之间的关系（间隙、距离）、调整尺寸、叠加图层、绘图等。如果你发现自己在编写

.crop()

、

.convert()

或直方图代码，而上述工具可以实现相同功能，请替换为工具调用——坐标格式一致，输出可直接传入下一个工具。

glance — ask about an image

glance — 图像问答

bash

glance <image>                                 # detailed description
glance <image> -q "<question>"                 # targeted question (qualitative only)
glance <image> --ocr                           # verbatim OCR
glance <image> --region X1,Y1,X2,Y2 -q "..."   # zoom into a crop
glance <img1> <img2> -q "..."                  # compare in ONE call

When you do compare with

glance

, pass all paths to one call — separate calls cannot see both images, so two descriptions compared afterwards are two hallucination surfaces, not a comparison.

--region

uploads only the crop, so small text and icons become readable.

But "what changed between these two?" is not a glance question. A one-word badge or a small shift is a rounding error to a vision model and exact to

scripts/pixel_diff.py

. Diff first to get the box, then

glance --region

that box to read what the change actually is.

For a tall scrolling screenshot, do not send the whole image through one OCR call and accept the model's downscaling loss. Run the long-screenshot workflow, which finds low-content cut bands, invokes

glance

on each chunk, uses structured extraction for chat histories, merges only duplicated overlap, and writes a boundary audit:

bash

python3 scripts/long_screenshot_ocr.py work/page.png -o work/page.ocr.md
python3 scripts/long_screenshot_ocr.py work/chat.png --mode chat --resume -o work/chat.ocr.md

Read

references/long-screenshot-ocr.md

before using it. It defines the verification pass for unsafe cuts and chat-message boundaries.

bash

glance <image>                                 # 生成详细描述
glance <image> -q "<question>"                 # 针对性问答（仅定性问题）
glance <image> --ocr                           # 逐字OCR识别
glance <image> --region X1,Y1,X2,Y2 -q "..."   # 放大指定裁剪区域进行问答
glance <img1> <img2> -q "..."                  # 单次调用对比多张图像

使用

glance

对比图像时，请将所有路径传入同一个调用——分开调用无法同时查看两张图像，事后对比两个描述会增加幻觉风险，而非直接对比。

--region

参数仅上传裁剪区域，因此小文本和图标会更清晰。

但“这两张图像有什么变化？”不是

glance

擅长的问题。一个单词的徽章或微小位移对视觉模型来说是误差，但对

scripts/pixel_diff.py

来说是精确的。先使用差异工具获取变化区域的框，再用

glance --region

查看具体变化内容。

对于长滚动截图，不要将整张图像传入单次OCR调用并接受模型的缩放损失。请运行长截图工作流，该工作流会找到低内容分割区域，对每个片段调用

glance

，针对聊天记录使用结构化提取，仅合并重复重叠部分，并生成边界审计报告：

bash

python3 scripts/long_screenshot_ocr.py work/page.png -o work/page.ocr.md
python3 scripts/long_screenshot_ocr.py work/chat.png --mode chat --resume -o work/chat.ocr.md

使用前请阅读

references/long-screenshot-ocr.md

，其中定义了不安全分割和聊天消息边界的验证流程。

ground — locate a named target

ground — 定位指定目标

bash

ground <image> "<target description>"
ground <image> "<target>" --region X1,Y1,X2,Y2

Output:

x1: .., y1: .., x2: .., y2: ..

in original-image pixels — with

--region

too (crop hits are mapped back).

If several boxes come back numbered, your description matched more than one element rather than picking out a single thing. Narrow it with what distinguishes the one you mean — its text, its position, the block it sits in — and ask again.

The box is a handle, not just an answer — it feeds the next call:

bash

$ ground screenshot.png "the send button"
x1: 1067, y1: 841, x2: 1108, y2: 881
$ glance screenshot.png --region 1067,841,1108,881 -q "is it enabled or greyed out?"

That two-step is how you inspect anything too small to survive a full-image pass.

bash

ground <image> "<target description>"
ground <image> "<target>" --region X1,Y1,X2,Y2

输出：

x1: .., y1: .., x2: .., y2: ..

（原始图像像素坐标）——即使使用

--region

参数，裁剪区域的结果也会映射回原始图像坐标。

如果返回多个带编号的框，说明你的描述匹配了多个元素，而非单个目标。请添加区分特征（比如文本内容、位置、所在区块）来缩小范围，重新调用。

返回的框不仅是答案，还可作为下一个调用的输入：

bash

$ ground screenshot.png "发送按钮"
x1: 1067, y1: 841, x2: 1108, y2: 881
$ glance screenshot.png --region 1067,841,1108,881 -q "它是启用状态还是灰色禁用状态？"

这种两步操作是检查过小而无法在全图识别中清晰显示的元素的可靠方法。

detect — find every instance of a kind

detect — 查找某类对象的所有实例

bash

detect <image>                        # every UI element
detect <image> "buttons"              # one kind only
detect <image> --region X1,Y1,X2,Y2   # inside one box

You name a particular thing for

ground

; you name a kind for

detect

and it enumerates the instances. Output is a numbered list with each item's visible text and box. A full-screen pass is a fast first draft — counts vary run to run on dense screens. For completeness, detect the layout blocks first, then

detect --region

each block.

bash

detect <image>                        # 识别所有UI元素
detect <image> "buttons"              # 仅识别按钮类元素
detect <image> --region X1,Y1,X2,Y2   # 仅在指定区域内识别

ground

需要指定单个对象的描述；

detect

需要指定对象类别，它会枚举所有实例。输出是带编号的列表，包含每个元素的可见文本和坐标框。全屏幕识别是快速初稿——在密集界面上每次运行的计数可能不同。为保证完整性，可先识别布局区块，再对每个区块调用

detect --region

。

trace — exact shape geometry (local, no vision API)

trace — 精确形状几何信息（本地工具，无需视觉API）

bash

trace <image>                                  # b/w spline SVG to stdout
trace <image> --polygon                        # boxy diagrams/wireframes
trace <image> --region X1,Y1,X2,Y2 -o out.svg  # crop first

Coordinates come from the actual pixels, not a model's estimate. Flat, high-contrast graphics only; text becomes curves (pair with

--ocr

when the text matters). Small images are upscaled automatically before tracing, so a 30px icon traces as readily as a screenshot — size is not a reason to skip the tool. Before shipping or reusing a traced SVG, read

references/restore-graphic.md

— it holds the reuse traps and the ship-vs-hand-write call.

bash

trace <image>                                  # 将黑白图像转为样条曲线SVG并输出到标准输出
trace <image> --polygon                        # 适用于方正的图表/线框图
trace <image> --region X1,Y1,X2,Y2 -o out.svg  # 先裁剪区域再转换

坐标来自实际像素，而非模型估算。仅适用于平坦、高对比度图形；文本会转为曲线（当文本重要时需搭配

--ocr

使用）。小图像会自动 upscale 后再转换，因此30px的图标和截图一样可以轻松转换——尺寸不是跳过该工具的理由。在交付或重用转换后的SVG前，请阅读

references/restore-graphic.md

，其中包含重用陷阱和交付与手写的决策依据。

crop — cut a pixel box out of an image (local, no vision API)

crop — 从图像中裁剪像素框（本地工具，无需视觉API）

bash

crop <image> --region X1,Y1,X2,Y2             # writes <image-stem>.crop.png next to the input
crop <image> --region X1,Y1,X2,Y2 -o out.png
crop <image> --region X1,Y1,X2,Y2 --scale 4   # upscale the cut-out 4x (LANCZOS) first

The same X1,Y1,X2,Y2 pixel boxes

ground

detect

print, clamped to the image bounds. Once a box is worth keeping — the same crop is about to feed

pixel_diff

dominant_colors

, and

trace

in turn — cut it to a file once and reuse it, instead of re-cropping in memory on every call.

--scale N

upscales the cut-out before writing (default output name becomes

<image-stem>.crop@Nx.png

): for icons too small for

ground

trace

to see clearly, crop with

--scale 4

, then run

ground

trace

on the upscaled file — coordinates it returns are in the upscaled grid, divide by

to map back to the original image. Requires the optional

pillow

bash

crop <image> --region X1,Y1,X2,Y2             # 在输入图像旁生成<image-stem>.crop.png
crop <image> --region X1,Y1,X2,Y2 -o out.png
crop <image> --region X1,Y1,X2,Y2 --scale 4   # 先将裁剪区域 upscale 4倍（使用LANCZOS算法）再保存

使用与

ground

detect

输出相同的X1,Y1,X2,Y2像素框，会自动限制在图像边界内。如果某个框需要重复使用——比如要依次传入

pixel_diff

、

dominant_colors

和

trace

——请先裁剪保存为文件，而非每次调用都在内存中重新裁剪。

--scale N

参数会在保存前放大裁剪区域（默认输出名为

<image-stem>.crop@Nx.png

）：对于

ground

trace

无法清晰识别的小图标，可使用

--scale 4

裁剪，再对放大后的文件调用

ground

trace

——返回的坐标基于放大后的网格，除以

即可映射回原始图像坐标。需要依赖可选的

pillow

库。

extract_fg — icon foreground as transparent PNG: manual region or auto (local, no vision API)

extract_fg — 将图标前景提取为透明PNG：手动指定区域或自动处理（本地工具，无需视觉API）

bash

undefined

bash

undefined

manual: you know the region (and optionally the background colour)

手动模式：已知目标区域（可选指定背景色）

python3 scripts/extract_fg.py shot.png --region X1,Y1,X2,Y2 -o icon.png python3 scripts/extract_fg.py shot.png --region X1,Y1,X2,Y2 --mode dark # grey/black line logos python3 scripts/extract_fg.py shot.png --region X1,Y1,X2,Y2 --exclude-color '#E6E6E6'

python3 scripts/extract_fg.py shot.png --region X1,Y1,X2,Y2 -o icon.png python3 scripts/extract_fg.py shot.png --region X1,Y1,X2,Y2 --mode dark # 适用于灰/黑色线条Logo python3 scripts/extract_fg.py shot.png --region X1,Y1,X2,Y2 --exclude-color '#E6E6E6'

auto:

crop --scale

cut-outs with the icon centred — no region needed

自动模式：使用

crop --scale

裁剪的居中图标——无需指定区域

crop shot.png --region X1,Y1,X2,Y2 --scale 4 -o d/icon1.png python3 scripts/extract_fg.py d/icon1.png d/icon2.png # writes <stem>.clean.png next to each input python3 scripts/extract_fg.py d/icon1.png --disc-radius 60 python3 scripts/extract_fg.py d/icon1.png --boxes "101,84,184,171"


Manual mode keeps every sufficiently large connected component of the
region (separate logo sub-shapes stay together; specks drop out). Auto mode
takes a `crop --scale` cut-out with the icon centred (disc + glyph): the
disc centre is the image centre, the disc radius defaults to
`min(w,h)/2 * 0.6`, and the disc colour is sampled from a ring around the
centre; that colour is excluded and the glyph is picked as the most
saturated among the three largest coloured components (white rings,
ripples, and text fall away), output as a 1:1 transparent PNG. When auto
inference fails, override the radius with `--disc-radius`, or pass a
`ground` box (in the upscaled grid) as `--boxes` to recentre and re-filter
by overlap. Multiple images may be passed at once (auto mode).
Requires the optional `pillow` (and `numpy` for auto mode).

crop shot.png --region X1,Y1,X2,Y2 --scale 4 -o d/icon1.png python3 scripts/extract_fg.py d/icon1.png d/icon2.png # 在每个输入文件旁生成<stem>.clean.png python3 scripts/extract_fg.py d/icon1.png --disc-radius 60 python3 scripts/extract_fg.py d/icon1.png --boxes "101,84,184,171"


手动模式保留区域内所有足够大的连通组件（Logo的子形状会保持完整；斑点会被过滤）。自动模式接受`crop --scale`裁剪的居中图标（圆形+符号）：圆心为图像中心，圆半径默认是`min(w,h)/2 * 0.6`，圆颜色从中心周围的环形区域采样；该颜色会被排除，符号会被选为三个最大颜色组件中饱和度最高的那个（白色圆环、波纹和文本会被过滤），输出为1:1的透明PNG。当自动识别失败时，可使用`--disc-radius`覆盖半径，或传入`ground`返回的框（基于放大后的网格）作为`--boxes`参数来重新居中并按重叠度过滤。可同时传入多张图像（自动模式）。需要依赖可选的`pillow`库（自动模式还需`numpy`）。

html_shot — render an HTML file to an image (local, needs a Chrome-family browser)

html_shot — 将HTML文件渲染为图像（本地工具，需Chrome系列浏览器）

bash

python3 scripts/html_shot.py page.html                      # writes page.png, 1280x800
python3 scripts/html_shot.py page.html --width 1440 --height 900 -o page.png
python3 scripts/html_shot.py page.html --scale 2            # 2x pixels: small text stays readable

The visual-alignment loop: write HTML, screenshot it at the reference viewport, then compare it with the design. Use

pixel_diff

to locate material differences, not to chase a zero-difference score. Rendering happens in headless Chrome/Chromium/Edge — no Python dependencies. Only the viewport is captured, so pass

--height

for pages taller than the window;

--wait-ms N

pauses for fonts, images, or animation before capturing. Paths are relative to this skill's own directory.

bash

python3 scripts/html_shot.py page.html                      # 生成page.png，尺寸1280x800
python3 scripts/html_shot.py page.html --width 1440 --height 900 -o page.png
python3 scripts/html_shot.py page.html --scale 2            # 2倍像素：小文本保持清晰

视觉对齐流程：编写HTML，在参考视口生成截图，再与设计图对比。使用

pixel_diff

定位实质性差异，而非追求零差异分数。渲染在无头Chrome/Chromium/Edge中进行——无需Python依赖。仅捕获视口内容，因此对于比窗口高的页面需传入

--height

参数；

--wait-ms N

参数会在捕获前暂停指定时间，等待字体、图像或动画加载完成。路径相对于本工具集的目录。

pixel_diff — where two images differ (local, no vision API)

pixel_diff — 两张图像的差异位置（本地工具，无需视觉API）

bash

python3 scripts/pixel_diff.py <a> <b>      # path is relative to this skill dir

Prints an overall difference percentage plus the worst regions as

x1: ..

boxes you can feed straight into

glance --region

. Exact where a vision model rounds off.

bash

python3 scripts/pixel_diff.py <a> <b>      # 路径相对于本工具集目录

输出整体差异百分比，以及差异最明显区域的

x1: ..

坐标框，可直接传入

glance --region

。视觉模型会模糊处理的地方，该工具能给出精确结果。

dominant_colors — a region's palette, and the exact value among candidates (local, no vision API)

dominant_colors — 区域调色板，以及候选颜色中最匹配的精确值（本地工具，无需视觉API）

bash

python3 scripts/dominant_colors.py <image> --region X1,Y1,X2,Y2          # top colour clusters + shares
python3 scripts/dominant_colors.py <image> --region X1,Y1,X2,Y2 \
  --candidates '#F9FAFA,#F5F5F5,#F3F3F3,#EDEDED'                        # pick the best candidate

A vision model names a colour ("light gray") but not its value. The first mode downsamples, quantizes, and merges near-duplicates to list the region's significant colours with the share each owns — the histogram shows which colour is the background and which is the accent. Given the candidate palette your label implies, the second mode scores each candidate by how close the region's pixels are to it and prints the winner. Take the value from here, never from

glance

's prose. Paths are relative to this skill's own directory.

bash

python3 scripts/dominant_colors.py <image> --region X1,Y1,X2,Y2          # 输出主要颜色聚类及占比
python3 scripts/dominant_colors.py <image> --region X1,Y1,X2,Y2 \
  --candidates '#F9FAFA,#F5F5F5,#F3F3F3,#EDEDED'                        # 选择最匹配的候选颜色

视觉模型会命名颜色（比如“浅灰色”）但不会给出精确值。第一种模式会降采样、量化并合并近似颜色，列出区域的主要颜色及各自占比——直方图可显示背景色和强调色。给定标签对应的候选调色板，第二种模式会根据区域像素与候选颜色的接近度打分并输出最优结果。请从此工具获取颜色值，而非

glance

的文本描述。路径相对于本工具集的目录。

Work from a copy, not a temp path

基于副本操作，而非临时路径

If the image lives in a temp directory, before your first tool call on one, copy it somewhere durable and run everything against the copy — that is what keeps the image reachable later:

bash

cp "<the temp path>" work/shot.png
glance work/shot.png -q "..."

Exception: the user asked for the image to stay in a temp folder.

如果图像位于临时目录，在首次调用工具前，请将其复制到持久化目录，然后基于副本进行所有操作——这样后续才能继续访问该图像：

bash

cp "<临时路径>" work/shot.png
glance work/shot.png -q "..."

例外情况：用户要求图像保留在临时文件夹中。

When you have a description instead of the image

只有图像描述而无图像文件时

If an image reached you only as text — a description written by a person, a tool, or another model — and the image's file path is visible in the conversation, do not reason past a missing detail. Look again yourself:

glance <path> -q "<the specific detail>"

— one qualitative follow-up.

```
ground <path> "<target>"
```
then
```
glance <path> --region <that box> -q "..."
```
— locate, then zoom. The reliable way to inspect one element closely.

If the file no longer exists, say so instead of guessing.

如果仅收到图像的文本描述——由人、工具或其他模型编写——且对话中可见图像的文件路径，请不要对缺失细节进行推理。请自行重新查看：

```
glance <path> -q "<具体细节问题>"
```
——一次定性跟进查询。
```
ground <path> "<目标>"
```
然后
```
glance <path> --region <返回的框> -q "..."
```
——先定位，再放大查看。这是近距离检查单个元素的可靠方法。

如果文件已不存在，请如实告知，不要猜测。

Coarse to fine — the method behind every task above

从粗到细——所有任务背后的方法

For a single question about an image,

glance

is the whole answer. For anything multi-step, work outside-in:

One full-image pass (
```
glance
```
, or a description you already have) for the layout and an inventory of what is where.
For any element that matters,
```
ground
```
it, then zoom with
```
glance --region <box> -q "..."
```
. Full-image passes routinely miss small text and icons; a crop puts all the pixels on one detail, so the model sees it at effectively higher resolution. When the same box will be checked more than once, cut it to a file first with
```
crop
```
.
Never take a prose answer for a pixel-level fact — exact colors, small offsets, sizes. Vision models confidently report styling that is not there: coloured syntax highlighting in a monochrome code block, a border that does not exist. Get the number from
```
trace
```
, from a
```
ground
```
box, or from
```
pixel_diff
```
; sample the pixels yourself only for what those cannot return.

对于单个图像问题，

glance

即可给出完整答案。对于多步骤任务，请从外到内操作：

先进行一次全图识别（使用
```
glance
```
或已有描述），了解布局和元素分布。
对所有重要元素，先用
```
ground
```
定位，再用
```
glance --region <框> -q "..."
```
放大查看。全图识别通常会遗漏小文本和图标；裁剪区域会将所有像素集中在单个细节上，因此模型能以更高的有效分辨率查看。如果同一个框需要多次检查，请先用
```
crop
```
保存为文件。
永远不要将文本描述当作像素级事实——比如精确颜色、微小偏移量、尺寸。视觉模型会自信地描述不存在的样式：比如单色代码块中的彩色语法高亮、不存在的边框。请从
```
trace
```
、
```
ground
```
的框或
```
pixel_diff
```
获取精确数值；只有当这些工具都无法返回时，才自行采样像素。

Use cases

使用场景

Each file below is one job, start to finish: when it applies, the call sequence, and how to tell you got it right.

The job	Read
OCR a long screenshot, scrolling page, or chat history without losing text at chunk boundaries	`references/long-screenshot-ocr.md`
Rebuild a page or component as HTML/CSS, including a roughly three-minute fast approximation mode, or align an existing UI with its reference image	`references/restore-ui.md`
Extract or rebuild an icon, logo, illustration, or other isolated graphic as transparent PNG/SVG	`references/restore-graphic.md`
Turn a sketch, diagram, or whiteboard into Mermaid, Graphviz, or another structured representation	`references/restore-structure.md`
Operate a GUI from screenshots — locate, act, verify each step	`references/gui.md`

以下每个文件对应一个完整任务，包括适用场景、调用序列和验证方法：

任务	参考文档
OCR识别长截图、滚动页面或聊天记录，避免片段边界丢失文本	`references/long-screenshot-ocr.md`
将页面或组件重构为HTML/CSS，包括约三分钟的快速近似模式，或对齐现有UI与参考图像	`references/restore-ui.md`
提取或重构图标、Logo、插图或其他独立图形为透明PNG/SVG	`references/restore-graphic.md`
将草图、图表或白板内容转换为Mermaid、Graphviz或其他结构化格式	`references/restore-structure.md`
通过截图操作GUI——定位、操作、验证每一步	`references/gui.md`

Notes

注意事项

Only PNG / JPEG / GIF / WebP images are supported.
If a command is not found, the optional tools were not installed — report this to the user instead of improvising a replacement.
If the vision API fails, relay the error faithfully; never fabricate image content.

Source repository: https://github.com/Anionex/agent-vision-toolkit

Installation guide: https://github.com/Anionex/agent-vision-toolkit/blob/main/AGENT_INSTALL.md

仅支持PNG / JPEG / GIF / WebP格式的图像。
如果命令未找到，说明可选工具未安装——请告知用户，不要自行替换实现。
如果视觉API调用失败，请如实转发错误信息；切勿编造图像内容。

源码仓库：https://github.com/Anionex/agent-vision-toolkit

安装指南：https://github.com/Anionex/agent-vision-toolkit/blob/main/AGENT_INSTALL.md

vision-tools

Original

Translation

vision-tools

视觉工具集

Use the provided tools before hand-rolled pixels

优先使用内置工具而非自定义像素代码

glance — ask about an image

glance — 图像问答

ground — locate a named target

ground — 定位指定目标

detect — find every instance of a kind

detect — 查找某类对象的所有实例

trace — exact shape geometry (local, no vision API)

trace — 精确形状几何信息（本地工具，无需视觉API）

crop — cut a pixel box out of an image (local, no vision API)

crop — 从图像中裁剪像素框（本地工具，无需视觉API）

extract_fg — icon foreground as transparent PNG: manual region or auto (local, no vision API)

extract_fg — 将图标前景提取为透明PNG：手动指定区域或自动处理（本地工具，无需视觉API）

manual: you know the region (and optionally the background colour)

手动模式：已知目标区域（可选指定背景色）

auto:
`crop --scale`
cut-outs with the icon centred — no region needed

自动模式：使用
`crop --scale`
裁剪的居中图标——无需指定区域

html_shot — render an HTML file to an image (local, needs a Chrome-family browser)

html_shot — 将HTML文件渲染为图像（本地工具，需Chrome系列浏览器）

pixel_diff — where two images differ (local, no vision API)

pixel_diff — 两张图像的差异位置（本地工具，无需视觉API）

dominant_colors — a region's palette, and the exact value among candidates (local, no vision API)

dominant_colors — 区域调色板，以及候选颜色中最匹配的精确值（本地工具，无需视觉API）

Work from a copy, not a temp path

基于副本操作，而非临时路径

When you have a description instead of the image

只有图像描述而无图像文件时

Coarse to fine — the method behind every task above

从粗到细——所有任务背后的方法

Use cases

使用场景

Notes

注意事项

vision-tools

Original

Translation

vision-tools

视觉工具集

Use the provided tools before hand-rolled pixels

优先使用内置工具而非自定义像素代码

glance — ask about an image

glance — 图像问答

ground — locate a named target

ground — 定位指定目标

detect — find every instance of a kind

detect — 查找某类对象的所有实例

trace — exact shape geometry (local, no vision API)

trace — 精确形状几何信息（本地工具，无需视觉API）

crop — cut a pixel box out of an image (local, no vision API)

crop — 从图像中裁剪像素框（本地工具，无需视觉API）

extract_fg — icon foreground as transparent PNG: manual region or auto (local, no vision API)

extract_fg — 将图标前景提取为透明PNG：手动指定区域或自动处理（本地工具，无需视觉API）

manual: you know the region (and optionally the background colour)

手动模式：已知目标区域（可选指定背景色）

auto: crop --scale cut-outs with the icon centred — no region needed

自动模式：使用crop --scale裁剪的居中图标——无需指定区域

html_shot — render an HTML file to an image (local, needs a Chrome-family browser)

html_shot — 将HTML文件渲染为图像（本地工具，需Chrome系列浏览器）

pixel_diff — where two images differ (local, no vision API)

pixel_diff — 两张图像的差异位置（本地工具，无需视觉API）

dominant_colors — a region's palette, and the exact value among candidates (local, no vision API)

dominant_colors — 区域调色板，以及候选颜色中最匹配的精确值（本地工具，无需视觉API）

Work from a copy, not a temp path

基于副本操作，而非临时路径

When you have a description instead of the image

只有图像描述而无图像文件时

Coarse to fine — the method behind every task above

从粗到细——所有任务背后的方法

Use cases

使用场景

Notes

注意事项

auto:
`crop --scale`
cut-outs with the icon centred — no region needed

自动模式：使用
`crop --scale`
裁剪的居中图标——无需指定区域