see-through-anime-layer-decomposition
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseSee-through: Anime Character Layer Decomposition
See-through:动漫角色图层分解
Skill by ara.so — Daily 2026 Skills collection.
See-through is a research framework (SIGGRAPH 2026, conditionally accepted) that decomposes a single anime illustration into up to 23 fully inpainted, semantically distinct layers with inferred drawing orders — exporting a layered PSD file suitable for 2.5D animation workflows.
技能由 ara.so 提供 — 2026年度技能合集。
See-through是一个研究框架(SIGGRAPH 2026,条件性收录),可将单张动漫插画分解为最多23个完全补全、语义独立的图层并推断绘制顺序,导出适合2.5D动画工作流的分层PSD文件。
What It Does
功能介绍
- Decomposes a single anime image into semantic layers (hair, face, eyes, clothing, accessories, etc.)
- Inpaints occluded regions so each layer is complete
- Infers pseudo-depth ordering using a fine-tuned Marigold model
- Exports layered files with depth maps and segmentation masks
.psd - Supports depth-based and left-right stratification for further refinement
- 将单张动漫图像分解为语义图层(头发、脸部、眼睛、服装、配饰等)
- 补全被遮挡区域,确保每个图层完整可用
- 使用微调的Marigold模型推断伪深度顺序
- 导出包含深度图和分割掩码的分层文件
.psd - 支持基于深度和左右维度的分层,方便后续优化
Installation
安装
bash
undefinedbash
undefined1. Create and activate environment
1. 创建并激活环境
conda create -n see_through python=3.12 -y
conda activate see_through
conda create -n see_through python=3.12 -y
conda activate see_through
2. Install PyTorch with CUDA 12.8
2. 安装适配CUDA 12.8的PyTorch
pip install torch==2.8.0+cu128 torchvision==0.23.0+cu128 torchaudio==2.8.0+cu128
--index-url https://download.pytorch.org/whl/cu128
--index-url https://download.pytorch.org/whl/cu128
pip install torch==2.8.0+cu128 torchvision==0.23.0+cu128 torchaudio==2.8.0+cu128
--index-url https://download.pytorch.org/whl/cu128
--index-url https://download.pytorch.org/whl/cu128
3. Install core dependencies
3. 安装核心依赖
pip install -r requirements.txt
pip install -r requirements.txt
4. Create assets symlink
4. 创建资源软链接
ln -sf common/assets assets
undefinedln -sf common/assets assets
undefinedOptional Annotator Tiers
可选标注工具包
Install only what you need:
bash
undefined按需安装所需组件:
bash
undefinedBody parsing (detectron2 — for body attribute tagging)
人体解析(detectron2 — 用于身体属性标注)
pip install --no-build-isolation -r requirements-inference-annotators.txt
pip install --no-build-isolation -r requirements-inference-annotators.txt
SAM2 (language-guided segmentation)
SAM2(语言引导的分割)
pip install --no-build-isolation -r requirements-inference-sam2.txt
pip install --no-build-isolation -r requirements-inference-sam2.txt
Instance segmentation (mmcv/mmdet — recommended for UI)
实例分割(mmcv/mmdet — UI运行推荐安装)
pip install -r requirements-inference-mmdet.txt
> Always run all scripts from the **repository root** as the working directory.pip install -r requirements-inference-mmdet.txt
> 所有脚本都必须以**仓库根目录**作为工作目录运行。Models
模型
Models are hosted on HuggingFace and downloaded automatically on first use:
| Model | HuggingFace ID | Purpose |
|---|---|---|
| LayerDiff 3D | | SDXL-based transparent layer generation |
| Marigold Depth | | Anime pseudo-depth estimation |
| SAM Body Parsing | | 19-part semantic body segmentation |
模型托管在HuggingFace上,首次使用时会自动下载:
| 模型 | HuggingFace ID | 用途 |
|---|---|---|
| LayerDiff 3D | | 基于SDXL的透明图层生成 |
| Marigold Depth | | 动漫伪深度估计 |
| SAM Body Parsing | | 19个部位的语义人体分割 |
Key CLI Commands
核心CLI命令
Main Pipeline: Layer Decomposition to PSD
主流程:图层分解导出PSD
bash
undefinedbash
undefinedSingle image → layered PSD
单张图像 → 分层PSD
python inference/scripts/inference_psd.py
--srcp assets/test_image.png
--save_to_psd
--srcp assets/test_image.png
--save_to_psd
python inference/scripts/inference_psd.py
--srcp assets/test_image.png
--save_to_psd
--srcp assets/test_image.png
--save_to_psd
Entire directory of images
批量处理整个目录的图像
python inference/scripts/inference_psd.py
--srcp path/to/image_folder/
--save_to_psd
--srcp path/to/image_folder/
--save_to_psd
Output is saved to `workspace/layerdiff_output/` by default. Each run produces:
- A layered `.psd` file with semantically separated layers
- Intermediate depth maps
- Segmentation maskspython inference/scripts/inference_psd.py
--srcp path/to/image_folder/
--save_to_psd
--srcp path/to/image_folder/
--save_to_psd
输出默认保存在`workspace/layerdiff_output/`目录下,每次运行会生成:
- 语义分层的`.psd`文件
- 中间深度图
- 分割掩码Heuristic Post-Processing
启发式后处理
After the main pipeline, further split layers using :
heuristic_partseg.pybash
undefined主流程运行完成后,可以使用进一步拆分图层:
heuristic_partseg.pybash
undefinedDepth-based stratification (e.g., separate near/far handwear)
基于深度分层(例如拆分近处/远处的手部配饰)
python inference/scripts/heuristic_partseg.py seg_wdepth
--srcp workspace/test_samples_output/PV_0047_A0020.psd
--target_tags handwear
--srcp workspace/test_samples_output/PV_0047_A0020.psd
--target_tags handwear
python inference/scripts/heuristic_partseg.py seg_wdepth
--srcp workspace/test_samples_output/PV_0047_A0020.psd
--target_tags handwear
--srcp workspace/test_samples_output/PV_0047_A0020.psd
--target_tags handwear
Left-right stratification
基于左右分层
python inference/scripts/heuristic_partseg.py seg_wlr
--srcp workspace/test_samples_output/PV_0047_A0020_wdepth.psd
--target_tags handwear-1
--srcp workspace/test_samples_output/PV_0047_A0020_wdepth.psd
--target_tags handwear-1
undefinedpython inference/scripts/heuristic_partseg.py seg_wlr
--srcp workspace/test_samples_output/PV_0047_A0020_wdepth.psd
--target_tags handwear-1
--srcp workspace/test_samples_output/PV_0047_A0020_wdepth.psd
--target_tags handwear-1
undefinedSynthetic Training Data Generation
合成训练数据生成
bash
python inference/scripts/syn_data.pybash
python inference/scripts/syn_data.pyPython API Usage
Python API使用
Running the Full Pipeline Programmatically
编程式运行完整流程
python
import subprocess
import os
def decompose_anime_image(image_path: str, output_dir: str = "workspace/layerdiff_output") -> str:
"""
Run See-through layer decomposition on a single anime image.
Returns path to the output PSD file.
"""
result = subprocess.run(
[
"python", "inference/scripts/inference_psd.py",
"--srcp", image_path,
"--save_to_psd",
],
capture_output=True,
text=True,
cwd=os.getcwd() # Must run from repo root
)
if result.returncode != 0:
raise RuntimeError(f"Decomposition failed:\n{result.stderr}")
# Derive expected output filename
base_name = os.path.splitext(os.path.basename(image_path))[0]
psd_path = os.path.join(output_dir, f"{base_name}.psd")
return psd_pathpython
import subprocess
import os
def decompose_anime_image(image_path: str, output_dir: str = "workspace/layerdiff_output") -> str:
"""
对单张动漫图像运行See-through图层分解
返回输出PSD文件的路径
"""
result = subprocess.run(
[
"python", "inference/scripts/inference_psd.py",
"--srcp", image_path,
"--save_to_psd",
],
capture_output=True,
text=True,
cwd=os.getcwd() # 必须在仓库根目录运行
)
if result.returncode != 0:
raise RuntimeError(f"分解失败:\n{result.stderr}")
# 推导预期的输出文件名
base_name = os.path.splitext(os.path.basename(image_path))[0]
psd_path = os.path.join(output_dir, f"{base_name}.psd")
return psd_pathExample usage
示例用法
psd_output = decompose_anime_image("assets/test_image.png")
print(f"PSD saved to: {psd_output}")
undefinedpsd_output = decompose_anime_image("assets/test_image.png")
print(f"PSD文件保存至:{psd_output}")
undefinedBatch Processing a Directory
目录批量处理
python
import subprocess
from pathlib import Path
def batch_decompose(input_dir: str, output_dir: str = "workspace/layerdiff_output"):
"""Process all images in a directory."""
result = subprocess.run(
[
"python", "inference/scripts/inference_psd.py",
"--srcp", input_dir,
"--save_to_psd",
],
capture_output=True,
text=True,
)
if result.returncode != 0:
raise RuntimeError(f"Batch processing failed:\n{result.stderr}")
output_psds = list(Path(output_dir).glob("*.psd"))
print(f"Generated {len(output_psds)} PSD files in {output_dir}")
return output_psdspython
import subprocess
from pathlib import Path
def batch_decompose(input_dir: str, output_dir: str = "workspace/layerdiff_output"):
"""处理目录下的所有图像"""
result = subprocess.run(
[
"python", "inference/scripts/inference_psd.py",
"--srcp", input_dir,
"--save_to_psd",
],
capture_output=True,
text=True,
)
if result.returncode != 0:
raise RuntimeError(f"批量处理失败:\n{result.stderr}")
output_psds = list(Path(output_dir).glob("*.psd"))
print(f"在{output_dir}中生成了{len(output_psds)}个PSD文件")
return output_psdsExample
示例
psds = batch_decompose("path/to/my_anime_images/")
undefinedpsds = batch_decompose("path/to/my_anime_images/")
undefinedPost-Processing: Depth and LR Splits
后处理:深度和左右拆分
python
import subprocess
def split_by_depth(psd_path: str, target_tags: list[str]) -> str:
"""Apply depth-based layer stratification to a PSD."""
tags_str = " ".join(target_tags)
result = subprocess.run(
[
"python", "inference/scripts/heuristic_partseg.py",
"seg_wdepth",
"--srcp", psd_path,
"--target_tags", *target_tags,
],
capture_output=True, text=True,
)
if result.returncode != 0:
raise RuntimeError(result.stderr)
# Output naming convention: original name + _wdepth suffix
base = psd_path.replace(".psd", "_wdepth.psd")
return base
def split_by_lr(psd_path: str, target_tags: list[str]) -> str:
"""Apply left-right layer stratification to a PSD."""
result = subprocess.run(
[
"python", "inference/scripts/heuristic_partseg.py",
"seg_wlr",
"--srcp", psd_path,
"--target_tags", *target_tags,
],
capture_output=True, text=True,
)
if result.returncode != 0:
raise RuntimeError(result.stderr)
return psd_path.replace(".psd", "_wlr.psd")python
import subprocess
def split_by_depth(psd_path: str, target_tags: list[str]) -> str:
"""对PSD应用基于深度的图层分层"""
tags_str = " ".join(target_tags)
result = subprocess.run(
[
"python", "inference/scripts/heuristic_partseg.py",
"seg_wdepth",
"--srcp", psd_path,
"--target_tags", *target_tags,
],
capture_output=True, text=True,
)
if result.returncode != 0:
raise RuntimeError(result.stderr)
# 输出命名规则:原名称 + _wdepth后缀
base = psd_path.replace(".psd", "_wdepth.psd")
return base
def split_by_lr(psd_path: str, target_tags: list[str]) -> str:
"""对PSD应用左右分层"""
result = subprocess.run(
[
"python", "inference/scripts/heuristic_partseg.py",
"seg_wlr",
"--srcp", psd_path,
"--target_tags", *target_tags,
],
capture_output=True, text=True,
)
if result.returncode != 0:
raise RuntimeError(result.stderr)
return psd_path.replace(".psd", "_wlr.psd")Full post-processing pipeline example
完整后处理流程示例
psd = "workspace/test_samples_output/PV_0047_A0020.psd"
depth_psd = split_by_depth(psd, ["handwear"])
lr_psd = split_by_lr(depth_psd, ["handwear-1"])
print(f"Final PSD with depth+LR splits: {lr_psd}")
undefinedpsd = "workspace/test_samples_output/PV_0047_A0020.psd"
depth_psd = split_by_depth(psd, ["handwear"])
lr_psd = split_by_lr(depth_psd, ["handwear-1"])
print(f"完成深度+左右拆分的最终PSD:{lr_psd}")
undefinedLoading and Inspecting PSD Output
加载和检查PSD输出
python
from psd_tools import PSDImage # pip install psd-tools
def inspect_psd_layers(psd_path: str):
"""List all layers in a See-through output PSD."""
psd = PSDImage.open(psd_path)
print(f"Canvas size: {psd.width}x{psd.height}")
print(f"Total layers: {len(list(psd.descendants()))}")
print("\nLayer structure:")
for layer in psd:
print(f" [{layer.kind}] '{layer.name}' — "
f"bbox: {layer.bbox}, visible: {layer.is_visible()}")
return psd
psd = inspect_psd_layers("workspace/layerdiff_output/my_character.psd")python
from psd_tools import PSDImage # pip install psd-tools
def inspect_psd_layers(psd_path: str):
"""列出See-through输出PSD中的所有图层"""
psd = PSDImage.open(psd_path)
print(f"画布尺寸:{psd.width}x{psd.height}")
print(f"总图层数:{len(list(psd.descendants()))}")
print("\n图层结构:")
for layer in psd:
print(f" [{layer.kind}] '{layer.name}' — "
f"边界框:{layer.bbox}, 是否可见:{layer.is_visible()}")
return psd
psd = inspect_psd_layers("workspace/layerdiff_output/my_character.psd")Interactive Body Part Segmentation (Notebook)
交互式身体部位分割(Notebook)
Open and run the provided demo notebook:
bash
jupyter notebook inference/demo/bodypartseg_sam.ipynbThis demonstrates interactive 19-part body segmentation with visualization using the SAM body parsing model.
打开并运行提供的演示笔记本:
bash
jupyter notebook inference/demo/bodypartseg_sam.ipynb该演示展示了使用SAM人体解析模型实现的19部位交互式人体分割及可视化功能。
Dataset Preparation for Training
训练数据集准备
See-through uses Live2D model files as training data. Setup requires a separate repo:
bash
undefinedSee-through使用Live2D模型文件作为训练数据,设置需要单独的仓库:
bash
undefined1. Clone the CubismPartExtr utility
1. 克隆CubismPartExtr工具
Follow its README to download sample model files and prepare workspace/
按照其README说明下载示例模型文件并准备workspace/目录
2. Run data parsing scripts per README_datapipeline.md
2. 按照README_datapipeline.md运行数据解析脚本
(scripts are in inference/scripts/ — check docstrings for details)
(脚本位于inference/scripts/目录下,详细信息查看文档字符串)
undefinedundefinedLaunching the UI
启动UI
bash
undefinedbash
undefinedRequires workspace/datasets/ at repo root (contains sample data)
需要仓库根目录下存在workspace/datasets/目录(包含示例数据)
Recommended: install mmdet tier first
推荐:先安装mmdet工具包
pip install -r requirements-inference-mmdet.txt
pip install -r requirements-inference-mmdet.txt
Then follow ui/README.md for launch instructions
然后按照ui/README.md的说明启动
cd ui
cd ui
See ui/README.md for the specific launch command
具体启动命令查看ui/README.md
undefinedundefinedDirectory Structure
目录结构
see-through/
├── inference/
│ ├── scripts/
│ │ ├── inference_psd.py # Main pipeline
│ │ ├── heuristic_partseg.py # Depth/LR post-processing
│ │ └── syn_data.py # Synthetic data generation
│ └── demo/
│ └── bodypartseg_sam.ipynb # Interactive segmentation demo
├── common/
│ ├── assets/ # Test images, etc.
│ └── live2d/
│ └── scrap_model.py # Full body tag definitions
├── ui/ # User interface
├── workspace/ # Runtime outputs (auto-created)
│ ├── layerdiff_output/ # Default PSD output location
│ ├── datasets/ # Required for UI
│ └── test_samples_output/ # Sample outputs
├── requirements.txt
├── requirements-inference-annotators.txt
├── requirements-inference-sam2.txt
├── requirements-inference-mmdet.txt
└── README_datapipeline.mdsee-through/
├── inference/
│ ├── scripts/
│ │ ├── inference_psd.py # 主流程
│ │ ├── heuristic_partseg.py # 深度/左右后处理
│ │ └── syn_data.py # 合成数据生成
│ └── demo/
│ └── bodypartseg_sam.ipynb # 交互式分割演示
├── common/
│ ├── assets/ # 测试图像等资源
│ └── live2d/
│ └── scrap_model.py # 全身标签定义
├── ui/ # 用户界面
├── workspace/ # 运行时输出(自动创建)
│ ├── layerdiff_output/ # PSD默认输出位置
│ ├── datasets/ # UI运行必需
│ └── test_samples_output/ # 示例输出
├── requirements.txt
├── requirements-inference-annotators.txt
├── requirements-inference-sam2.txt
├── requirements-inference-mmdet.txt
└── README_datapipeline.mdCommon Patterns
常用使用模式
Pattern: End-to-End Single Image Workflow
模式:端到端单图像工作流
bash
undefinedbash
undefinedStep 1: Decompose
步骤1:分解图像
python inference/scripts/inference_psd.py
--srcp assets/test_image.png
--save_to_psd
--srcp assets/test_image.png
--save_to_psd
python inference/scripts/inference_psd.py
--srcp assets/test_image.png
--save_to_psd
--srcp assets/test_image.png
--save_to_psd
Step 2: Depth-split a specific part tag
步骤2:对指定部位标签进行深度拆分
python inference/scripts/heuristic_partseg.py seg_wdepth
--srcp workspace/layerdiff_output/test_image.psd
--target_tags arm sleeve
--srcp workspace/layerdiff_output/test_image.psd
--target_tags arm sleeve
python inference/scripts/heuristic_partseg.py seg_wdepth
--srcp workspace/layerdiff_output/test_image.psd
--target_tags arm sleeve
--srcp workspace/layerdiff_output/test_image.psd
--target_tags arm sleeve
Step 3: Left-right split
步骤3:左右拆分
python inference/scripts/heuristic_partseg.py seg_wlr
--srcp workspace/layerdiff_output/test_image_wdepth.psd
--target_tags arm-1 sleeve-1
--srcp workspace/layerdiff_output/test_image_wdepth.psd
--target_tags arm-1 sleeve-1
undefinedpython inference/scripts/heuristic_partseg.py seg_wlr
--srcp workspace/layerdiff_output/test_image_wdepth.psd
--target_tags arm-1 sleeve-1
--srcp workspace/layerdiff_output/test_image_wdepth.psd
--target_tags arm-1 sleeve-1
undefinedPattern: Check Available Body Tags
模式:查看可用的身体标签
python
undefinedpython
undefinedBody tag definitions are in common/live2d/scrap_model.py
身体标签定义位于common/live2d/scrap_model.py
import importlib.util, sys
spec = importlib.util.spec_from_file_location(
"scrap_model", "common/live2d/scrap_model.py"
)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
import importlib.util, sys
spec = importlib.util.spec_from_file_location(
"scrap_model", "common/live2d/scrap_model.py"
)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
Inspect the module for tag constants/enums
查看模块中的标签常量/枚举
print(dir(mod))
undefinedprint(dir(mod))
undefinedComfyUI Integration
ComfyUI集成
A community-maintained ComfyUI node is available:
https://github.com/jtydhr88/ComfyUI-See-throughInstall via ComfyUI Manager or clone into .
ComfyUI/custom_nodes/社区维护的ComfyUI节点已发布:
https://github.com/jtydhr88/ComfyUI-See-through通过ComfyUI管理器安装,或克隆到目录即可使用。
ComfyUI/custom_nodes/Troubleshooting
故障排查
| Problem | Solution |
|---|---|
| Install the appropriate optional tier: |
| Scripts fail with path errors | Always run from the repository root, not from within subdirectories |
| UI fails to launch | Install mmdet tier: |
| CUDA out of memory | Use a GPU with ≥16GB VRAM; SDXL-based LayerDiff 3D is memory-intensive |
| Assets not found | Re-run |
| SAM2 install fails | Use |
| Output PSD empty or malformed | Check |
| 问题 | 解决方案 |
|---|---|
detectron2/mmcv 报 | 安装对应的可选工具包: |
| 脚本运行报错路径不存在 | 始终在仓库根目录运行,不要在子目录内运行 |
| UI启动失败 | 安装mmdet工具包: |
| CUDA显存不足 | 使用显存≥16GB的GPU;基于SDXL的LayerDiff 3D对显存要求较高 |
| 找不到资源文件 | 在仓库根目录重新运行 |
| SAM2安装失败 | 按照安装命令所示使用 |
| 输出PSD为空或格式错误 | 检查 |
Citation
引用
bibtex
@article{lin2026seethrough,
title={See-through: Single-image Layer Decomposition for Anime Characters},
author={Lin, Jian and Li, Chengze and Qin, Haoyun and Chan, Kwun Wang and
Jin, Yanghua and Liu, Hanyuan and Choy, Stephen Chun Wang and Liu, Xueting},
journal={arXiv preprint arXiv:2602.03749},
year={2026}
}bibtex
@article{lin2026seethrough,
title={See-through: Single-image Layer Decomposition for Anime Characters},
author={Lin, Jian and Li, Chengze and Qin, Haoyun and Chan, Kwun Wang and
Jin, Yanghua and Liu, Hanyuan and Choy, Stephen Chun Wang and Liu, Xueting},
journal={arXiv preprint arXiv:2602.03749},
year={2026}
}