see-through-anime-layer-decomposition

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

See-through: Anime Character Layer Decomposition

See-through:动漫角色图层分解

Skill by ara.so — Daily 2026 Skills collection.
See-through is a research framework (SIGGRAPH 2026, conditionally accepted) that decomposes a single anime illustration into up to 23 fully inpainted, semantically distinct layers with inferred drawing orders — exporting a layered PSD file suitable for 2.5D animation workflows.
技能由 ara.so 提供 — 2026年度技能合集。
See-through是一个研究框架(SIGGRAPH 2026,条件性收录),可将单张动漫插画分解为最多23个完全补全、语义独立的图层并推断绘制顺序,导出适合2.5D动画工作流的分层PSD文件。

What It Does

功能介绍

  • Decomposes a single anime image into semantic layers (hair, face, eyes, clothing, accessories, etc.)
  • Inpaints occluded regions so each layer is complete
  • Infers pseudo-depth ordering using a fine-tuned Marigold model
  • Exports layered
    .psd
    files with depth maps and segmentation masks
  • Supports depth-based and left-right stratification for further refinement
  • 将单张动漫图像分解为语义图层(头发、脸部、眼睛、服装、配饰等)
  • 补全被遮挡区域,确保每个图层完整可用
  • 使用微调的Marigold模型推断伪深度顺序
  • 导出包含深度图和分割掩码的分层
    .psd
    文件
  • 支持基于深度和左右维度的分层,方便后续优化

Installation

安装

bash
undefined
bash
undefined

1. Create and activate environment

1. 创建并激活环境

conda create -n see_through python=3.12 -y conda activate see_through
conda create -n see_through python=3.12 -y conda activate see_through

2. Install PyTorch with CUDA 12.8

2. 安装适配CUDA 12.8的PyTorch

pip install torch==2.8.0+cu128 torchvision==0.23.0+cu128 torchaudio==2.8.0+cu128
--index-url https://download.pytorch.org/whl/cu128
pip install torch==2.8.0+cu128 torchvision==0.23.0+cu128 torchaudio==2.8.0+cu128
--index-url https://download.pytorch.org/whl/cu128

3. Install core dependencies

3. 安装核心依赖

pip install -r requirements.txt
pip install -r requirements.txt

4. Create assets symlink

4. 创建资源软链接

ln -sf common/assets assets
undefined
ln -sf common/assets assets
undefined

Optional Annotator Tiers

可选标注工具包

Install only what you need:
bash
undefined
按需安装所需组件:
bash
undefined

Body parsing (detectron2 — for body attribute tagging)

人体解析(detectron2 — 用于身体属性标注)

pip install --no-build-isolation -r requirements-inference-annotators.txt
pip install --no-build-isolation -r requirements-inference-annotators.txt

SAM2 (language-guided segmentation)

SAM2(语言引导的分割)

pip install --no-build-isolation -r requirements-inference-sam2.txt
pip install --no-build-isolation -r requirements-inference-sam2.txt

Instance segmentation (mmcv/mmdet — recommended for UI)

实例分割(mmcv/mmdet — UI运行推荐安装)

pip install -r requirements-inference-mmdet.txt

> Always run all scripts from the **repository root** as the working directory.
pip install -r requirements-inference-mmdet.txt

> 所有脚本都必须以**仓库根目录**作为工作目录运行。

Models

模型

Models are hosted on HuggingFace and downloaded automatically on first use:
ModelHuggingFace IDPurpose
LayerDiff 3D
layerdifforg/seethroughv0.0.2_layerdiff3d
SDXL-based transparent layer generation
Marigold Depth
24yearsold/seethroughv0.0.1_marigold
Anime pseudo-depth estimation
SAM Body Parsing
24yearsold/l2d_sam_iter2
19-part semantic body segmentation
模型托管在HuggingFace上,首次使用时会自动下载:
模型HuggingFace ID用途
LayerDiff 3D
layerdifforg/seethroughv0.0.2_layerdiff3d
基于SDXL的透明图层生成
Marigold Depth
24yearsold/seethroughv0.0.1_marigold
动漫伪深度估计
SAM Body Parsing
24yearsold/l2d_sam_iter2
19个部位的语义人体分割

Key CLI Commands

核心CLI命令

Main Pipeline: Layer Decomposition to PSD

主流程:图层分解导出PSD

bash
undefined
bash
undefined

Single image → layered PSD

单张图像 → 分层PSD

python inference/scripts/inference_psd.py
--srcp assets/test_image.png
--save_to_psd
python inference/scripts/inference_psd.py
--srcp assets/test_image.png
--save_to_psd

Entire directory of images

批量处理整个目录的图像

python inference/scripts/inference_psd.py
--srcp path/to/image_folder/
--save_to_psd

Output is saved to `workspace/layerdiff_output/` by default. Each run produces:
- A layered `.psd` file with semantically separated layers
- Intermediate depth maps
- Segmentation masks
python inference/scripts/inference_psd.py
--srcp path/to/image_folder/
--save_to_psd

输出默认保存在`workspace/layerdiff_output/`目录下,每次运行会生成:
- 语义分层的`.psd`文件
- 中间深度图
- 分割掩码

Heuristic Post-Processing

启发式后处理

After the main pipeline, further split layers using
heuristic_partseg.py
:
bash
undefined
主流程运行完成后,可以使用
heuristic_partseg.py
进一步拆分图层:
bash
undefined

Depth-based stratification (e.g., separate near/far handwear)

基于深度分层(例如拆分近处/远处的手部配饰)

python inference/scripts/heuristic_partseg.py seg_wdepth
--srcp workspace/test_samples_output/PV_0047_A0020.psd
--target_tags handwear
python inference/scripts/heuristic_partseg.py seg_wdepth
--srcp workspace/test_samples_output/PV_0047_A0020.psd
--target_tags handwear

Left-right stratification

基于左右分层

python inference/scripts/heuristic_partseg.py seg_wlr
--srcp workspace/test_samples_output/PV_0047_A0020_wdepth.psd
--target_tags handwear-1
undefined
python inference/scripts/heuristic_partseg.py seg_wlr
--srcp workspace/test_samples_output/PV_0047_A0020_wdepth.psd
--target_tags handwear-1
undefined

Synthetic Training Data Generation

合成训练数据生成

bash
python inference/scripts/syn_data.py
bash
python inference/scripts/syn_data.py

Python API Usage

Python API使用

Running the Full Pipeline Programmatically

编程式运行完整流程

python
import subprocess
import os

def decompose_anime_image(image_path: str, output_dir: str = "workspace/layerdiff_output") -> str:
    """
    Run See-through layer decomposition on a single anime image.
    Returns path to the output PSD file.
    """
    result = subprocess.run(
        [
            "python", "inference/scripts/inference_psd.py",
            "--srcp", image_path,
            "--save_to_psd",
        ],
        capture_output=True,
        text=True,
        cwd=os.getcwd()  # Must run from repo root
    )
    if result.returncode != 0:
        raise RuntimeError(f"Decomposition failed:\n{result.stderr}")
    
    # Derive expected output filename
    base_name = os.path.splitext(os.path.basename(image_path))[0]
    psd_path = os.path.join(output_dir, f"{base_name}.psd")
    return psd_path
python
import subprocess
import os

def decompose_anime_image(image_path: str, output_dir: str = "workspace/layerdiff_output") -> str:
    """
    对单张动漫图像运行See-through图层分解
    返回输出PSD文件的路径
    """
    result = subprocess.run(
        [
            "python", "inference/scripts/inference_psd.py",
            "--srcp", image_path,
            "--save_to_psd",
        ],
        capture_output=True,
        text=True,
        cwd=os.getcwd()  # 必须在仓库根目录运行
    )
    if result.returncode != 0:
        raise RuntimeError(f"分解失败:\n{result.stderr}")
    
    # 推导预期的输出文件名
    base_name = os.path.splitext(os.path.basename(image_path))[0]
    psd_path = os.path.join(output_dir, f"{base_name}.psd")
    return psd_path

Example usage

示例用法

psd_output = decompose_anime_image("assets/test_image.png") print(f"PSD saved to: {psd_output}")
undefined
psd_output = decompose_anime_image("assets/test_image.png") print(f"PSD文件保存至:{psd_output}")
undefined

Batch Processing a Directory

目录批量处理

python
import subprocess
from pathlib import Path

def batch_decompose(input_dir: str, output_dir: str = "workspace/layerdiff_output"):
    """Process all images in a directory."""
    result = subprocess.run(
        [
            "python", "inference/scripts/inference_psd.py",
            "--srcp", input_dir,
            "--save_to_psd",
        ],
        capture_output=True,
        text=True,
    )
    if result.returncode != 0:
        raise RuntimeError(f"Batch processing failed:\n{result.stderr}")
    
    output_psds = list(Path(output_dir).glob("*.psd"))
    print(f"Generated {len(output_psds)} PSD files in {output_dir}")
    return output_psds
python
import subprocess
from pathlib import Path

def batch_decompose(input_dir: str, output_dir: str = "workspace/layerdiff_output"):
    """处理目录下的所有图像"""
    result = subprocess.run(
        [
            "python", "inference/scripts/inference_psd.py",
            "--srcp", input_dir,
            "--save_to_psd",
        ],
        capture_output=True,
        text=True,
    )
    if result.returncode != 0:
        raise RuntimeError(f"批量处理失败:\n{result.stderr}")
    
    output_psds = list(Path(output_dir).glob("*.psd"))
    print(f"在{output_dir}中生成了{len(output_psds)}个PSD文件")
    return output_psds

Example

示例

psds = batch_decompose("path/to/my_anime_images/")
undefined
psds = batch_decompose("path/to/my_anime_images/")
undefined

Post-Processing: Depth and LR Splits

后处理:深度和左右拆分

python
import subprocess

def split_by_depth(psd_path: str, target_tags: list[str]) -> str:
    """Apply depth-based layer stratification to a PSD."""
    tags_str = " ".join(target_tags)
    result = subprocess.run(
        [
            "python", "inference/scripts/heuristic_partseg.py",
            "seg_wdepth",
            "--srcp", psd_path,
            "--target_tags", *target_tags,
        ],
        capture_output=True, text=True,
    )
    if result.returncode != 0:
        raise RuntimeError(result.stderr)
    # Output naming convention: original name + _wdepth suffix
    base = psd_path.replace(".psd", "_wdepth.psd")
    return base


def split_by_lr(psd_path: str, target_tags: list[str]) -> str:
    """Apply left-right layer stratification to a PSD."""
    result = subprocess.run(
        [
            "python", "inference/scripts/heuristic_partseg.py",
            "seg_wlr",
            "--srcp", psd_path,
            "--target_tags", *target_tags,
        ],
        capture_output=True, text=True,
    )
    if result.returncode != 0:
        raise RuntimeError(result.stderr)
    return psd_path.replace(".psd", "_wlr.psd")
python
import subprocess

def split_by_depth(psd_path: str, target_tags: list[str]) -> str:
    """对PSD应用基于深度的图层分层"""
    tags_str = " ".join(target_tags)
    result = subprocess.run(
        [
            "python", "inference/scripts/heuristic_partseg.py",
            "seg_wdepth",
            "--srcp", psd_path,
            "--target_tags", *target_tags,
        ],
        capture_output=True, text=True,
    )
    if result.returncode != 0:
        raise RuntimeError(result.stderr)
    # 输出命名规则:原名称 + _wdepth后缀
    base = psd_path.replace(".psd", "_wdepth.psd")
    return base


def split_by_lr(psd_path: str, target_tags: list[str]) -> str:
    """对PSD应用左右分层"""
    result = subprocess.run(
        [
            "python", "inference/scripts/heuristic_partseg.py",
            "seg_wlr",
            "--srcp", psd_path,
            "--target_tags", *target_tags,
        ],
        capture_output=True, text=True,
    )
    if result.returncode != 0:
        raise RuntimeError(result.stderr)
    return psd_path.replace(".psd", "_wlr.psd")

Full post-processing pipeline example

完整后处理流程示例

psd = "workspace/test_samples_output/PV_0047_A0020.psd" depth_psd = split_by_depth(psd, ["handwear"]) lr_psd = split_by_lr(depth_psd, ["handwear-1"]) print(f"Final PSD with depth+LR splits: {lr_psd}")
undefined
psd = "workspace/test_samples_output/PV_0047_A0020.psd" depth_psd = split_by_depth(psd, ["handwear"]) lr_psd = split_by_lr(depth_psd, ["handwear-1"]) print(f"完成深度+左右拆分的最终PSD:{lr_psd}")
undefined

Loading and Inspecting PSD Output

加载和检查PSD输出

python
from psd_tools import PSDImage  # pip install psd-tools

def inspect_psd_layers(psd_path: str):
    """List all layers in a See-through output PSD."""
    psd = PSDImage.open(psd_path)
    print(f"Canvas size: {psd.width}x{psd.height}")
    print(f"Total layers: {len(list(psd.descendants()))}")
    print("\nLayer structure:")
    for layer in psd:
        print(f"  [{layer.kind}] '{layer.name}' — "
              f"bbox: {layer.bbox}, visible: {layer.is_visible()}")
    return psd

psd = inspect_psd_layers("workspace/layerdiff_output/my_character.psd")
python
from psd_tools import PSDImage  # pip install psd-tools

def inspect_psd_layers(psd_path: str):
    """列出See-through输出PSD中的所有图层"""
    psd = PSDImage.open(psd_path)
    print(f"画布尺寸:{psd.width}x{psd.height}")
    print(f"总图层数:{len(list(psd.descendants()))}")
    print("\n图层结构:")
    for layer in psd:
        print(f"  [{layer.kind}] '{layer.name}' — "
              f"边界框:{layer.bbox}, 是否可见:{layer.is_visible()}")
    return psd

psd = inspect_psd_layers("workspace/layerdiff_output/my_character.psd")

Interactive Body Part Segmentation (Notebook)

交互式身体部位分割(Notebook)

Open and run the provided demo notebook:
bash
jupyter notebook inference/demo/bodypartseg_sam.ipynb
This demonstrates interactive 19-part body segmentation with visualization using the SAM body parsing model.
打开并运行提供的演示笔记本:
bash
jupyter notebook inference/demo/bodypartseg_sam.ipynb
该演示展示了使用SAM人体解析模型实现的19部位交互式人体分割及可视化功能。

Dataset Preparation for Training

训练数据集准备

See-through uses Live2D model files as training data. Setup requires a separate repo:
bash
undefined
See-through使用Live2D模型文件作为训练数据,设置需要单独的仓库:
bash
undefined

1. Clone the CubismPartExtr utility

1. 克隆CubismPartExtr工具

Follow its README to download sample model files and prepare workspace/

按照其README说明下载示例模型文件并准备workspace/目录

2. Run data parsing scripts per README_datapipeline.md

2. 按照README_datapipeline.md运行数据解析脚本

(scripts are in inference/scripts/ — check docstrings for details)

(脚本位于inference/scripts/目录下,详细信息查看文档字符串)

undefined
undefined

Launching the UI

启动UI

bash
undefined
bash
undefined

Requires workspace/datasets/ at repo root (contains sample data)

需要仓库根目录下存在workspace/datasets/目录(包含示例数据)

Recommended: install mmdet tier first

推荐:先安装mmdet工具包

pip install -r requirements-inference-mmdet.txt
pip install -r requirements-inference-mmdet.txt

Then follow ui/README.md for launch instructions

然后按照ui/README.md的说明启动

cd ui
cd ui

See ui/README.md for the specific launch command

具体启动命令查看ui/README.md

undefined
undefined

Directory Structure

目录结构

see-through/
├── inference/
│   ├── scripts/
│   │   ├── inference_psd.py        # Main pipeline
│   │   ├── heuristic_partseg.py    # Depth/LR post-processing
│   │   └── syn_data.py             # Synthetic data generation
│   └── demo/
│       └── bodypartseg_sam.ipynb   # Interactive segmentation demo
├── common/
│   ├── assets/                     # Test images, etc.
│   └── live2d/
│       └── scrap_model.py          # Full body tag definitions
├── ui/                             # User interface
├── workspace/                      # Runtime outputs (auto-created)
│   ├── layerdiff_output/           # Default PSD output location
│   ├── datasets/                   # Required for UI
│   └── test_samples_output/        # Sample outputs
├── requirements.txt
├── requirements-inference-annotators.txt
├── requirements-inference-sam2.txt
├── requirements-inference-mmdet.txt
└── README_datapipeline.md
see-through/
├── inference/
│   ├── scripts/
│   │   ├── inference_psd.py        # 主流程
│   │   ├── heuristic_partseg.py    # 深度/左右后处理
│   │   └── syn_data.py             # 合成数据生成
│   └── demo/
│       └── bodypartseg_sam.ipynb   # 交互式分割演示
├── common/
│   ├── assets/                     # 测试图像等资源
│   └── live2d/
│       └── scrap_model.py          # 全身标签定义
├── ui/                             # 用户界面
├── workspace/                      # 运行时输出(自动创建)
│   ├── layerdiff_output/           # PSD默认输出位置
│   ├── datasets/                   # UI运行必需
│   └── test_samples_output/        # 示例输出
├── requirements.txt
├── requirements-inference-annotators.txt
├── requirements-inference-sam2.txt
├── requirements-inference-mmdet.txt
└── README_datapipeline.md

Common Patterns

常用使用模式

Pattern: End-to-End Single Image Workflow

模式:端到端单图像工作流

bash
undefined
bash
undefined

Step 1: Decompose

步骤1:分解图像

python inference/scripts/inference_psd.py
--srcp assets/test_image.png
--save_to_psd
python inference/scripts/inference_psd.py
--srcp assets/test_image.png
--save_to_psd

Step 2: Depth-split a specific part tag

步骤2:对指定部位标签进行深度拆分

python inference/scripts/heuristic_partseg.py seg_wdepth
--srcp workspace/layerdiff_output/test_image.psd
--target_tags arm sleeve
python inference/scripts/heuristic_partseg.py seg_wdepth
--srcp workspace/layerdiff_output/test_image.psd
--target_tags arm sleeve

Step 3: Left-right split

步骤3:左右拆分

python inference/scripts/heuristic_partseg.py seg_wlr
--srcp workspace/layerdiff_output/test_image_wdepth.psd
--target_tags arm-1 sleeve-1
undefined
python inference/scripts/heuristic_partseg.py seg_wlr
--srcp workspace/layerdiff_output/test_image_wdepth.psd
--target_tags arm-1 sleeve-1
undefined

Pattern: Check Available Body Tags

模式:查看可用的身体标签

python
undefined
python
undefined

Body tag definitions are in common/live2d/scrap_model.py

身体标签定义位于common/live2d/scrap_model.py

import importlib.util, sys spec = importlib.util.spec_from_file_location( "scrap_model", "common/live2d/scrap_model.py" ) mod = importlib.util.module_from_spec(spec) spec.loader.exec_module(mod)
import importlib.util, sys spec = importlib.util.spec_from_file_location( "scrap_model", "common/live2d/scrap_model.py" ) mod = importlib.util.module_from_spec(spec) spec.loader.exec_module(mod)

Inspect the module for tag constants/enums

查看模块中的标签常量/枚举

print(dir(mod))
undefined
print(dir(mod))
undefined

ComfyUI Integration

ComfyUI集成

A community-maintained ComfyUI node is available:
https://github.com/jtydhr88/ComfyUI-See-through
Install via ComfyUI Manager or clone into
ComfyUI/custom_nodes/
.
社区维护的ComfyUI节点已发布:
https://github.com/jtydhr88/ComfyUI-See-through
通过ComfyUI管理器安装,或克隆到
ComfyUI/custom_nodes/
目录即可使用。

Troubleshooting

故障排查

ProblemSolution
ModuleNotFoundError
for detectron2/mmcv
Install the appropriate optional tier:
pip install --no-build-isolation -r requirements-inference-annotators.txt
Scripts fail with path errorsAlways run from the repository root, not from within subdirectories
UI fails to launchInstall mmdet tier:
pip install -r requirements-inference-mmdet.txt
; ensure
workspace/datasets/
exists
CUDA out of memoryUse a GPU with ≥16GB VRAM; SDXL-based LayerDiff 3D is memory-intensive
Assets not foundRe-run
ln -sf common/assets assets
from repo root
SAM2 install failsUse
--no-build-isolation
flag as shown in the install commands
Output PSD empty or malformedCheck
workspace/layerdiff_output/
for intermediate depth/mask files to diagnose which stage failed
问题解决方案
detectron2/mmcv 报
ModuleNotFoundError
安装对应的可选工具包:
pip install --no-build-isolation -r requirements-inference-annotators.txt
脚本运行报错路径不存在始终在仓库根目录运行,不要在子目录内运行
UI启动失败安装mmdet工具包:
pip install -r requirements-inference-mmdet.txt
;确保
workspace/datasets/
目录存在
CUDA显存不足使用显存≥16GB的GPU;基于SDXL的LayerDiff 3D对显存要求较高
找不到资源文件在仓库根目录重新运行
ln -sf common/assets assets
SAM2安装失败按照安装命令所示使用
--no-build-isolation
参数
输出PSD为空或格式错误检查
workspace/layerdiff_output/
中的中间深度/掩码文件,定位故障阶段

Citation

引用

bibtex
@article{lin2026seethrough,
  title={See-through: Single-image Layer Decomposition for Anime Characters},
  author={Lin, Jian and Li, Chengze and Qin, Haoyun and Chan, Kwun Wang and
          Jin, Yanghua and Liu, Hanyuan and Choy, Stephen Chun Wang and Liu, Xueting},
  journal={arXiv preprint arXiv:2602.03749},
  year={2026}
}
bibtex
@article{lin2026seethrough,
  title={See-through: Single-image Layer Decomposition for Anime Characters},
  author={Lin, Jian and Li, Chengze and Qin, Haoyun and Chan, Kwun Wang and
          Jin, Yanghua and Liu, Hanyuan and Choy, Stephen Chun Wang and Liu, Xueting},
  journal={arXiv preprint arXiv:2602.03749},
  year={2026}
}