ComfyUI Workflow Builder

Workflow Generation Process

Step 1: Understand the Request

• Output type: Image, video, or audio
• Source material: Text-only, reference image(s), existing video
• Identity method: None, zero-shot (InstantID/PuLID), LoRA, Kontext
• Quality level: Draft (fast iteration) vs production (maximum quality)
• Special requirements: ControlNet, inpainting, upscaling, lip-sync

Step 2: Check Inventory

state/inventory.json

• Available checkpoints → select best match for task
• Available identity models → determine which methods are possible
• Available ControlNet models → enable pose/depth control if available
• Custom nodes installed → verify all required nodes exist
• VRAM available → optimize settings accordingly

Step 3: Select Pipeline Pattern

Step 4: Generate Workflow JSON

{
  "{node_id}": {
    "class_type": "{NodeClassName}",
    "inputs": {
      "{param_name}": "{value}",
      "{connected_param}": ["{source_node_id}", {output_index}]
    }
  }
}

• Node IDs are strings (typically "1", "2", "3"...)
• Connected inputs use array format:

  ["source_node_id", output_index]

• Output index is 0-based integer
• Filenames must match exactly what's in inventory
• Seed values: use random large integer or fixed for reproducibility

Step 5: Validate

1. Every

   class_type

   exists in inventory's node list
2. Every model filename exists in inventory's model list
3. All required connections are present (no dangling inputs)
4. VRAM estimate doesn't exceed available VRAM
5. Resolution is compatible with chosen model (512 for SD1.5, 1024 for SDXL/FLUX)

Step 6: Output

comfyui-api

projects/{project}/workflows/

Workflow Templates

Basic Text-to-Image (FLUX)

{
  "1": {
    "class_type": "LoadCheckpoint",
    "inputs": {"ckpt_name": "flux1-dev.safetensors"}
  },
  "2": {
    "class_type": "CLIPTextEncode",
    "inputs": {"text": "{positive_prompt}", "clip": ["1", 1]}
  },
  "3": {
    "class_type": "CLIPTextEncode",
    "inputs": {"text": "{negative_prompt}", "clip": ["1", 1]}
  },
  "4": {
    "class_type": "EmptyLatentImage",
    "inputs": {"width": 1024, "height": 1024, "batch_size": 1}
  },
  "5": {
    "class_type": "KSampler",
    "inputs": {
      "seed": 42,
      "steps": 25,
      "cfg": 3.5,
      "sampler_name": "euler",
      "scheduler": "normal",
      "denoise": 1.0,
      "model": ["1", 0],
      "positive": ["2", 0],
      "negative": ["3", 0],
      "latent_image": ["4", 0]
    }
  },
  "6": {
    "class_type": "VAEDecode",
    "inputs": {"samples": ["5", 0], "vae": ["1", 2]}
  },
  "7": {
    "class_type": "SaveImage",
    "inputs": {"filename_prefix": "output", "images": ["6", 0]}
  }
}

With Identity Preservation (InstantID + IP-Adapter)

• Load reference image node
• InstantID Model Loader + Apply InstantID
• IPAdapter Unified Loader + Apply IPAdapter
• FaceDetailer post-processing

references/workflows.md

Video Generation (Wan I2V)

• Load Diffusion Model (not LoadCheckpoint)
• Wan I2V Conditioning
• EmptySD3LatentImage (with frame count)
• Video Combine (VHS)

references/workflows.md

VRAM Estimation

Common Mistakes to Avoid

1. Wrong output index: CheckpointLoader outputs

   [model, clip, vae]

   at indices

   [0, 1, 2]

2. CFG too high for InstantID: Use 4-5, not default 7-8
3. Wrong resolution for model: FLUX/SDXL=1024, SD1.5=512
4. Missing VAE: FLUX needs explicit VAE (

   ae.safetensors

   )
5. Wrong model in wrong loader: Diffusion models need

   LoadDiffusionModel

   , not

   LoadCheckpoint

Reference Files

• references/workflows.md

  - Detailed node-by-node templates

• references/models.md

  - Model files and paths

• references/prompt-templates.md

  - Model-specific prompts

• state/inventory.json

  - Current inventory cache