AI Image Generation Workflow

The Five High-Level APIs

1. generateImage()

   — Create new images from text prompts. Supports

   referenceImageUrl

   for character consistency.

2. editImage()

   — Transform existing images via img2img, inpainting, or outpainting.

3. upscaleImage()

   — Increase resolution (2x or 4x super-resolution).

4. variateImage()

   — Generate visual variations of an existing image.

5. transferStyle()

   — Apply the visual aesthetic of a reference image to a source image via Flux Redux.

generate_image

extensions_enable image-generation

Provider Selection Guide

OPENAI_API_KEY

STABILITY_API_KEY

BFL_API_KEY

FAL_API_KEY

REPLICATE_API_TOKEN

OPENROUTER_API_KEY

STABLE_DIFFUSION_LOCAL_BASE_URL

Operation Decision Tree

• "Generate / create / draw / imagine" ->

  generateImage()

• "Edit / change / modify / transform" ->

  editImage()

  with

  mode: 'img2img'

• "Remove / fill in / fix this area" ->

  editImage()

  with

  mode: 'inpaint'

  + mask
• "Extend / expand the borders" ->

  editImage()

  with

  mode: 'outpaint'

• "Make it higher resolution / sharper" ->

  upscaleImage()

  with

  scale: 2

  or

  4

• "Show me variations / alternatives" ->

  variateImage()

  with

  n: 3-4

• "Make it look like this style" ->

  transferStyle()

  with source image + style reference
• "Same character but different expression/pose" ->

  generateImage()

  with

  referenceImageUrl

  +

  consistencyMode: 'strict'

• "Generate a character sheet / expression sheet" -> Use the

  AvatarPipeline

  which handles multi-stage consistency automatically

Character Consistency

referenceImageUrl

consistencyMode

• 'strict'

  — Face must match exactly. Best for expression sheets. Auto-selects Pulid on Replicate.

• 'balanced'

  — Recognizable but allows natural variation. Good for full-body shots and different angles.

• 'loose'

  — Light influence from the reference. Good for "inspired by" mood pieces.

Prompt Engineering Tips

1. Subject — What is in the image. Be specific: "a red panda sitting on a mossy branch" not "an animal."
2. Style — Artistic approach: photorealistic, watercolor, pixel art, oil painting, vector illustration, cinematic, anime.
3. Composition — Camera angle and framing: close-up portrait, wide establishing shot, overhead flat lay, isometric.
4. Lighting and Color — Mood through light: golden hour, dramatic side-lighting, neon glow, muted earth tones, high contrast.
5. Atmosphere — Emotional tone: serene, ominous, whimsical, nostalgic, futuristic.

• Front-load the most important elements. Models weight earlier tokens more heavily.
• Use negative prompts (Stability, Local SD) to exclude unwanted elements: "no text, no watermark, no blurry."
• For text-in-image, OpenAI GPT-Image-1 is the most reliable. Other models struggle with legible text.
• Request

  quality: 'hd'

  for DALL-E 3 when detail matters (doubles cost).
• For consistent characters across multiple images, describe the character in detail each time or use img2img with a reference.

Sizes and Aspect Ratios

1:1

16:9

9:16

4:3

3:4

aspectRatio

width

height

Examples

• "Generate a photorealistic image of a cozy cabin in the mountains at sunset."
• "Create a professional logo for a coffee shop called 'Bean There' — vector illustration style, clean lines."
• "Edit this photo: make the sky more dramatic with storm clouds." (img2img)
• "Remove the person from the background of this product photo." (inpaint + mask)
• "Upscale this thumbnail to 4x resolution for print."
• "Show me 3 variations of this hero image with different color palettes."
• "Generate a 16:9 cinematic landscape of a neon-lit Tokyo street at night in the rain."

Provider Preferences

providerPreferences

providerPreferences.image

agent.config.json

preferred

string[]

['stability', 'openai']

weights

Record<string, number>

{ stability: 0.7, openai: 0.3 }

blocked

string[]

['replicate']

generateImage({
  prompt: 'A neon-lit Tokyo alley in the rain',
  providerPreferences: {
    preferred: ['stability', 'openai'],
    blocked: ['replicate'],
  },
});

agent.config.json

{
  "providerPreferences": {
    "image": {
      "preferred": ["stability", "bfl"],
      "weights": { "stability": 0.6, "bfl": 0.4 },
      "blocked": ["replicate"]
    }
  }
}

providerPreferences.image

Constraints

• Image generation costs API credits per request; inform the user of approximate costs when possible.
• Content policy restrictions apply per provider: no realistic faces of real people, no violent/explicit content.
• DALL-E 3 does not support native inpainting — use GPT-Image-1 or Stability for mask-based editing.
• Upscaling is not supported by OpenAI or OpenRouter — use Stability, Replicate, or Local SD.
• Generated images may not perfectly match the prompt; iterative refinement is expected.
• Maximum prompt length varies by model (DALL-E 3: 4,000 chars; Stability: 2,000 chars).
• Local SD requires a running A1111 or ComfyUI instance with the API enabled.
• The fallback chain only activates when the primary provider fails; it does not merge results from multiple providers.