Back to Details

aigw-orchestrator

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese
Orchestrate a full Envoy AI Gateway deployment by asking intake questions and composing the appropriate atomic skills. Use this when the user wants to set up AI Gateway from scratch or add a new provider.
通过询问需求问题并组合对应的原子技能，编排完整的Envoy AI Gateway部署流程。当用户希望从零开始搭建AI Gateway或添加新的服务提供商时，可使用本流程。

Intake Questions

需求收集问题

Before generating configuration, ask:
  1. Installation
    • Do you already have Envoy Gateway installed? If not, we need 
      /aigw-install
      .
    • Do you need rate limiting or InferencePool? (addons)
  2. Provider
    • Which AI provider(s)? (OpenAI, Anthropic, AWS Bedrock, Azure OpenAI, GCP Vertex AI, Cohere, self-hosted/Ollama, etc.)
    • For cloud providers: How will you authenticate? (API key, IRSA/Pod Identity, service account, etc.)
  3. Routing
    • Route by model? (e.g., gpt-4o-mini → backend A, claude-3-5-sonnet → backend B)
    • Need failover or traffic splitting?
  4. Environment
    • Namespace for Gateway and routes?
    • Gateway name (if reusing existing)?
在生成配置之前，请询问以下问题：
  1. 安装相关
    • 你是否已经安装了Envoy Gateway？如果没有，我们需要使用
      /aigw-install
      。
    • 是否需要限流功能或InferencePool？（附加组件）
  2. 服务提供商
    • 选择的AI服务提供商有哪些？（OpenAI、Anthropic、AWS Bedrock、Azure OpenAI、GCP Vertex AI、Cohere、自托管/Ollama等）
    • 对于云服务提供商：将采用何种认证方式？（API密钥、IRSA/Pod身份、服务账号等）
  3. 路由配置
    • 是否按模型路由？（例如：gpt-4o-mini → 后端A，claude-3-5-sonnet → 后端B）
    • 是否需要故障转移或流量拆分？
  4. 环境配置
    • Gateway和路由所在的命名空间？
    • Gateway名称（如果复用现有实例）？

Composition Flow

编排流程

  1. If fresh install: Run 
    /aigw-install
    with user's version/namespace preferences.
  2. Gateway + ClientTrafficPolicy: Ensure Gateway exists and has ClientTrafficPolicy with 
    bufferLimit: 50Mi
    .
  3. For each provider:
    • Run 
      /aigw-backend
      with BackendName, Schema, Hostname, Port. (AIServiceBackend must reference Backend, not K8s Service.)
    • Run 
      /aigw-auth
      with PolicyType and AIServiceBackendName; create Secret if API key. At most one BackendSecurityPolicy per backend.
    • Add BackendTLSPolicy for HTTPS backends.
  4. Route: Run 
    /aigw-route
    with GatewayName, BackendNames, and optional ModelHeader for each rule.
  1. 全新安装场景：根据用户选择的版本和命名空间，执行
    /aigw-install
    。
  2. Gateway + ClientTrafficPolicy：确保Gateway已存在，且配置了带有
    bufferLimit: 50Mi
    的ClientTrafficPolicy。
  3. 针对每个服务提供商：
    • 执行
      /aigw-backend
      ，传入BackendName、Schema、Hostname、Port参数。（AIServiceBackend必须关联Backend，而非K8s Service。）
    • 执行
      /aigw-auth
      ，传入PolicyType和AIServiceBackendName；若使用API密钥则创建Secret。每个后端最多配置一个BackendSecurityPolicy。
    • 为HTTPS后端添加BackendTLSPolicy。
  4. 路由配置：执行
    /aigw-route
    ，传入GatewayName、BackendNames，以及每个规则可选的ModelHeader。

Example: OpenAI + Anthropic

示例：OpenAI + Anthropic

Intake: User wants OpenAI (gpt-4o-mini) and Anthropic (claude-3-5-sonnet) behind one Gateway.
Generated flow:
  1. Install (if needed): 
    /aigw-install
  2. Gateway + ClientTrafficPolicy (from aigw-route skill)
  3. Backend + AIServiceBackend for OpenAI: 
    /aigw-backend
    BackendName=openai, Schema=OpenAI, Hostname=api.openai.com, Port=443
  4. BackendSecurityPolicy + Secret for OpenAI: 
    /aigw-auth
    PolicyType=APIKey, AIServiceBackendName=openai
  5. BackendTLSPolicy for api.openai.com
  6. Backend + AIServiceBackend for Anthropic: 
    /aigw-backend
    BackendName=anthropic, Schema=Anthropic, Hostname=api.anthropic.com, Port=443
  7. BackendSecurityPolicy + Secret for Anthropic: 
    /aigw-auth
    PolicyType=AnthropicAPIKey, AIServiceBackendName=anthropic
  8. BackendTLSPolicy for api.anthropic.com
  9. AIGatewayRoute with two rules:
    • Match x-ai-eg-model=gpt-4o-mini → openai
    • Match x-ai-eg-model=claude-3-5-sonnet → anthropic
需求收集：用户希望在同一个Gateway后部署OpenAI（gpt-4o-mini）和Anthropic（claude-3-5-sonnet）。
生成的编排流程：
  1. 安装（若未安装）：
    /aigw-install
  2. 配置Gateway + ClientTrafficPolicy（来自aigw-route技能）
  3. 为OpenAI配置Backend + AIServiceBackend：
    /aigw-backend
    BackendName=openai, Schema=OpenAI, Hostname=api.openai.com, Port=443
  4. 为OpenAI配置BackendSecurityPolicy + Secret：
    /aigw-auth
    PolicyType=APIKey, AIServiceBackendName=openai
  5. 为api.openai.com配置BackendTLSPolicy
  6. 为Anthropic配置Backend + AIServiceBackend：
    /aigw-backend
    BackendName=anthropic, Schema=Anthropic, Hostname=api.anthropic.com, Port=443
  7. 为Anthropic配置BackendSecurityPolicy + Secret：
    /aigw-auth
    PolicyType=AnthropicAPIKey, AIServiceBackendName=anthropic
  8. 为api.anthropic.com配置BackendTLSPolicy
  9. 配置AIGatewayRoute，包含两条规则：
    • 匹配x-ai-eg-model=gpt-4o-mini → 路由至openai
    • 匹配x-ai-eg-model=claude-3-5-sonnet → 路由至anthropic

Checklist

检查清单

  • All intake questions answered
  • Install steps included if needed
  • ClientTrafficPolicy with bufferLimit on Gateway
  • Each provider has Backend + AIServiceBackend + BackendSecurityPolicy + BackendTLSPolicy (for HTTPS)
  • At most one BackendSecurityPolicy per AIServiceBackend (or InferencePool)
  • AIGatewayRoute rules match user's routing intent
  • Secrets created for API keys (never hardcode keys in YAML)
  • 所有需求收集问题已答复
  • 按需包含安装步骤
  • Gateway已配置带bufferLimit的ClientTrafficPolicy
  • 每个服务提供商都配置了Backend + AIServiceBackend + BackendSecurityPolicy + BackendTLSPolicy（针对HTTPS）
  • 每个AIServiceBackend（或InferencePool）最多配置一个BackendSecurityPolicy
  • AIGatewayRoute规则符合用户的路由需求
  • 已为API密钥创建Secret（绝不在YAML中硬编码密钥）