specs-e2e-verification

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Specs E2E Verification

规范端到端验证

Overview

概述

Performs real environment verification after a specification has been implemented and cleaned up. This skill bridges the gap between unit-tested code and observable runtime behavior by:
  1. Detecting the application type from project files
  2. Starting the local runtime (Docker Compose, dev server, Spring Boot, etc.)
  3. Deriving tests from
    [IMP]
    acceptance criteria in the functional specification
  4. Executing real tests (
    curl
    , Playwright, computer-use)
  5. Mapping results to acceptance criteria
  6. Generating a verification report
  7. Tearing down the environment
Input:
docs/specs/[id]/
(spec folder with functional specification and tasks)
Output:
docs/specs/[id]/e2e-report-YYYY-MM-DD-HHMMSS.md
在规范实现并清理后执行真实环境验证。该技能通过以下方式弥补单元测试代码与可观察运行时行为之间的差距:
  1. 从项目文件中检测应用类型
  2. 启动本地运行时(Docker Compose、开发服务器、Spring Boot等)
  3. 从功能规范中的
    [IMP]
    验收标准推导测试用例
  4. 执行真实测试(
    curl
    、Playwright、computer-use)
  5. 将结果映射到验收标准
  6. 生成验证报告
  7. 清理环境
输入
docs/specs/[id]/
(包含功能规范和任务的规范文件夹) 输出
docs/specs/[id]/e2e-report-YYYY-MM-DD-HHMMSS.md

When to Use

使用场景

  • Use after
    specs.task-implementation
    and
    specs.code-cleanup
    to confirm the feature works in reality.
  • Use when a developer says "test it for real", "verify the API actually works", "run e2e checks", or "validate acceptance criteria live".
  • Use to generate evidence of feature completion before closing a specification.
  • Do NOT use for unit testing, static analysis, or code review — this is runtime behavioral verification only.
  • specs.task-implementation
    specs.code-cleanup
    之后使用,确认功能在实际环境中正常工作。
  • 当开发者说“真实测试一下”、“验证API是否真的可用”、“运行端到端检查”或“实时验证验收标准”时使用。
  • 在关闭规范之前,用于生成功能完成的证据。
  • 请勿用于单元测试、静态分析或代码审查——这仅针对运行时行为验证。

Arguments

参数

ArgumentRequiredDescription
--spec
YesPath to the specification folder (e.g.,
docs/specs/001-feature/
)
--task
NoSpecific task ID to limit verification scope (e.g.,
TASK-003
)
--keep-alive
NoIf present, skip teardown and leave the environment running
--timeout
NoStartup and test timeout in seconds (default: 120)
--insecure
NoIf present, allow curl to use
-k
/
--insecure
(TLS bypass opt-in)
参数是否必填描述
--spec
规范文件夹路径(例如:
docs/specs/001-feature/
--task
用于限制验证范围的特定任务ID(例如:
TASK-003
--keep-alive
如果存在,则跳过清理步骤,保持环境运行
--timeout
启动和测试超时时间(秒),默认值:120
--insecure
如果存在,则允许curl使用
-k
/
--insecure
(可选跳过TLS验证)

Best Practices

最佳实践

  • Safety first: Never run destructive commands (
    rm -rf
    ,
    docker system prune
    ,
    sudo
    ).
  • Detect, don’t assume: Use file heuristics to determine the app type; ask the user only when ambiguous.
  • AC-driven: Every test must trace back to an
    [IMP]
    acceptance criterion from the functional specification.
  • Clean up: Always teardown unless
    --keep-alive
    is passed; warn about leftover processes.
  • No code changes: This skill is read-only regarding source code. It may create reports but never patches logic.
  • Use TodoWrite: Track progress across all 8 phases.
  • 安全第一:绝不运行破坏性命令(
    rm -rf
    docker system prune
    sudo
    )。
  • 检测而非假设:使用文件启发式方法确定应用类型;仅在模糊不清时询问用户。
  • 以验收标准为驱动:每个测试必须追溯到功能规范中的
    [IMP]
    验收标准。
  • 清理环境:除非传递
    --keep-alive
    ,否则始终执行清理;提醒用户注意残留进程。
  • 不修改代码:该技能对源代码是只读的。它可能生成报告,但绝不会修补逻辑。
  • 使用TodoWrite:跟踪所有8个阶段的进度。

Instructions

操作步骤

Phase 1: Parse Arguments and Load Context

阶段1:解析参数并加载上下文

  1. Parse
    $ARGUMENTS
    :
    • --spec
      (required): spec folder path. Validate that the directory exists and contains at least one functional specification file (
      YYYY-MM-DD--*.md
      ). If missing or invalid, abort with an error.
    • --task
      (optional): task ID filter (e.g.,
      TASK-003
      ). If provided, validate that
      tasks/<task-id>.md
      exists inside the spec folder.
    • --keep-alive
      (optional): boolean flag. If present, skip teardown at the end.
    • --timeout
      (optional): positive integer in seconds. Default is
      120
      . Validate that the value is a positive integer; if not, abort with an error.
    • --insecure
      (optional): boolean flag. If present, curl commands MAY use
      -k
      /
      --insecure
      for local development with self-signed certificates. By default, TLS bypass is forbidden (REQ-NR003).
  2. Read the functional specification and extract:
    • All acceptance criteria with their taxonomy tags (
      [IMP]
      ,
      [SEF]
      ,
      [EXT]
      )
    • Only
      [IMP]
      criteria will generate runtime tests
  3. If
    --task
    is provided, read the task file and limit scope to its
    provides
    files and related AC.
  4. Use
    TodoWrite
    to create a todo list for all 8 phases.
  1. 解析
    $ARGUMENTS
    • --spec
      (必填):规范文件夹路径。验证目录存在且至少包含一个功能规范文件(
      YYYY-MM-DD--*.md
      )。如果缺失或无效,报错终止。
    • --task
      (可选):任务ID筛选器(例如:
      TASK-003
      )。如果提供,验证规范文件夹内存在
      tasks/<task-id>.md
    • --keep-alive
      (可选):布尔标记。如果存在,在结束时跳过清理步骤。
    • --timeout
      (可选):正整数(秒)。默认值为
      120
      。验证值为正整数;否则报错终止。
    • --insecure
      (可选):布尔标记。如果存在,curl命令可以使用
      -k
      /
      --insecure
      ,适用于使用自签名证书的本地开发场景。默认情况下,禁止跳过TLS验证(REQ-NR003)。
  2. 读取功能规范并提取:
    • 所有带分类标签的验收标准(
      [IMP]
      [SEF]
      [EXT]
    • [IMP]
      标准会生成运行时测试用例
  3. 如果提供了
    --task
    ,读取任务文件并将范围限制在其
    provides
    文件和相关验收标准中。
  4. 使用
    TodoWrite
    为所有8个阶段创建待办事项列表。

Phase 1.5: Security Validation Gate

阶段1.5:安全验证关卡

Before any command is executed, run the following security checks:
  1. Command Whitelist Check:
    • The startup command derived in Phase 3 MUST match a pattern documented in
      references/test-execution-patterns.md
      .
    • IF the command is NOT in the whitelist → use
      AskUserQuestion
      to request explicit user confirmation before execution.
    • Whitelisted commands include:
      docker compose up -d --build
      ,
      ./mvnw spring-boot:run
      ,
      ./gradlew bootRun
      ,
      npm run dev
      ,
      npm run start:dev
      ,
      npm start
      ,
      cargo tauri build --debug
      ,
      cargo tauri dev
      ,
      npm run electron:dev
      ,
      npx electron .
      ,
      open *.app
      , and equivalent local process launchers.
    • Any command containing
      sudo
      ,
      rm -rf
      ,
      docker system prune
      ,
      mkfs
      ,
      dd
      , or similar destructive operations is NOT whitelisted and SHALL be rejected.
  2. Forbidden Pattern Scan (REQ-NR001):
    • Scan the derived startup command and all generated test commands for:
      • sudo
        → abort with: "Forbidden: sudo is not permitted during E2E verification."
      • rm -rf
        → abort with: "Forbidden: rm -rf is not permitted during E2E verification."
      • docker system prune
        → abort with: "Forbidden: docker system prune is not permitted during E2E verification."
      • Any
        rm
        ,
        drop
        ,
        destroy
        ,
        prune
        targeting databases, volumes, or local data → abort with: "Forbidden: destructive data operations are not permitted."
    • IF any forbidden pattern is detected → abort immediately; do NOT proceed to startup.
  3. TLS Enforcement Check (REQ-NR003):
    • For any generated curl command:
      • IF it contains
        -k
        or
        --insecure
        AND
        --insecure
        was NOT passed → abort with: "Forbidden: curl TLS bypass (-k / --insecure) is disabled by default. Pass --insecure to opt-in."
      • IF
        --insecure
        was passed → log a warning: "WARNING: TLS certificate verification is disabled. Use only for local development."
  4. Data Integrity Pre-Check (REQ-NR004):
    • Before executing startup commands, inspect them for patterns that would overwrite or delete existing data (e.g.,
      rm
      ,
      drop
      ,
      prune
      , volume deletion flags).
    • IF the command would modify existing databases, volumes, or local data directories → abort with: "Forbidden: startup commands must not overwrite or delete existing data."
在执行任何命令之前,运行以下安全检查:
  1. 命令白名单检查
    • 阶段3推导的启动命令必须与
      references/test-execution-patterns.md
      中记录的模式匹配。
    • 如果命令不在白名单中→使用
      AskUserQuestion
      请求用户明确确认后再执行。
    • 白名单命令包括:
      docker compose up -d --build
      ./mvnw spring-boot:run
      ./gradlew bootRun
      npm run dev
      npm run start:dev
      npm start
      cargo tauri build --debug
      cargo tauri dev
      npm run electron:dev
      npx electron .
      open *.app
      以及等效的本地进程启动器。
    • 任何包含
      sudo
      rm -rf
      docker system prune
      mkfs
      dd
      或类似破坏性操作的命令均不在白名单中,应被拒绝。
  2. 禁止模式扫描(REQ-NR001):
    • 扫描推导的启动命令和所有生成的测试命令,检查是否包含:
      • sudo
        → 终止并提示:“禁止:E2E验证期间不允许使用sudo。”
      • rm -rf
        → 终止并提示:“禁止:E2E验证期间不允许使用rm -rf。”
      • docker system prune
        → 终止并提示:“禁止:E2E验证期间不允许使用docker system prune。”
      • 任何针对数据库、卷或本地数据的
        rm
        drop
        destroy
        prune
        操作→终止并提示:“禁止:不允许执行破坏性数据操作。”
    • 如果检测到任何禁止模式→立即终止;不继续执行启动步骤。
  3. TLS强制检查(REQ-NR003):
    • 对于任何生成的curl命令:
      • 如果包含
        -k
        --insecure
        且未传递
        --insecure
        参数→终止并提示:“禁止:默认禁用curl TLS绕过(-k / --insecure)。传递--insecure参数以启用该选项。”
      • 如果传递了
        --insecure
        →记录警告:“警告:已禁用TLS证书验证。仅用于本地开发场景。”
  4. 数据完整性预检查(REQ-NR004):
    • 在执行启动命令之前,检查命令是否包含会覆盖或删除现有数据的模式(例如:
      rm
      drop
      prune
      、卷删除标记)。
    • 如果命令会修改现有数据库、卷或本地数据目录→终止并提示:“禁止:启动命令不得覆盖或删除现有数据。”

Phase 2: Detect Application Type and Discover Port

阶段2:检测应用类型并发现端口

  1. Set
    PROJECT_ROOT
    to the directory containing
    .git
    or the parent directory of
    --spec
    .
  2. Inspect
    PROJECT_ROOT
    for configuration files using the following heuristics (execute in order):
    Docker-managed (highest priority):
    bash
    [ -f "$PROJECT_ROOT/docker-compose.yml" ] || [ -f "$PROJECT_ROOT/docker-compose.yaml" ] || [ -f "$PROJECT_ROOT/compose.yml" ]
    If any of these files exist, classify as Docker-managed regardless of other framework configs.
    JVM / Spring Boot:
    bash
    [ -f "$PROJECT_ROOT/pom.xml" ] || [ -f "$PROJECT_ROOT/build.gradle" ] || [ -f "$PROJECT_ROOT/build.gradle.kts" ]
    AND verify source directory exists:
    bash
    [ -d "$PROJECT_ROOT/src/main/java" ]
    If both conditions are true, classify as JVM-based service.
    NestJS:
    bash
    [ -f "$PROJECT_ROOT/package.json" ] && grep -q '"@nestjs/core"' "$PROJECT_ROOT/package.json"
    If true, classify as NestJS.
    Web SPA (React / Vue / Angular):
    bash
    [ -f "$PROJECT_ROOT/package.json" ] && ( grep -q '"react"' "$PROJECT_ROOT/package.json" || grep -q '"vue"' "$PROJECT_ROOT/package.json" || grep -q '"@angular/core"' "$PROJECT_ROOT/package.json" )
    If true, classify as Web SPA.
    Desktop App:
    bash
    [ -f "$PROJECT_ROOT/src-tauri/Cargo.toml" ] || ( [ -f "$PROJECT_ROOT/package.json" ] && grep -q '"electron"' "$PROJECT_ROOT/package.json" ) || [ -n "$(find "$PROJECT_ROOT" -maxdepth 2 -name '*.csproj' -print -quit 2>/dev/null)" ]
    If true, classify as Desktop App.
    Python:
    bash
    [ -f "$PROJECT_ROOT/requirements.txt" ] || [ -f "$PROJECT_ROOT/pyproject.toml" ] || [ -f "$PROJECT_ROOT/app.py" ] || [ -f "$PROJECT_ROOT/manage.py" ]
    If true, classify as Python.
  3. Apply priority rules:
    • Docker Compose is always prioritized: If a Docker Compose file exists AND any framework config also exists, classify as Docker-managed. The compose stack defines the runtime.
    • If multiple non-Docker configs are detected (e.g., both
      pom.xml
      and
      package.json
      without Docker Compose):
      • If the spec's domain clearly indicates backend vs frontend (e.g., spec title contains "API", "backend", "service"), prefer JVM or NestJS.
      • If the spec's domain clearly indicates frontend (e.g., spec title contains "UI", "page", "component"), prefer Web SPA.
      • If still ambiguous, proceed to user prompt (step 4).
  4. If no recognizable config is found, OR if multiple non-Docker configs exist and the spec domain is ambiguous, use
    AskUserQuestion
    with exactly these options:
    • "REST API"
    • "Web SPA"
    • "Desktop"
    • "Skip"
  5. Port Discovery: Once the application type is known, determine the target port by inspecting framework configuration files in this order:
    • Vite projects (
      vite.config.ts
      or
      vite.config.js
      ):
      bash
      grep -oE 'port:\s*[0-9]+' "$PROJECT_ROOT/vite.config.ts" 2>/dev/null | grep -oE '[0-9]+' || \
      grep -oE 'port:\s*[0-9]+' "$PROJECT_ROOT/vite.config.js" 2>/dev/null | grep -oE '[0-9]+'
    • Spring Boot (
      application.yml
      ):
      bash
      grep -A5 '^server:' "$PROJECT_ROOT/src/main/resources/application.yml" 2>/dev/null | grep 'port:' | head -1 | tr -dc '0-9'
    • Spring Boot (
      application.properties
      ):
      bash
      grep '^server.port=' "$PROJECT_ROOT/src/main/resources/application.properties" 2>/dev/null | cut -d= -f2 | tr -dc '0-9'
    • Node.js / package.json scripts:
      bash
      grep -oE -- '--port [0-9]+' "$PROJECT_ROOT/package.json" 2>/dev/null | grep -oE '[0-9]+' | head -1
      Also check for
      PORT
      environment variable in scripts:
      bash
      grep -oE 'PORT=[0-9]+' "$PROJECT_ROOT/package.json" 2>/dev/null | grep -oE '[0-9]+' | head -1
    • Fallback defaults (if no port is found in any config file):
      App TypeDefault Port
      Node.js / NestJS3000
      Spring Boot (JVM)8080
      Angular4200
      Vite (React/Vue)5173
      Python8000
  6. Log the detected type and discovered port; both will be recorded in the report.
  1. PROJECT_ROOT
    设置为包含
    .git
    的目录或
    --spec
    的父目录。
  2. 使用以下启发式规则检查
    PROJECT_ROOT
    中的配置文件(按顺序执行):
    Docker管理(最高优先级):
    bash
    [ -f "$PROJECT_ROOT/docker-compose.yml" ] || [ -f "$PROJECT_ROOT/docker-compose.yaml" ] || [ -f "$PROJECT_ROOT/compose.yml" ]
    如果存在上述任一文件,则归类为Docker管理,无论是否存在其他框架配置。
    JVM / Spring Boot
    bash
    [ -f "$PROJECT_ROOT/pom.xml" ] || [ -f "$PROJECT_ROOT/build.gradle" ] || [ -f "$PROJECT_ROOT/build.gradle.kts" ]
    同时验证源码目录存在:
    bash
    [ -d "$PROJECT_ROOT/src/main/java" ]
    如果两个条件都满足,则归类为JVM服务
    NestJS
    bash
    [ -f "$PROJECT_ROOT/package.json" ] && grep -q '"@nestjs/core"' "$PROJECT_ROOT/package.json"
    如果为真,则归类为NestJS
    Web SPA(React / Vue / Angular)
    bash
    [ -f "$PROJECT_ROOT/package.json" ] && ( grep -q '"react"' "$PROJECT_ROOT/package.json" || grep -q '"vue"' "$PROJECT_ROOT/package.json" || grep -q '"@angular/core"' "$PROJECT_ROOT/package.json" )
    如果为真,则归类为Web SPA
    桌面应用
    bash
    [ -f "$PROJECT_ROOT/src-tauri/Cargo.toml" ] || ( [ -f "$PROJECT_ROOT/package.json" ] && grep -q '"electron"' "$PROJECT_ROOT/package.json" ) || [ -n "$(find "$PROJECT_ROOT" -maxdepth 2 -name '*.csproj' -print -quit 2>/dev/null)" ]
    如果为真,则归类为桌面应用
    Python
    bash
    [ -f "$PROJECT_ROOT/requirements.txt" ] || [ -f "$PROJECT_ROOT/pyproject.toml" ] || [ -f "$PROJECT_ROOT/app.py" ] || [ -f "$PROJECT_ROOT/manage.py" ]
    如果为真,则归类为Python
  3. 应用优先级规则:
    • Docker Compose始终优先:如果存在Docker Compose文件且同时存在其他框架配置,则归类为Docker管理。Compose栈定义运行时环境。
    • 如果检测到多个非Docker配置(例如:同时存在
      pom.xml
      package.json
      但无Docker Compose):
      • 如果规范领域明确表明后端(例如:规范标题包含“API”、“backend”、“service”),优先选择JVMNestJS
      • 如果规范领域明确表明前端(例如:规范标题包含“UI”、“page”、“component”),优先选择Web SPA
      • 如果仍不明确,则进入用户提示步骤(步骤4)。
  4. 如果未识别到任何配置,或者存在多个非Docker配置且规范领域不明确,使用
    AskUserQuestion
    提供以下选项:
    • "REST API"
    • "Web SPA"
    • "Desktop"
    • "Skip"
  5. 端口发现:确定应用类型后,按以下顺序检查框架配置文件以确定目标端口:
    • Vite项目
      vite.config.ts
      vite.config.js
      ):
      bash
      grep -oE 'port:\s*[0-9]+' "$PROJECT_ROOT/vite.config.ts" 2>/dev/null | grep -oE '[0-9]+' || \
      grep -oE 'port:\s*[0-9]+' "$PROJECT_ROOT/vite.config.js" 2>/dev/null | grep -oE '[0-9]+'
    • Spring Boot
      application.yml
      ):
      bash
      grep -A5 '^server:' "$PROJECT_ROOT/src/main/resources/application.yml" 2>/dev/null | grep 'port:' | head -1 | tr -dc '0-9'
    • Spring Boot
      application.properties
      ):
      bash
      grep '^server.port=' "$PROJECT_ROOT/src/main/resources/application.properties" 2>/dev/null | cut -d= -f2 | tr -dc '0-9'
    • Node.js / package.json脚本
      bash
      grep -oE -- '--port [0-9]+' "$PROJECT_ROOT/package.json" 2>/dev/null | grep -oE '[0-9]+' | head -1
      同时检查脚本中的
      PORT
      环境变量:
      bash
      grep -oE 'PORT=[0-9]+' "$PROJECT_ROOT/package.json" 2>/dev/null | grep -oE '[0-9]+' | head -1
    • 默认回退端口(如果在任何配置文件中未找到端口):
      应用类型默认端口
      Node.js / NestJS3000
      Spring Boot (JVM)8080
      Angular4200
      Vite (React/Vue)5173
      Python8000
  6. 记录检测到的类型和发现的端口;两者都将记录在报告中。

Phase 3: Start Environment and Wait for Readiness

阶段3:启动环境并等待就绪

Read
references/test-execution-patterns.md
(shipped with this skill) for the command mapping. Based on detection:
Data integrity pre-flight (REQ-NR004): Before executing the startup command, verify it does not contain patterns that overwrite or delete existing databases, volumes, or local data (e.g.,
rm
,
--volumes
,
prune
,
drop
). If a destructive pattern is detected, abort immediately with: "Startup aborted: command would destroy existing data."
  1. Initialize runtime state:
    • STARTUP_COMMAND=""
      — the exact command used to start the environment
    • HEALTH_CHECK_METHOD=""
      — description of how readiness was determined
    • STARTUP_LOGS_FILE="$(mktemp)"
      — temp file capturing stdout/stderr from startup
    • STARTUP_TIMEOUT="${TIMEOUT:-120}"
      — seconds to wait for readiness
    • START_TIME="$(date +%s)"
    • STARTUP_PID=""
      — background process PID (for local processes)
  2. Pre-startup port check (all types):
    bash
    if lsof -i :"$TARGET_PORT" >/dev/null 2>&1 || nc -z localhost "$TARGET_PORT" 2>/dev/null; then
        echo "Port $TARGET_PORT is already in use."
        EXISTING_PID=$(lsof -ti:"$TARGET_PORT" | head -n1)
        EXISTING_CMD=$(ps -p "$EXISTING_PID" -o comm= 2>/dev/null || echo "unknown")
        if [ "$APP_TYPE" = "Docker-managed" ] && docker ps --format '{{.Names}}' 2>/dev/null | grep -q "$EXISTING_CMD"; then
            echo "Existing Docker container detected on port $TARGET_PORT; reusing it."
            STARTUP_COMMAND="(existing container reused)"
        elif [ "$APP_TYPE" = "JVM-based service" ] && echo "$EXISTING_CMD" | grep -q "java"; then
            echo "Existing Java process detected on port $TARGET_PORT; reusing it."
            STARTUP_COMMAND="(existing Java process reused)"
        else
            echo "Port $TARGET_PORT is occupied by an unrelated process ($EXISTING_CMD). Free the port and retry."
            exit 1
        fi
    fi
  3. Docker-managed startup:
    • Verify Docker daemon is reachable:
      bash
      if ! docker info >/dev/null 2>&1; then
          echo "Docker is not available. Please start Docker Desktop or use direct framework startup."
          exit 1
      fi
    • Set
      STARTUP_COMMAND="docker compose up -d --build"
    • Execute:
      bash
      cd "$PROJECT_ROOT" && docker compose up -d --build >> "$STARTUP_LOGS_FILE" 2>&1
    • Health check loop (timeout enforced):
      bash
      HEALTH_CHECK_METHOD="docker compose ps --format json"
      READY=false
      while [ $(( $(date +%s) - START_TIME )) -lt "$STARTUP_TIMEOUT" ]; do
          # Option A: Docker native health status (supports both JSON array and NDJSON)
          if docker compose ps --format json 2>/dev/null | jq -s -e '.[] | select(.Health=="healthy")' >/dev/null 2>&1; then
              READY=true
              HEALTH_CHECK_METHOD="docker compose ps (HEALTHCHECK=healthy)"
              break
          fi
          # Option B: Fallback port polling if no HEALTHCHECK defined
          if curl -sf "http://localhost:${TARGET_PORT}" >/dev/null 2>&1; then
              READY=true
              HEALTH_CHECK_METHOD="port polling via curl on localhost:${TARGET_PORT} (every 3s)"
              break
          fi
          sleep 3
      done
      if [ "$READY" != "true" ]; then
          echo "Startup timeout (${STARTUP_TIMEOUT}s) exceeded for Docker Compose."
          docker compose logs --tail=50 >> "$STARTUP_LOGS_FILE" 2>&1
          # Attempt cleanup to avoid orphan containers (REQ-NR006)
          docker compose down >/dev/null 2>&1 || true
          exit 1
      fi
  4. Spring Boot startup:
    • Determine build tool and launch in background with log capture:
      bash
      if [ -f "$PROJECT_ROOT/pom.xml" ]; then
          cd "$PROJECT_ROOT" && ./mvnw spring-boot:run -Dspring-boot.run.profiles=e2e >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="./mvnw spring-boot:run -Dspring-boot.run.profiles=e2e"
      elif [ -f "$PROJECT_ROOT/build.gradle" ] || [ -f "$PROJECT_ROOT/build.gradle.kts" ]; then
          cd "$PROJECT_ROOT" && ./gradlew bootRun --args='--spring.profiles.active=e2e' >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="./gradlew bootRun --args='--spring.profiles.active=e2e'"
      fi
    • Health check loop (timeout enforced):
      bash
      HEALTH_CHECK_METHOD="Spring Boot actuator /actuator/health"
      READY=false
      while [ $(( $(date +%s) - START_TIME )) -lt "$STARTUP_TIMEOUT" ]; do
          # Option A: Actuator health endpoint
          if curl -sf "http://localhost:${TARGET_PORT}/actuator/health" >/dev/null 2>&1; then
              READY=true
              HEALTH_CHECK_METHOD="Spring Boot actuator /actuator/health"
              break
          fi
          # Option B: Fallback to raw port readiness
          if nc -z localhost "$TARGET_PORT" 2>/dev/null; then
              READY=true
              HEALTH_CHECK_METHOD="port polling via nc on localhost:${TARGET_PORT} (every 3s)"
              break
          fi
          # Fail fast if the background process exited early
          if [ -n "$STARTUP_PID" ] && ! kill -0 "$STARTUP_PID" 2>/dev/null; then
              echo "Spring Boot process exited before reaching healthy state."
              break
          fi
          sleep 3
      done
      if [ "$READY" != "true" ]; then
          echo "Startup timeout (${STARTUP_TIMEOUT}s) exceeded for Spring Boot."
          # Capture last lines of startup logs for the report
          tail -n 100 "$STARTUP_LOGS_FILE" >> "$STARTUP_LOGS_FILE".final 2>&1 || true
          # Attempt cleanup (REQ-NR006)
          [ -n "$STARTUP_PID" ] && kill -TERM "$STARTUP_PID" 2>/dev/null || true
          exit 1
      fi
  5. NestJS / Node.js startup:
    • Verify
      node_modules
      exists to avoid cryptic errors:
      bash
      if [ ! -d "$PROJECT_ROOT/node_modules" ]; then
          echo "node_modules not found. Run 'npm install' before verification."
          exit 1
      fi
    • Determine the startup command from
      package.json
      scripts:
      bash
      if [ "$APP_TYPE" = "NestJS" ] && grep -q '"start:dev"' "$PROJECT_ROOT/package.json" 2>/dev/null; then
          cd "$PROJECT_ROOT" && npm run start:dev >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="npm run start:dev"
      elif grep -q '"dev"' "$PROJECT_ROOT/package.json" 2>/dev/null; then
          cd "$PROJECT_ROOT" && npm run dev >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="npm run dev"
      else
          cd "$PROJECT_ROOT" && npm start >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="npm start"
      fi
    • Health check loop (timeout enforced):
      bash
      HEALTH_CHECK_METHOD="port polling via nc/curl on localhost:${TARGET_PORT} (every 3s)"
      READY=false
      while [ $(( $(date +%s) - START_TIME )) -lt "$STARTUP_TIMEOUT" ]; do
          # Option A: HTTP readiness via curl
          if curl -sf "http://localhost:${TARGET_PORT}" >/dev/null 2>&1; then
              READY=true
              HEALTH_CHECK_METHOD="curl on localhost:${TARGET_PORT}"
              break
          fi
          # Option B: Raw port readiness via nc
          if nc -z localhost "$TARGET_PORT" 2>/dev/null; then
              READY=true
              HEALTH_CHECK_METHOD="nc -z localhost:${TARGET_PORT}"
              break
          fi
          # Fail fast if the background process exited early
          if [ -n "$STARTUP_PID" ] && ! kill -0 "$STARTUP_PID" 2>/dev/null; then
              echo "Node.js/NestJS process exited before reaching healthy state."
              break
          fi
          sleep 3
      done
      if [ "$READY" != "true" ]; then
          echo "Startup timeout (${STARTUP_TIMEOUT}s) exceeded for Node.js/NestJS."
          tail -n 100 "$STARTUP_LOGS_FILE" >> "$STARTUP_LOGS_FILE".final 2>&1 || true
          # Attempt cleanup (REQ-NR006)
          [ -n "$STARTUP_PID" ] && kill -TERM "$STARTUP_PID" 2>/dev/null || true
          exit 1
      fi
    • Console ready detection: After readiness is confirmed, scan
      $STARTUP_LOGS_FILE
      for common server-ready messages (e.g.,
      Nest application successfully started
      ,
      Local:
      ,
      ready in
      ,
      Server running
      ) and record the first matching line in the report as evidence of successful startup.
  6. Python (FastAPI / Django / Flask) (see
    references/test-execution-patterns.md
    ):
    • Run
      uvicorn main:app --reload
      ,
      python manage.py runserver
      , or
      flask run
    • Wait for port readiness
  7. Desktop App (Tauri / Electron / .NET MAUI) (see
    references/test-execution-patterns.md
    ):
    Framework detection:
    bash
    DESKTOP_FRAMEWORK=""
    if [ -f "$PROJECT_ROOT/src-tauri/Cargo.toml" ]; then
        DESKTOP_FRAMEWORK="tauri"
    elif [ -f "$PROJECT_ROOT/package.json" ] && grep -q '"electron"' "$PROJECT_ROOT/package.json"; then
        DESKTOP_FRAMEWORK="electron"
    elif [ -n "$(find "$PROJECT_ROOT" -maxdepth 2 -name '*.csproj' -print -quit 2>/dev/null)" ]; then
        DESKTOP_FRAMEWORK="dotnet-maui"
    fi
    Build step (triggered when no pre-built debug binary exists or when source is newer than target):
    • Tauri:
      bash
      if [ "$DESKTOP_FRAMEWORK" = "tauri" ]; then
          if ! command -v cargo >/dev/null 2>&1; then
              echo "ERROR: Rust/Cargo is not installed. Tauri build requires cargo."
              exit 1
          fi
          BUILD_COMMAND="cargo tauri build --debug"
          cd "$PROJECT_ROOT" && $BUILD_COMMAND >> "$STARTUP_LOGS_FILE" 2>&1
          BUILD_EXIT_CODE=$?
          if [ "$BUILD_EXIT_CODE" -ne 0 ]; then
              echo "Tauri build failed (exit code $BUILD_EXIT_CODE). See startup logs for details."
              exit 1
          fi
          STARTUP_COMMAND="$BUILD_COMMAND (build succeeded)"
      fi
    • Electron:
      bash
      if [ "$DESKTOP_FRAMEWORK" = "electron" ]; then
          if [ ! -d "$PROJECT_ROOT/node_modules" ]; then
              echo "ERROR: node_modules not found. Run 'npm install' before verification."
              exit 1
          fi
          BUILD_COMMAND="(no separate build required for dev mode)"
          STARTUP_COMMAND="npm run electron:dev"
          if grep -q '"electron:build"' "$PROJECT_ROOT/package.json" 2>/dev/null; then
              BUILD_COMMAND="npm run electron:build"
              cd "$PROJECT_ROOT" && $BUILD_COMMAND >> "$STARTUP_LOGS_FILE" 2>&1
              BUILD_EXIT_CODE=$?
              if [ "$BUILD_EXIT_CODE" -ne 0 ]; then
                  echo "Electron build failed (exit code $BUILD_EXIT_CODE). See startup logs for details."
                  exit 1
              fi
          fi
      fi
    Launch the built application binary:
    • Tauri (macOS):
      bash
      APP_BUNDLE=$(find "$PROJECT_ROOT/src-tauri/target/debug/bundle" -name "*.app" -print -quit 2>/dev/null)
      if [ -n "$APP_BUNDLE" ]; then
          open "$APP_BUNDLE" >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="open $APP_BUNDLE"
      else
          DEV_BINARY=$(find "$PROJECT_ROOT/src-tauri/target/debug" -maxdepth 1 -type f -executable ! -name '*.dylib' ! -name '*.so' -print -quit 2>/dev/null)
          if [ -n "$DEV_BINARY" ]; then
              "$DEV_BINARY" >> "$STARTUP_LOGS_FILE" 2>&1 &
              STARTUP_PID=$!
              STARTUP_COMMAND="$DEV_BINARY"
          else
              cargo tauri dev >> "$STARTUP_LOGS_FILE" 2>&1 &
              STARTUP_PID=$!
              STARTUP_COMMAND="cargo tauri dev"
          fi
      fi
    • Tauri (Linux):
      bash
      APP_BINARY=$(find "$PROJECT_ROOT/src-tauri/target/debug" -maxdepth 1 -type f -executable ! -name '*.so' -print -quit 2>/dev/null)
      if [ -n "$APP_BINARY" ]; then
          "$APP_BINARY" >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="$APP_BINARY"
      else
          cargo tauri dev >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="cargo tauri dev"
      fi
    • Electron:
      bash
      if grep -q '"electron:dev"' "$PROJECT_ROOT/package.json" 2>/dev/null; then
          cd "$PROJECT_ROOT" && npm run electron:dev >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="npm run electron:dev"
      elif grep -q '"start"' "$PROJECT_ROOT/package.json" 2>/dev/null; then
          cd "$PROJECT_ROOT" && npm start >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="npm start"
      else
          cd "$PROJECT_ROOT" && npx electron . >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="npx electron ."
      fi
    Health check (process appearance, timeout enforced):
    bash
    HEALTH_CHECK_METHOD="process polling via ps/kill -0 (every 3s)"
    READY=false
    while [ $(( $(date +%s) - START_TIME )) -lt "$STARTUP_TIMEOUT" ]; do
        if [ -n "$STARTUP_PID" ] && kill -0 "$STARTUP_PID" 2>/dev/null; then
            READY=true
            HEALTH_CHECK_METHOD="process PID ${STARTUP_PID} confirmed alive"
            break
        fi
        sleep 3
    done
    if [ "$READY" != "true" ]; then
        echo "Startup timeout (${STARTUP_TIMEOUT}s) exceeded for Desktop app."
        tail -n 100 "$STARTUP_LOGS_FILE" >> "$STARTUP_LOGS_FILE".final 2>&1 || true
        [ -n "$STARTUP_PID" ] && kill -TERM "$STARTUP_PID" 2>/dev/null || true
        exit 1
    fi
  8. Post-startup bookkeeping (all types):
    • Record
      STARTUP_COMMAND
      and
      HEALTH_CHECK_METHOD
      in report metadata (AC-010).
    • Append
      STARTUP_LOGS_FILE
      contents to the report under Raw Output (REQ-020).
    • If startup fails for any reason, capture all available logs and abort with a clear error message.
阅读与该技能配套的
references/test-execution-patterns.md
获取命令映射。根据检测结果执行:
数据完整性预检(REQ-NR004):在执行启动命令之前,验证它不包含会覆盖或删除现有数据库、卷或本地数据的模式(例如:
rm
--volumes
prune
drop
)。如果检测到破坏性模式,立即终止并提示:“启动终止:命令会破坏现有数据。”
  1. 初始化运行时状态
    • STARTUP_COMMAND=""
      — 用于启动环境的确切命令
    • HEALTH_CHECK_METHOD=""
      — 确定就绪状态的方式描述
    • STARTUP_LOGS_FILE="$(mktemp)"
      — 捕获启动stdout/stderr的临时文件
    • STARTUP_TIMEOUT="${TIMEOUT:-120}"
      — 等待就绪的秒数
    • START_TIME="$(date +%s)"
    • STARTUP_PID=""
      — 后台进程PID(针对本地进程)
  2. 启动前端口检查(所有类型):
    bash
    if lsof -i :"$TARGET_PORT" >/dev/null 2>&1 || nc -z localhost "$TARGET_PORT" 2>/dev/null; then
        echo "Port $TARGET_PORT is already in use."
        EXISTING_PID=$(lsof -ti:"$TARGET_PORT" | head -n1)
        EXISTING_CMD=$(ps -p "$EXISTING_PID" -o comm= 2>/dev/null || echo "unknown")
        if [ "$APP_TYPE" = "Docker-managed" ] && docker ps --format '{{.Names}}' 2>/dev/null | grep -q "$EXISTING_CMD"; then
            echo "Existing Docker container detected on port $TARGET_PORT; reusing it."
            STARTUP_COMMAND="(existing container reused)"
        elif [ "$APP_TYPE" = "JVM-based service" ] && echo "$EXISTING_CMD" | grep -q "java"; then
            echo "Existing Java process detected on port $TARGET_PORT; reusing it."
            STARTUP_COMMAND="(existing Java process reused)"
        else
            echo "Port $TARGET_PORT is occupied by an unrelated process ($EXISTING_CMD). Free the port and retry."
            exit 1
        fi
    fi
  3. Docker管理启动
    • 验证Docker守护进程可访问:
      bash
      if ! docker info >/dev/null 2>&1; then
          echo "Docker is not available. Please start Docker Desktop or use direct framework startup."
          exit 1
      fi
    • 设置
      STARTUP_COMMAND="docker compose up -d --build"
    • 执行:
      bash
      cd "$PROJECT_ROOT" && docker compose up -d --build >> "$STARTUP_LOGS_FILE" 2>&1
    • 健康检查循环(强制执行超时):
      bash
      HEALTH_CHECK_METHOD="docker compose ps --format json"
      READY=false
      while [ $(( $(date +%s) - START_TIME )) -lt "$STARTUP_TIMEOUT" ]; do
          # 选项A:Docker原生健康状态(支持JSON数组和NDJSON)
          if docker compose ps --format json 2>/dev/null | jq -s -e '.[] | select(.Health=="healthy")' >/dev/null 2>&1; then
              READY=true
              HEALTH_CHECK_METHOD="docker compose ps (HEALTHCHECK=healthy)"
              break
          fi
          # 选项B:如果未定义HEALTHCHECK,则回退到端口轮询
          if curl -sf "http://localhost:${TARGET_PORT}" >/dev/null 2>&1; then
              READY=true
              HEALTH_CHECK_METHOD="port polling via curl on localhost:${TARGET_PORT} (every 3s)"
              break
          fi
          sleep 3
      done
      if [ "$READY" != "true" ]; then
          echo "Startup timeout (${STARTUP_TIMEOUT}s) exceeded for Docker Compose."
          docker compose logs --tail=50 >> "$STARTUP_LOGS_FILE" 2>&1
          # 尝试清理以避免遗留容器(REQ-NR006)
          docker compose down >/dev/null 2>&1 || true
          exit 1
      fi
  4. Spring Boot启动
    • 确定构建工具并在后台启动,同时捕获日志:
      bash
      if [ -f "$PROJECT_ROOT/pom.xml" ]; then
          cd "$PROJECT_ROOT" && ./mvnw spring-boot:run -Dspring-boot.run.profiles=e2e >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="./mvnw spring-boot:run -Dspring-boot.run.profiles=e2e"
      elif [ -f "$PROJECT_ROOT/build.gradle" ] || [ -f "$PROJECT_ROOT/build.gradle.kts" ]; then
          cd "$PROJECT_ROOT" && ./gradlew bootRun --args='--spring.profiles.active=e2e' >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="./gradlew bootRun --args='--spring.profiles.active=e2e'"
      fi
    • 健康检查循环(强制执行超时):
      bash
      HEALTH_CHECK_METHOD="Spring Boot actuator /actuator/health"
      READY=false
      while [ $(( $(date +%s) - START_TIME )) -lt "$STARTUP_TIMEOUT" ]; do
          # 选项A:Actuator健康端点
          if curl -sf "http://localhost:${TARGET_PORT}/actuator/health" >/dev/null 2>&1; then
              READY=true
              HEALTH_CHECK_METHOD="Spring Boot actuator /actuator/health"
              break
          fi
          # 选项B:回退到原始端口就绪检查
          if nc -z localhost "$TARGET_PORT" 2>/dev/null; then
              READY=true
              HEALTH_CHECK_METHOD="port polling via nc on localhost:${TARGET_PORT} (every 3s)"
              break
          fi
          # 如果后台进程提前退出,立即终止
          if [ -n "$STARTUP_PID" ] && ! kill -0 "$STARTUP_PID" 2>/dev/null; then
              echo "Spring Boot process exited before reaching healthy state."
              break
          fi
          sleep 3
      done
      if [ "$READY" != "true" ]; then
          echo "Startup timeout (${STARTUP_TIMEOUT}s) exceeded for Spring Boot."
          # 捕获启动日志的最后几行用于报告
          tail -n 100 "$STARTUP_LOGS_FILE" >> "$STARTUP_LOGS_FILE".final 2>&1 || true
          # 尝试清理(REQ-NR006)
          [ -n "$STARTUP_PID" ] && kill -TERM "$STARTUP_PID" 2>/dev/null || true
          exit 1
      fi
  5. NestJS / Node.js启动
    • 验证
      node_modules
      存在以避免模糊错误:
      bash
      if [ ! -d "$PROJECT_ROOT/node_modules" ]; then
          echo "node_modules not found. Run 'npm install' before verification."
          exit 1
      fi
    • package.json
      脚本中确定启动命令:
      bash
      if [ "$APP_TYPE" = "NestJS" ] && grep -q '"start:dev"' "$PROJECT_ROOT/package.json" 2>/dev/null; then
          cd "$PROJECT_ROOT" && npm run start:dev >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="npm run start:dev"
      elif grep -q '"dev"' "$PROJECT_ROOT/package.json" 2>/dev/null; then
          cd "$PROJECT_ROOT" && npm run dev >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="npm run dev"
      else
          cd "$PROJECT_ROOT" && npm start >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="npm start"
      fi
    • 健康检查循环(强制执行超时):
      bash
      HEALTH_CHECK_METHOD="port polling via nc/curl on localhost:${TARGET_PORT} (every 3s)"
      READY=false
      while [ $(( $(date +%s) - START_TIME )) -lt "$STARTUP_TIMEOUT" ]; do
          # 选项A:通过curl检查HTTP就绪状态
          if curl -sf "http://localhost:${TARGET_PORT}" >/dev/null 2>&1; then
              READY=true
              HEALTH_CHECK_METHOD="curl on localhost:${TARGET_PORT}"
              break
          fi
          # 选项B:通过nc检查原始端口就绪状态
          if nc -z localhost "$TARGET_PORT" 2>/dev/null; then
              READY=true
              HEALTH_CHECK_METHOD="nc -z localhost:${TARGET_PORT}"
              break
          fi
          # 如果后台进程提前退出,立即终止
          if [ -n "$STARTUP_PID" ] && ! kill -0 "$STARTUP_PID" 2>/dev/null; then
              echo "Node.js/NestJS process exited before reaching healthy state."
              break
          fi
          sleep 3
      done
      if [ "$READY" != "true" ]; then
          echo "Startup timeout (${STARTUP_TIMEOUT}s) exceeded for Node.js/NestJS."
          tail -n 100 "$STARTUP_LOGS_FILE" >> "$STARTUP_LOGS_FILE".final 2>&1 || true
          # 尝试清理(REQ-NR006)
          [ -n "$STARTUP_PID" ] && kill -TERM "$STARTUP_PID" 2>/dev/null || true
          exit 1
      fi
    • 控制台就绪检测:确认就绪后,扫描
      $STARTUP_LOGS_FILE
      查找常见的服务器就绪消息(例如:
      Nest application successfully started
      Local:
      ready in
      Server running
      ),并将第一个匹配的行记录在报告中作为成功启动的证据。
  6. Python(FastAPI / Django / Flask)(参见
    references/test-execution-patterns.md
    ):
    • 运行
      uvicorn main:app --reload
      python manage.py runserver
      flask run
    • 等待端口就绪
  7. 桌面应用(Tauri / Electron / .NET MAUI)(参见
    references/test-execution-patterns.md
    ):
    框架检测
    bash
    DESKTOP_FRAMEWORK=""
    if [ -f "$PROJECT_ROOT/src-tauri/Cargo.toml" ]; then
        DESKTOP_FRAMEWORK="tauri"
    elif [ -f "$PROJECT_ROOT/package.json" ] && grep -q '"electron"' "$PROJECT_ROOT/package.json"; then
        DESKTOP_FRAMEWORK="electron"
    elif [ -n "$(find "$PROJECT_ROOT" -maxdepth 2 -name '*.csproj' -print -quit 2>/dev/null)" ]; then
        DESKTOP_FRAMEWORK="dotnet-maui"
    fi
    构建步骤(当不存在预构建的调试二进制文件或源代码比目标文件新时触发):
    • Tauri
      bash
      if [ "$DESKTOP_FRAMEWORK" = "tauri" ]; then
          if ! command -v cargo >/dev/null 2>&1; then
              echo "ERROR: Rust/Cargo is not installed. Tauri build requires cargo."
              exit 1
          fi
          BUILD_COMMAND="cargo tauri build --debug"
          cd "$PROJECT_ROOT" && $BUILD_COMMAND >> "$STARTUP_LOGS_FILE" 2>&1
          BUILD_EXIT_CODE=$?
          if [ "$BUILD_EXIT_CODE" -ne 0 ]; then
              echo "Tauri build failed (exit code $BUILD_EXIT_CODE). See startup logs for details."
              exit 1
          fi
          STARTUP_COMMAND="$BUILD_COMMAND (build succeeded)"
      fi
    • Electron
      bash
      if [ "$DESKTOP_FRAMEWORK" = "electron" ]; then
          if [ ! -d "$PROJECT_ROOT/node_modules" ]; then
              echo "ERROR: node_modules not found. Run 'npm install' before verification."
              exit 1
          fi
          BUILD_COMMAND="(no separate build required for dev mode)"
          STARTUP_COMMAND="npm run electron:dev"
          if grep -q '"electron:build"' "$PROJECT_ROOT/package.json" 2>/dev/null; then
              BUILD_COMMAND="npm run electron:build"
              cd "$PROJECT_ROOT" && $BUILD_COMMAND >> "$STARTUP_LOGS_FILE" 2>&1
              BUILD_EXIT_CODE=$?
              if [ "$BUILD_EXIT_CODE" -ne 0 ]; then
                  echo "Electron build failed (exit code $BUILD_EXIT_CODE). See startup logs for details."
                  exit 1
              fi
          fi
      fi
    启动已构建的应用二进制文件
    • Tauri(macOS)
      bash
      APP_BUNDLE=$(find "$PROJECT_ROOT/src-tauri/target/debug/bundle" -name "*.app" -print -quit 2>/dev/null)
      if [ -n "$APP_BUNDLE" ]; then
          open "$APP_BUNDLE" >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="open $APP_BUNDLE"
      else
          DEV_BINARY=$(find "$PROJECT_ROOT/src-tauri/target/debug" -maxdepth 1 -type f -executable ! -name '*.dylib' ! -name '*.so' -print -quit 2>/dev/null)
          if [ -n "$DEV_BINARY" ]; then
              "$DEV_BINARY" >> "$STARTUP_LOGS_FILE" 2>&1 &
              STARTUP_PID=$!
              STARTUP_COMMAND="$DEV_BINARY"
          else
              cargo tauri dev >> "$STARTUP_LOGS_FILE" 2>&1 &
              STARTUP_PID=$!
              STARTUP_COMMAND="cargo tauri dev"
          fi
      fi
    • Tauri(Linux)
      bash
      APP_BINARY=$(find "$PROJECT_ROOT/src-tauri/target/debug" -maxdepth 1 -type f -executable ! -name '*.so' -print -quit 2>/dev/null)
      if [ -n "$APP_BINARY" ]; then
          "$APP_BINARY" >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="$APP_BINARY"
      else
          cargo tauri dev >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="cargo tauri dev"
      fi
    • Electron
      bash
      if grep -q '"electron:dev"' "$PROJECT_ROOT/package.json" 2>/dev/null; then
          cd "$PROJECT_ROOT" && npm run electron:dev >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="npm run electron:dev"
      elif grep -q '"start"' "$PROJECT_ROOT/package.json" 2>/dev/null; then
          cd "$PROJECT_ROOT" && npm start >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="npm start"
      else
          cd "$PROJECT_ROOT" && npx electron . >> "$STARTUP_LOGS_FILE" 2>&1 &
          STARTUP_PID=$!
          STARTUP_COMMAND="npx electron ."
      fi
    健康检查(进程存在性检查,强制执行超时):
    bash
    HEALTH_CHECK_METHOD="process polling via ps/kill -0 (every 3s)"
    READY=false
    while [ $(( $(date +%s) - START_TIME )) -lt "$STARTUP_TIMEOUT" ]; do
        if [ -n "$STARTUP_PID" ] && kill -0 "$STARTUP_PID" 2>/dev/null; then
            READY=true
            HEALTH_CHECK_METHOD="process PID ${STARTUP_PID} confirmed alive"
            break
        fi
        sleep 3
    done
    if [ "$READY" != "true" ]; then
        echo "Startup timeout (${STARTUP_TIMEOUT}s) exceeded for Desktop app."
        tail -n 100 "$STARTUP_LOGS_FILE" >> "$STARTUP_LOGS_FILE".final 2>&1 || true
        [ -n "$STARTUP_PID" ] && kill -TERM "$STARTUP_PID" 2>/dev/null || true
        exit 1
    fi
  8. 启动后记录(所有类型):
    • 在报告元数据中记录
      STARTUP_COMMAND
      HEALTH_CHECK_METHOD
      (AC-010)。
    • STARTUP_LOGS_FILE
      的内容追加到报告的原始输出部分(REQ-020)。
    • 如果启动因任何原因失败,捕获所有可用日志并以清晰的错误消息终止。

Phase 4: Generate and Execute Tests

阶段4:生成并执行测试

CRITICAL: Only test
[IMP]
acceptance criteria. Translate each into one or more concrete runtime actions.
关键要求:仅测试
[IMP]
验收标准。将每个标准转换为一个或多个具体的运行时操作。

REST API Tests (curl)

REST API测试(curl)

Prerequisite check: Before generating any tests, verify
curl
is installed:
bash
if ! command -v curl >/dev/null 2>&1; then
    echo "ERROR: curl is not installed."
    echo "Install instructions:"
    echo "  macOS:    brew install curl"
    echo "  Ubuntu:   sudo apt-get install curl"
    echo "  Windows:  choco install curl   or   winget install curl"
    exit 1
fi
if ! command -v jq >/dev/null 2>&1; then
    echo "WARNING: jq is not installed. JSON body assertions will fall back to grep (less precise)."
    echo "Install instructions:"
    echo "  macOS:    brew install jq"
    echo "  Ubuntu:   sudo apt-get install jq"
    echo "  Windows:  choco install jq   or   winget install jqlang.jq"
fi
1. Parse
[IMP]
AC for endpoint hints
For each
[IMP]
acceptance criterion in the specification:
  • Extract the HTTP method by searching for keywords:
    GET
    ,
    POST
    ,
    PUT
    ,
    PATCH
    ,
    DELETE
    (case-insensitive).
  • Extract the endpoint path by searching for patterns starting with
    /
    followed by alphanumeric segments, e.g.,
    /api/users
    ,
    /v1/health
    .
  • Extract the expected HTTP status code by searching for numeric patterns
    2xx
    ,
    3xx
    ,
    4xx
    ,
    5xx
    or specific codes like
    200
    ,
    201
    ,
    204
    ,
    400
    ,
    401
    ,
    403
    ,
    404
    ,
    500
    .
  • Extract expected
    Content-Type
    by searching for
    application/json
    ,
    text/plain
    ,
    text/html
    , etc.
  • Extract expected response body hints (field names, array presence, string values) from the AC text.
If an
[IMP]
AC does not contain a parseable endpoint path and method, mark it
MANUAL CHECK REQUIRED
and skip to the next criterion.
2. Discover authentication credentials
Before constructing curl commands, attempt to locate test credentials by scanning the following files in
PROJECT_ROOT
(in order):
FileKey Patterns
.env.test
E2E_AUTH_TOKEN=...
,
E2E_USERNAME=...
,
E2E_PASSWORD=...
.env.local
E2E_AUTH_TOKEN=...
,
E2E_USERNAME=...
,
E2E_PASSWORD=...
application-test.yml
e2e.auth-token: ...
,
e2e.username: ...
,
e2e.password: ...
application-test.properties
e2e.auth-token=...
,
e2e.username=...
,
e2e.password=...
e2e.credentials.json
Top-level keys
E2E_AUTH_TOKEN
,
E2E_USERNAME
,
E2E_PASSWORD
Discovery logic:
bash
undefined
前提检查:在生成任何测试之前,验证
curl
已安装:
bash
if ! command -v curl >/dev/null 2>&1; then
    echo "ERROR: curl is not installed."
    echo "Install instructions:"
    echo "  macOS:    brew install curl"
    echo "  Ubuntu:   sudo apt-get install curl"
    echo "  Windows:  choco install curl   or   winget install curl"
    exit 1
fi
if ! command -v jq >/dev/null 2>&1; then
    echo "WARNING: jq is not installed. JSON body assertions will fall back to grep (less precise)."
    echo "Install instructions:"
    echo "  macOS:    brew install jq"
    echo "  Ubuntu:   sudo apt-get install jq"
    echo "  Windows:  choco install jq   or   winget install jqlang.jq"
fi
1. 解析
[IMP]
验收标准以获取端点提示
对于规范中的每个
[IMP]
验收标准:
  • 通过搜索关键字提取HTTP方法:
    GET
    POST
    PUT
    PATCH
    DELETE
    (不区分大小写)。
  • 通过搜索以
    /
    开头后跟字母数字段的模式提取端点路径,例如:
    /api/users
    /v1/health
  • 通过搜索数字模式
    2xx
    3xx
    4xx
    5xx
    或特定代码(如
    200
    201
    204
    400
    401
    403
    404
    500
    )提取预期HTTP状态码。
  • 通过搜索
    application/json
    text/plain
    text/html
    等提取预期
    Content-Type
  • 从验收标准文本中提取预期响应体提示(字段名、数组存在性、字符串值)。
如果
[IMP]
验收标准包含可解析的端点路径和方法,则标记为
MANUAL CHECK REQUIRED
并跳过下一个标准。
2. 发现认证凭证
在构建curl命令之前,尝试通过按顺序扫描
PROJECT_ROOT
中的以下文件来查找测试凭证:
文件关键字模式
.env.test
E2E_AUTH_TOKEN=...
E2E_USERNAME=...
E2E_PASSWORD=...
.env.local
E2E_AUTH_TOKEN=...
E2E_USERNAME=...
E2E_PASSWORD=...
application-test.yml
e2e.auth-token: ...
e2e.username: ...
e2e.password: ...
application-test.properties
e2e.auth-token=...
e2e.username=...
e2e.password=...
e2e.credentials.json
顶级键
E2E_AUTH_TOKEN
E2E_USERNAME
E2E_PASSWORD
发现逻辑:
bash
undefined

.env files

.env文件

[ -f "$PROJECT_ROOT/.env.test" ] && export $(grep -E '^(E2E_AUTH_TOKEN|E2E_USERNAME|E2E_PASSWORD)=' "$PROJECT_ROOT/.env.test" | xargs) [ -f "$PROJECT_ROOT/.env.local" ] && export $(grep -E '^(E2E_AUTH_TOKEN|E2E_USERNAME|E2E_PASSWORD)=' "$PROJECT_ROOT/.env.local" | xargs)
[ -f "$PROJECT_ROOT/.env.test" ] && export $(grep -E '^(E2E_AUTH_TOKEN|E2E_USERNAME|E2E_PASSWORD)=' "$PROJECT_ROOT/.env.test" | xargs) [ -f "$PROJECT_ROOT/.env.local" ] && export $(grep -E '^(E2E_AUTH_TOKEN|E2E_USERNAME|E2E_PASSWORD)=' "$PROJECT_ROOT/.env.local" | xargs)

Spring YAML

Spring YAML

[ -f "$PROJECT_ROOT/src/main/resources/application-test.yml" ] &&
E2E_AUTH_TOKEN=$(grep -A1 'e2e:' "$PROJECT_ROOT/src/main/resources/application-test.yml" | grep 'auth-token:' | sed 's/.*: *//')
[ -f "$PROJECT_ROOT/src/main/resources/application-test.yml" ] &&
E2E_AUTH_TOKEN=$(grep -A1 'e2e:' "$PROJECT_ROOT/src/main/resources/application-test.yml" | grep 'auth-token:' | sed 's/.*: *//')

Spring properties

Spring properties

[ -f "$PROJECT_ROOT/src/main/resources/application-test.properties" ] &&
E2E_AUTH_TOKEN=$(grep '^e2e.auth-token=' "$PROJECT_ROOT/src/main/resources/application-test.properties" | cut -d= -f2-)
[ -f "$PROJECT_ROOT/src/main/resources/application-test.properties" ] &&
E2E_AUTH_TOKEN=$(grep '^e2e.auth-token=' "$PROJECT_ROOT/src/main/resources/application-test.properties" | cut -d= -f2-)

JSON credentials file

JSON凭证文件

[ -f "$PROJECT_ROOT/e2e.credentials.json" ] &&
E2E_AUTH_TOKEN=$(jq -r '.E2E_AUTH_TOKEN // empty' "$PROJECT_ROOT/e2e.credentials.json")

If **no** credentials are found after scanning all files AND the AC text implies authentication is required (mentions "auth", "login", "token", "protected", "bearer", "API key"), use `AskUserQuestion` to prompt the user:
- "Enter E2E_AUTH_TOKEN (or leave blank if none)"
- "Enter E2E_USERNAME (or leave blank if none)"
- "Enter E2E_PASSWORD (or leave blank if none)"

**Security**: Redact token values in the E2E report; show only the header name (e.g., `Authorization: Bearer <redacted>`).

**3. Generate curl commands**

For each parseable `[IMP]` AC, construct the curl command using this exact pattern:
```bash
curl -s -w "\n%{http_code}" -o /tmp/e2e_resp.json \
  -X <METHOD> \
  -H "Content-Type: application/json" \
  <AUTH_HEADER> \
  -d '<REQUEST_BODY>' \
  "http://localhost:${TARGET_PORT}<PATH>"
Rules:
  • Always include
    -s -w "\n%{http_code}" -o /tmp/e2e_resp.json
    .
  • -X <METHOD>
    : only add if the method is not
    GET
    . For
    GET
    , omit
    -X
    entirely.
  • -H "Content-Type: application/json"
    : only add for
    POST
    ,
    PUT
    ,
    PATCH
    .
  • -d '<REQUEST_BODY>'
    : only add when the AC describes a request body. If no body is described, omit
    -d
    .
  • <AUTH_HEADER>
    :
    • If
      E2E_AUTH_TOKEN
      is set:
      -H "Authorization: Bearer ${E2E_AUTH_TOKEN}"
    • If
      E2E_USERNAME
      and
      E2E_PASSWORD
      are set:
      -u "${E2E_USERNAME}:${E2E_PASSWORD}"
    • If no credentials found: omit auth header.
  • TLS enforcement (REQ-NR003): NEVER add
    -k
    or
    --insecure
    to curl commands unless the
    --insecure
    flag was explicitly passed when invoking the skill. If
    --insecure
    was passed, log a warning that TLS verification is disabled.
  • Store the raw response body in
    /tmp/e2e_resp.json
    and the status code on the last line of stdout.
4. Execute curl and assert (no retry)
Execute each curl command exactly once (REQ-NR008). Do NOT retry on failure.
bash
HTTP_CODE=$(curl -s -w "\n%{http_code}" -o /tmp/e2e_resp.json <curl args> | tail -n 1)
Assertions (all must pass for the AC to be
VERIFIED
):
a. HTTP status code:
bash
if [ "$HTTP_CODE" -ne "$EXPECTED_STATUS" ]; then
    echo "FAIL: Expected status $EXPECTED_STATUS, got $HTTP_CODE"
    STATUS="FAILED"
fi
b. Content-Type header (only if specified in the AC):
bash
ACTUAL_CT=$(curl -s -o /dev/null -D - <curl args> | grep -i "Content-Type:" | head -1 | sed 's/Content-Type: //i' | tr -d '\r')
if [ -n "$EXPECTED_CT" ] && ! echo "$ACTUAL_CT" | grep -qi "$EXPECTED_CT"; then
    echo "FAIL: Expected Content-Type '$EXPECTED_CT', got '$ACTUAL_CT'"
    STATUS="FAILED"
fi
c. Response body structure using
jq
(preferred) or
grep
(fallback):
  • If
    jq
    is installed and the response is JSON:
    bash
    # Assert field exists
    jq -e '.fieldName' /tmp/e2e_resp.json >/dev/null || { echo "FAIL: Missing .fieldName"; STATUS="FAILED"; }
    
    # Assert field equals expected value
    jq -e '.fieldName == "expectedValue"' /tmp/e2e_resp.json >/dev/null || { echo "FAIL: .fieldName mismatch"; STATUS="FAILED"; }
    
    # Assert array length
    jq -e '(.items | length) > 0' /tmp/e2e_resp.json >/dev/null || { echo "FAIL: .items is empty"; STATUS="FAILED"; }
    
    # Assert nested field
    jq -e '.data.user.email' /tmp/e2e_resp.json >/dev/null || { echo "FAIL: Missing .data.user.email"; STATUS="FAILED"; }
  • If
    jq
    is NOT installed, use
    grep
    as fallback:
    bash
    grep -q '"fieldName"' /tmp/e2e_resp.json || { echo "FAIL: Missing fieldName"; STATUS="FAILED"; }
5. Record results
For each curl test, record:
  • AC ID and text (truncated)
  • Generated curl command (with auth tokens redacted)
  • Expected status code, Content-Type, body assertions
  • Actual status code, Content-Type, body snippet (first 500 chars)
  • Pass/fail status
  • Execution time (optional, for report reference)
If any assertion fails, mark the AC as
FAILED
immediately. Do not retry.
[ -f "$PROJECT_ROOT/e2e.credentials.json" ] &&
E2E_AUTH_TOKEN=$(jq -r '.E2E_AUTH_TOKEN // empty' "$PROJECT_ROOT/e2e.credentials.json")

如果扫描所有文件后**未**找到凭证,且验收标准文本暗示需要认证(提及“auth”、“login”、“token”、“protected”、“bearer”、“API key”),则使用`AskUserQuestion`提示用户:
- "输入E2E_AUTH_TOKEN(无则留空)"
- "输入E2E_USERNAME(无则留空)"
- "输入E2E_PASSWORD(无则留空)"

**安全要求**:在E2E报告中编辑令牌值;仅显示标头名称(例如:`Authorization: Bearer <redacted>`)。

**3. 生成curl命令**

对于每个可解析的`[IMP]`验收标准,使用以下精确模式构建curl命令:
```bash
curl -s -w "\n%{http_code}" -o /tmp/e2e_resp.json \
  -X <METHOD> \
  -H "Content-Type: application/json" \
  <AUTH_HEADER> \
  -d '<REQUEST_BODY>' \
  "http://localhost:${TARGET_PORT}<PATH>"
规则:
  • 始终包含
    -s -w "\n%{http_code}" -o /tmp/e2e_resp.json
  • -X <METHOD>
    :仅当方法不是
    GET
    时添加。对于
    GET
    ,完全省略
    -X
  • -H "Content-Type: application/json"
    :仅针对
    POST
    PUT
    PATCH
    添加。
  • -d '<REQUEST_BODY>'
    :仅当验收标准描述了请求体时添加。如果未描述请求体,则省略
    -d
  • <AUTH_HEADER>
    • 如果设置了
      E2E_AUTH_TOKEN
      -H "Authorization: Bearer ${E2E_AUTH_TOKEN}"
    • 如果设置了
      E2E_USERNAME
      E2E_PASSWORD
      -u "${E2E_USERNAME}:${E2E_PASSWORD}"
    • 如果未找到凭证:省略认证标头。
  • TLS强制要求(REQ-NR003):除非调用技能时明确传递了
    --insecure
    标志,否则绝不向curl命令添加
    -k
    --insecure
    。如果传递了
    --insecure
    ,则记录警告,提示已禁用TLS验证。
  • 将原始响应体存储在
    /tmp/e2e_resp.json
    中,状态码存储在stdout的最后一行。
4. 执行curl并断言(不重试)
每个curl命令仅执行一次(REQ-NR008)。失败时不重试。
bash
HTTP_CODE=$(curl -s -w "\n%{http_code}" -o /tmp/e2e_resp.json <curl args> | tail -n 1)
断言(所有断言必须通过,验收标准才能标记为
VERIFIED
):
a. HTTP状态码
bash
if [ "$HTTP_CODE" -ne "$EXPECTED_STATUS" ]; then
    echo "FAIL: Expected status $EXPECTED_STATUS, got $HTTP_CODE"
    STATUS="FAILED"
fi
b. Content-Type标头(仅当验收标准中指定时):
bash
ACTUAL_CT=$(curl -s -o /dev/null -D - <curl args> | grep -i "Content-Type:" | head -1 | sed 's/Content-Type: //i' | tr -d '\r')
if [ -n "$EXPECTED_CT" ] && ! echo "$ACTUAL_CT" | grep -qi "$EXPECTED_CT"; then
    echo "FAIL: Expected Content-Type '$EXPECTED_CT', got '$ACTUAL_CT'"
    STATUS="FAILED"
fi
c. 响应体结构(优先使用
jq
,回退使用
grep
):
  • 如果已安装
    jq
    且响应为JSON:
    bash
    # 断言字段存在
    jq -e '.fieldName' /tmp/e2e_resp.json >/dev/null || { echo "FAIL: Missing .fieldName"; STATUS="FAILED"; }
    
    # 断言字段等于预期值
    jq -e '.fieldName == "expectedValue"' /tmp/e2e_resp.json >/dev/null || { echo "FAIL: .fieldName mismatch"; STATUS="FAILED"; }
    
    # 断言数组长度
    jq -e '(.items | length) > 0' /tmp/e2e_resp.json >/dev/null || { echo "FAIL: .items is empty"; STATUS="FAILED"; }
    
    # 断言嵌套字段
    jq -e '.data.user.email' /tmp/e2e_resp.json >/dev/null || { echo "FAIL: Missing .data.user.email"; STATUS="FAILED"; }
  • 如果未安装
    jq
    ,则使用
    grep
    作为回退:
    bash
    grep -q '"fieldName"' /tmp/e2e_resp.json || { echo "FAIL: Missing fieldName"; STATUS="FAILED"; }
5. 记录结果
对于每个curl测试,记录:
  • 验收标准ID和文本(截断)
  • 生成的curl命令(已编辑认证令牌)
  • 预期状态码、Content-Type、响应体断言
  • 实际状态码、Content-Type、响应体片段(前500个字符)
  • 通过/失败状态
  • 执行时间(可选,用于报告参考)
如果任何断言失败,立即将验收标准标记为
FAILED
。不重试。

Web SPA Tests (Playwright)

Web SPA测试(Playwright)

Prerequisite check: Before generating any SPA tests, verify Playwright is installed:
bash
PLAYWRIGHT_VERSION=$(npx playwright --version 2>/dev/null || echo "")
if [ -z "$PLAYWRIGHT_VERSION" ]; then
    echo "ERROR: Playwright is not installed."
    echo "Install instructions:"
    echo "  npm install -D @playwright/test"
    echo "  npx playwright install chromium"
    echo ""
    echo "Skipping all Web SPA tests. Acceptance criteria for SPA interactions will be marked MANUAL CHECK REQUIRED."
    for ac_id in $(get_spa_ac_ids); do
        record_result "$ac_id" "MANUAL CHECK REQUIRED" "Playwright not installed"
    done
    # Continue to next test category; do not abort the whole verification
fi
echo "Playwright version: $PLAYWRIGHT_VERSION"
If Playwright is missing, the skill MUST report the gap with the install commands above and skip SPA tests. Do NOT attempt to auto-install.
1. Prepare artifact directory
bash
ARTIFACT_DIR="${SPEC_FOLDER}/e2e-artifacts"
mkdir -p "$ARTIFACT_DIR"
2. Parse
[IMP]
AC for UI behavior hints
For each
[IMP]
acceptance criterion in the specification that relates to Web SPA behavior:
  • Extract UI interaction keywords:
    click
    ,
    fill
    ,
    type
    ,
    select
    ,
    submit
    ,
    navigate
    ,
    scroll
    ,
    hover
    .
  • Extract target selectors by searching for patterns:
    • data-testid="..."
      or
      data-testid='...'
      [data-testid=...]
    • id="..."
      or
      id='...'
      #...
    • class="..."
      or
      class='...'
      .class-name
      (replace spaces with dots)
    • Button/link text mentions →
      text=...
    • URL path mentions →
      /path
  • Extract expected visible states: text content expectations, URL expectations, element presence/absence, count expectations.
  • Extract form field names and expected input values.
If an
[IMP]
AC does not contain parseable UI behavior or visible state hints, mark it
MANUAL CHECK REQUIRED
and skip to the next criterion.
3. Launch headless browser context
Browser MUST be headless by default. Only use headed mode if the user explicitly passes
--headed
.
For each SPA test, generate a temporary Playwright script and execute it with
node
:
bash
DEV_SERVER_URL="http://localhost:${TARGET_PORT}"
TEST_SCRIPT="$(mktemp /tmp/e2e-spa-XXXXXX.js)"

cat > "$TEST_SCRIPT" << 'PLAYWRIGHT_EOF'
const { chromium } = require('playwright');
(async () => {
  const browser = await chromium.launch({ headless: true });
  const context = await browser.newContext({
    viewport: { width: 1280, height: 720 },
    userAgent: 'DeveloperKit-E2E/1.0'
  });
  const page = await context.newPage();
  // Actions and assertions injected here
  await browser.close();
})();
PLAYWRIGHT_EOF

node "$TEST_SCRIPT"
4. Translate AC into Playwright actions
For each parsed UI interaction, generate the corresponding Playwright action inside the temporary script:
AC Description PatternPlaywright Action
"click [selector]"
await page.click('[data-testid=refresh]');
"fill [selector] with [value]"
await page.fill('#username', 'testuser');
"type [value] into [selector]"
await page.type('input[name=search]', 'query');
"select [value] in [selector]"
await page.selectOption('select[name=country]', 'US');
"submit [form]"
await page.click('button[type=submit]');
"navigate to [path]"
await page.goto('http://localhost:${TARGET_PORT}/path');
"hover over [selector]"
await page.hover('.tooltip-trigger');
"scroll to [selector]"
await page.locator('[data-testid=footer]').scrollIntoViewIfNeeded();
Selector precedence (most specific to least specific):
  1. [data-testid=...]
    — preferred, most stable
  2. #id
    — unique element ID
  3. .class-name
    — CSS class
  4. [name=...]
    — form element name
  5. text=...
    — visible text content (fallback)
5. Assert visible states
For each expected visible state, generate the corresponding assertion inside the temporary script:
AC Description PatternPlaywright Assertion
"page shows [text]"
await expect(page.locator('body')).toContainText('text');
"[selector] has text [value]"
await expect(page.locator('[data-testid=title]')).toHaveText('value');
"[selector] contains [text]"
await expect(page.locator('.message')).toContainText('text');
"table has [N] rows"
expect(await page.locator('table tbody tr').count()).toBe(N);
"URL is [path]"
expect(page.url()).toBe('http://localhost:${TARGET_PORT}/path');
"URL contains [fragment]"
expect(page.url()).toContain('/fragment');
"[selector] is visible"
await expect(page.locator('[data-testid=modal]')).toBeVisible();
"[selector] is hidden"
await expect(page.locator('[data-testid=spinner]')).toBeHidden();
6. Execute test with timeout enforcement
Each Playwright test MUST enforce a per-test timeout to prevent indefinite hangs (REQ-NR007). The wrapper script uses the
timeout
command:
bash
TEST_TIMEOUT_SEC=30

npx playwright --version >/dev/null 2>&1 || {
    echo "Playwright not installed; skipping SPA tests."
    record_spa_manual_check
    continue
}
前提检查:在生成任何SPA测试之前,验证Playwright已安装:
bash
PLAYWRIGHT_VERSION=$(npx playwright --version 2>/dev/null || echo "")
if [ -z "$PLAYWRIGHT_VERSION" ]; then
    echo "ERROR: Playwright is not installed."
    echo "Install instructions:"
    echo "  npm install -D @playwright/test"
    echo "  npx playwright install chromium"
    echo ""
    echo "Skipping all Web SPA tests. Acceptance criteria for SPA interactions will be marked MANUAL CHECK REQUIRED."
    for ac_id in $(get_spa_ac_ids); do
        record_result "$ac_id" "MANUAL CHECK REQUIRED" "Playwright not installed"
    done
    # 继续下一个测试类别;不终止整个验证流程
fi
echo "Playwright version: $PLAYWRIGHT_VERSION"
如果缺少Playwright,技能必须报告该问题并提供上述安装命令,然后跳过SPA测试。不尝试自动安装。
1. 准备工件目录
bash
ARTIFACT_DIR="${SPEC_FOLDER}/e2e-artifacts"
mkdir -p "$ARTIFACT_DIR"
2. 解析
[IMP]
验收标准以获取UI行为提示
对于规范中每个与Web SPA行为相关的
[IMP]
验收标准:
  • 提取UI交互关键字:
    click
    fill
    type
    select
    submit
    navigate
    scroll
    hover
  • 通过搜索以下模式提取目标选择器:
    • data-testid="..."
      data-testid='...'
      [data-testid=...]
    • id="..."
      id='...'
      #...
    • class="..."
      class='...'
      .class-name
      (将空格替换为点)
    • 按钮/链接文本提及 →
      text=...
    • URL路径提及 →
      /path
  • 提取预期可见状态:文本内容预期、URL预期、元素存在/不存在、计数预期。
  • 提取表单字段名称和预期输入值。
如果
[IMP]
验收标准包含可解析的UI行为或可见状态提示,则标记为
MANUAL CHECK REQUIRED
并跳过下一个标准。
3. 启动无头浏览器上下文
默认情况下,浏览器必须以无头模式运行。仅当用户明确传递
--headed
时才使用有头模式。
对于每个SPA测试,生成临时Playwright脚本并使用
node
执行:
bash
DEV_SERVER_URL="http://localhost:${TARGET_PORT}"
TEST_SCRIPT="$(mktemp /tmp/e2e-spa-XXXXXX.js)"

cat > "$TEST_SCRIPT" << 'PLAYWRIGHT_EOF'
const { chromium } = require('playwright');
(async () => {
  const browser = await chromium.launch({ headless: true });
  const context = await browser.newContext({
    viewport: { width: 1280, height: 720 },
    userAgent: 'DeveloperKit-E2E/1.0'
  });
  const page = await context.newPage();
  // 此处注入操作和断言
  await browser.close();
})();
PLAYWRIGHT_EOF

node "$TEST_SCRIPT"
4. 将验收标准转换为Playwright操作
对于每个解析的UI交互,在临时脚本中生成相应的Playwright操作:
验收标准描述模式Playwright操作
"click [selector]"
await page.click('[data-testid=refresh]');
"fill [selector] with [value]"
await page.fill('#username', 'testuser');
"type [value] into [selector]"
await page.type('input[name=search]', 'query');
"select [value] in [selector]"
await page.selectOption('select[name=country]', 'US');
"submit [form]"
await page.click('button[type=submit]');
"navigate to [path]"
await page.goto('http://localhost:${TARGET_PORT}/path');
"hover over [selector]"
await page.hover('.tooltip-trigger');
"scroll to [selector]"
await page.locator('[data-testid=footer]').scrollIntoViewIfNeeded();
选择器优先级(从最具体到最不具体):
  1. [data-testid=...]
    — 首选,最稳定
  2. #id
    — 唯一元素ID
  3. .class-name
    — CSS类
  4. [name=...]
    — 表单元素名称
  5. text=...
    — 可见文本内容(回退)
5. 断言可见状态
对于每个预期可见状态,在临时脚本中生成相应的断言:
验收标准描述模式Playwright断言
"page shows [text]"
await expect(page.locator('body')).toContainText('text');
"[selector] has text [value]"
await expect(page.locator('[data-testid=title]')).toHaveText('value');
"[selector] contains [text]"
await expect(page.locator('.message')).toContainText('text');
"table has [N] rows"
expect(await page.locator('table tbody tr').count()).toBe(N);
"URL is [path]"
expect(page.url()).toBe('http://localhost:${TARGET_PORT}/path');
"URL contains [fragment]"
expect(page.url()).toContain('/fragment');
"[selector] is visible"
await expect(page.locator('[data-testid=modal]')).toBeVisible();
"[selector] is hidden"
await expect(page.locator('[data-testid=spinner]')).toBeHidden();
6. 执行测试并强制执行超时
每个Playwright测试必须强制执行每个测试的超时,以防止无限挂起(REQ-NR007)。包装脚本使用
timeout
命令:
bash
TEST_TIMEOUT_SEC=30

npx playwright --version >/dev/null 2>&1 || {
    echo "Playwright not installed; skipping SPA tests."
    record_spa_manual_check
    continue
}

Build the inline test script

构建内联测试脚本

TEST_SCRIPT="$(mktemp /tmp/e2e-spa-XXXXXX.js)" cat > "$TEST_SCRIPT" << EOF const { chromium } = require('playwright'); (async () => { const browser = await chromium.launch({ headless: true }); const page = await browser.newPage(); page.setDefaultTimeout(${TEST_TIMEOUT_SEC}000); page.setDefaultNavigationTimeout(${TEST_TIMEOUT_SEC}000); try { await page.goto('${DEV_SERVER_URL}'); // --- GENERATED ACTIONS --- // --- GENERATED ASSERTIONS --- console.log('RESULT: PASS'); } catch (error) { console.error('RESULT: FAIL:', error.message); const screenshotPath = '${ARTIFACT_DIR}/screenshot-' + Date.now() + '-ac-${AC_ID}.png'; await page.screenshot({ path: screenshotPath, fullPage: true }).catch(() => {}); console.error('SCREENSHOT:', screenshotPath); process.exitCode = 1; } finally { await browser.close(); } })(); EOF
TEST_SCRIPT="$(mktemp /tmp/e2e-spa-XXXXXX.js)" cat > "$TEST_SCRIPT" << EOF const { chromium } = require('playwright'); (async () => { const browser = await chromium.launch({ headless: true }); const page = await browser.newPage(); page.setDefaultTimeout(${TEST_TIMEOUT_SEC}000); page.setDefaultNavigationTimeout(${TEST_TIMEOUT_SEC}000); try { await page.goto('${DEV_SERVER_URL}'); // --- 生成的操作 --- // --- 生成的断言 --- console.log('RESULT: PASS'); } catch (error) { console.error('RESULT: FAIL:', error.message); const screenshotPath = '${ARTIFACT_DIR}/screenshot-' + Date.now() + '-ac-${AC_ID}.png'; await page.screenshot({ path: screenshotPath, fullPage: true }).catch(() => {}); console.error('SCREENSHOT:', screenshotPath); process.exitCode = 1; } finally { await browser.close(); } })(); EOF

Run with timeout wrapper; if the test hangs, it is killed and marked FAILED

使用超时包装器运行;如果测试挂起,则终止并标记为FAILED

if timeout --signal=TERM $((TEST_TIMEOUT_SEC + 5)) node "$TEST_SCRIPT"; then STATUS="VERIFIED" else EXIT_CODE=$? if [ "$EXIT_CODE" -eq 124 ]; then echo "FAIL: Test hung and was terminated after ${TEST_TIMEOUT_SEC}s timeout" fi STATUS="FAILED" fi rm -f "$TEST_SCRIPT"

If the test process hangs beyond the timeout, the `timeout` command sends SIGTERM, the AC is marked `FAILED`, and the evidence records: "Test hung and was terminated after ${TEST_TIMEOUT_SEC}s".

**7. Screenshot capture on failure**

When any assertion or action fails:
1. Capture a full-page screenshot using `page.screenshot({ path: ..., fullPage: true })`.
2. Save to `${ARTIFACT_DIR}/screenshot-<timestamp>-ac-<AC_ID>.png`.
3. Record the screenshot path in the test result evidence.

When a test passes, screenshots are optional and only captured if `--capture-success` is passed.

**8. Record results**

For each SPA test, record:
- AC ID and text (truncated)
- Playwright actions executed
- Assertions performed
- Pass/fail status
- Screenshot path (on failure)
- Execution time
- Error message (on failure)

If any assertion fails, mark the AC as `FAILED` immediately. Do not retry.

**Edge cases — error handling**:
- **Playwright not installed**: Report clear error with `npm install -D @playwright/test` and `npx playwright install chromium` suggestion. Mark all SPA ACs as `MANUAL CHECK REQUIRED`. Continue with other test categories.
- **Dev server not running**: If `page.goto()` throws `net::ERR_CONNECTION_REFUSED` or similar, report: "Dev server not reachable at ${DEV_SERVER_URL}. Ensure the server is running before verification." Mark affected ACs as `FAILED`.
- **Browser launch failure**: If Chromium fails to launch (e.g., missing system dependencies), report the error and suggest `npx playwright install-deps chromium`. Mark affected ACs as `MANUAL CHECK REQUIRED`.
if timeout --signal=TERM $((TEST_TIMEOUT_SEC + 5)) node "$TEST_SCRIPT"; then STATUS="VERIFIED" else EXIT_CODE=$? if [ "$EXIT_CODE" -eq 124 ]; then echo "FAIL: Test hung and was terminated after ${TEST_TIMEOUT_SEC}s timeout" fi STATUS="FAILED" fi rm -f "$TEST_SCRIPT"

如果测试进程超时挂起,`timeout`命令发送SIGTERM,验收标准标记为`FAILED`,证据记录:“Test hung and was terminated after ${TEST_TIMEOUT_SEC}s”。

**7. 失败时捕获截图**

当任何断言或操作失败时:
1. 使用`page.screenshot({ path: ..., fullPage: true })`捕获全页截图。
2. 保存到`${ARTIFACT_DIR}/screenshot-<timestamp>-ac-<AC_ID>.png`。
3. 在测试结果证据中记录截图路径。

当测试通过时,仅在传递`--capture-success`时才可选捕获截图。

**8. 记录结果**

对于每个SPA测试,记录:
- 验收标准ID和文本(截断)
- 执行的Playwright操作
- 执行的断言
- 通过/失败状态
- 截图路径(失败时)
- 执行时间
- 错误消息(失败时)

如果任何断言失败,立即将验收标准标记为`FAILED`。不重试。

**边缘情况——错误处理**:
- **未安装Playwright**:报告清晰的错误,并建议执行`npm install -D @playwright/test`和`npx playwright install chromium`。将所有SPA验收标准标记为`MANUAL CHECK REQUIRED`。继续其他测试类别。
- **开发服务器未运行**:如果`page.goto()`抛出`net::ERR_CONNECTION_REFUSED`或类似错误,报告:“Dev server not reachable at ${DEV_SERVER_URL}. Ensure the server is running before verification.” 将受影响的验收标准标记为`FAILED`。
- **浏览器启动失败**:如果Chromium无法启动(例如:缺少系统依赖),报告错误并建议执行`npx playwright install-deps chromium`。将受影响的验收标准标记为`MANUAL CHECK REQUIRED`。

Desktop Tests (Computer-use / MCP)

桌面测试(Computer-use / MCP)

Prerequisite check: Before generating any desktop tests, verify that computer-use or MCP-based GUI automation tools are available. This skill does NOT auto-install these tools.
bash
GUI_TOOLS_AVAILABLE=false
GUI_TOOL_NAME=""

if [ -n "$CLAUDE_COMPUTER_USE_AVAILABLE" ] || command -v computer-use >/dev/null 2>&1; then
    GUI_TOOLS_AVAILABLE=true
    GUI_TOOL_NAME="computer-use"
fi

if [ -n "$MCP_GUI_SERVER_URL" ] || command -v mcp-gui >/dev/null 2>&1; then
    GUI_TOOLS_AVAILABLE=true
    GUI_TOOL_NAME="mcp-gui"
fi

if [ "$GUI_TOOLS_AVAILABLE" != "true" ]; then
    echo "ERROR: Desktop testing tools are not available."
    echo "This skill requires computer-use or MCP GUI automation tools to verify desktop applications."
    echo ""
    echo "To enable desktop verification:"
    echo "  - Run in an environment with computer-use support, OR"
    echo "  - Install and configure an MCP GUI automation server"
    echo ""
    echo "Aborting desktop verification. No tests will be executed."
    exit 1
fi
echo "Desktop GUI tool detected: $GUI_TOOL_NAME"
1. Prepare artifact directory
bash
ARTIFACT_DIR="${SPEC_FOLDER}/e2e-artifacts"
mkdir -p "$ARTIFACT_DIR"
2. Parse
[IMP]
AC for desktop behavior hints
For each
[IMP]
acceptance criterion in the specification that relates to Desktop App behavior:
  • Extract window title hints: look for phrases like "window opens", "dialog appears", "modal shows", "title contains", "screen shows".
  • Extract UI element descriptions: look for button labels, input placeholders, menu items, checkbox labels, toggle names.
  • Extract workflow steps: look for action sequences like "click ... then ...", "select ... and press ...", "toggle ...".
  • Extract expected state changes: look for visual changes like "theme changes to dark", "status indicator turns green", "list updates".
If an
[IMP]
AC does not contain parseable desktop behavior hints (no window, element, or workflow descriptions), mark it
MANUAL CHECK REQUIRED
and skip to the next criterion.
3. Verify the application is running
If the desktop app was not started in Phase 3 or its process has exited, re-launch using the commands from Phase 3:
bash
if [ -n "$STARTUP_PID" ] && ! kill -0 "$STARTUP_PID" 2>/dev/null; then
    echo "Desktop app process is not running. Re-launching..."
    # Re-run the appropriate launch command from Phase 3 based on DESKTOP_FRAMEWORK
fi
4. Verify window/UI elements via visual or accessibility-tree inspection
For each AC describing a window or UI element (AC-022):
  • Visual inspection (computer-use):
    • Capture a screenshot of the desktop.
    • Analyze the image for the expected window title, button label, or visual element.
    • Example: "Settings window opens" → capture screenshot, verify a window titled "Settings" is visible.
  • Accessibility-tree inspection (MCP GUI automation):
    • Query the accessibility tree for elements matching the expected name, role, or state.
    • Example: "Settings window opens" → query for a window element with name containing "Settings".
    • Example: "Toggle dark mode" → query for a switch/checkbox element with label containing "Dark mode".
5. Simulate user workflows through GUI automation
For each AC describing a user workflow (AC-023), translate the description into GUI automation actions:
AC Description PatternGUI Automation Action
"click [button label]"Click the UI element with the matching accessible name or label
"fill [field] with [value]"Focus the input field and type the value
"toggle [switch/checkbox]"Click the toggle or checkbox element
"select [option] from [dropdown]"Open the dropdown and click the option element
"navigate to [menu item]"Click the menu item with the matching label
"type [value] into [field]"Focus the field and type the value
6. Screenshot capture at each verification step
At EVERY step (before actions, after actions, and on assertions), capture a screenshot:
bash
SCREENSHOT_PATH="${ARTIFACT_DIR}/screenshot-$(date +%s)-ac-${AC_ID}-step-${STEP_NUM}.png"
前提检查:在生成任何桌面测试之前,验证computer-use或基于MCP的GUI自动化工具可用。该技能不会自动安装这些工具。
bash
GUI_TOOLS_AVAILABLE=false
GUI_TOOL_NAME=""

if [ -n "$CLAUDE_COMPUTER_USE_AVAILABLE" ] || command -v computer-use >/dev/null 2>&1; then
    GUI_TOOLS_AVAILABLE=true
    GUI_TOOL_NAME="computer-use"
fi

if [ -n "$MCP_GUI_SERVER_URL" ] || command -v mcp-gui >/dev/null 2>&1; then
    GUI_TOOLS_AVAILABLE=true
    GUI_TOOL_NAME="mcp-gui"
fi

if [ "$GUI_TOOLS_AVAILABLE" != "true" ]; then
    echo "ERROR: Desktop testing tools are not available."
    echo "This skill requires computer-use or MCP GUI automation tools to verify desktop applications."
    echo ""
    echo "To enable desktop verification:"
    echo "  - Run in an environment with computer-use support, OR"
    echo "  - Install and configure an MCP GUI automation server"
    echo ""
    echo "Aborting desktop verification. No tests will be executed."
    exit 1
fi
echo "Desktop GUI tool detected: $GUI_TOOL_NAME"
1. 准备工件目录
bash
ARTIFACT_DIR="${SPEC_FOLDER}/e2e-artifacts"
mkdir -p "$ARTIFACT_DIR"
2. 解析
[IMP]
验收标准以获取桌面行为提示
对于规范中每个与桌面应用行为相关的
[IMP]
验收标准:
  • 提取窗口标题提示:查找诸如“window opens”、“dialog appears”、“modal shows”、“title contains”、“screen shows”等短语。
  • 提取UI元素描述:查找按钮标签、输入占位符、菜单项、复选框标签、切换开关名称。
  • 提取工作流步骤:查找诸如“click ... then ...”、“select ... and press ...”、“toggle ...”等操作序列。
  • 提取预期状态变化:查找诸如“theme changes to dark”、“status indicator turns green”、“list updates”等视觉变化。
如果
[IMP]
验收标准包含可解析的桌面行为提示(无窗口、元素或工作流描述),则标记为
MANUAL CHECK REQUIRED
并跳过下一个标准。
3. 验证应用正在运行
如果桌面应用未在阶段3启动或其进程已退出,则使用阶段3中的命令重新启动:
bash
if [ -n "$STARTUP_PID" ] && ! kill -0 "$STARTUP_PID" 2>/dev/null; then
    echo "Desktop app process is not running. Re-launching..."
    # 根据DESKTOP_FRAMEWORK重新运行阶段3中的相应启动命令
fi
4. 通过视觉或可访问性树检查验证窗口/UI元素
对于每个描述窗口或UI元素的验收标准(AC-022):
  • 视觉检查(computer-use):
    • 捕获桌面截图。
    • 分析图像以查找预期的窗口标题、按钮标签或视觉元素。
    • 示例:“Settings window opens” → 捕获截图,验证标题为“Settings”的窗口可见。
  • 可访问性树检查(MCP GUI自动化):
    • 查询可访问性树以查找与预期名称、角色或状态匹配的元素。
    • 示例:“Settings window opens” → 查询名称包含“Settings”的窗口元素。
    • 示例:“Toggle dark mode” → 查询标签包含“Dark mode”的开关/复选框元素。
5. 通过GUI自动化模拟用户工作流
对于每个描述用户工作流的验收标准(AC-023),将描述转换为GUI自动化操作:
验收标准描述模式GUI自动化操作
"click [button label]"点击具有匹配可访问名称或标签的UI元素
"fill [field] with [value]"聚焦输入字段并输入值
"toggle [switch/checkbox]"点击切换开关或复选框元素
"select [option] from [dropdown]"打开下拉菜单并点击选项元素
"navigate to [menu item]"点击具有匹配标签的菜单项
"type [value] into [field]"聚焦字段并输入值
6. 在每个验证步骤捕获截图
每个步骤(操作前、操作后、断言时)捕获截图:
bash
SCREENSHOT_PATH="${ARTIFACT_DIR}/screenshot-$(date +%s)-ac-${AC_ID}-step-${STEP_NUM}.png"

computer-use: capture screenshot and save to SCREENSHOT_PATH

computer-use: 捕获截图并保存到SCREENSHOT_PATH

MCP: use the GUI automation server's screenshot capability

MCP: 使用GUI自动化服务器的截图功能


Screenshots are saved for every step regardless of pass/fail status, satisfying AC-024 (SEF).

**7. Per-test timeout enforcement**

Each desktop test MUST enforce a per-test timeout to prevent indefinite hangs (REQ-NR007):

```bash
TEST_TIMEOUT_SEC=60
TEST_START_TIME=$(date +%s)
TEST_STATUS="RUNNING"

while [ "$TEST_STATUS" = "RUNNING" ]; do
    if [ $(( $(date +%s) - TEST_START_TIME )) -gt "$TEST_TIMEOUT_SEC" ]; then
        echo "FAIL: Desktop test hung and was terminated after ${TEST_TIMEOUT_SEC}s timeout"
        STATUS="FAILED"
        TEST_STATUS="TIMEOUT"
        FINAL_SCREENSHOT="${ARTIFACT_DIR}/screenshot-$(date +%s)-ac-${AC_ID}-timeout.png"
        # capture screenshot to FINAL_SCREENSHOT
        break
    fi
    # Execute next workflow step; if all steps complete, set TEST_STATUS="COMPLETED"
done
If a test times out:
  • Mark the AC as
    FAILED
  • Capture a final timeout screenshot
  • Record evidence: "Test hung and was terminated after ${TEST_TIMEOUT_SEC}s timeout"
  • Attempt to reset application state (close and reopen the window) before the next AC
8. Record results
For each desktop test, record:
  • AC ID and text (truncated)
  • GUI automation actions executed (step list)
  • Screenshots captured (list of paths with step descriptions)
  • Pass/fail status
  • Error message, timeout reason, or assertion mismatch (on failure)
If any assertion or action fails, mark the AC as
FAILED
immediately. Do not retry.
Edge cases — error handling:
  • No computer-use or MCP tools available: Abort verification with a clear error message. Do NOT silently skip desktop tests.
  • Build failure: Build output is captured in
    $STARTUP_LOGS_FILE
    and reported. Verification aborts before launch.
  • App crash during test: If the desktop process exits during testing, capture any crash output from logs, mark the current AC as
    FAILED
    , and attempt to restart the app for the next AC.
  • Window/element not found: If the expected window or element does not appear within the test timeout, mark the AC as
    FAILED
    and capture a screenshot of the current desktop state.
Fallback: If a specific AC cannot be translated into an automated test (e.g., it requires human aesthetic judgment), mark it as
MANUAL CHECK REQUIRED
and continue.

无论通过或失败,每个步骤都保存截图,满足AC-024(SEF)要求。

**7. 强制执行每个测试的超时**

每个桌面测试必须强制执行每个测试的超时,以防止无限挂起(REQ-NR007):

```bash
TEST_TIMEOUT_SEC=60
TEST_START_TIME=$(date +%s)
TEST_STATUS="RUNNING"

while [ "$TEST_STATUS" = "RUNNING" ]; do
    if [ $(( $(date +%s) - TEST_START_TIME )) -gt "$TEST_TIMEOUT_SEC" ]; then
        echo "FAIL: Desktop test hung and was terminated after ${TEST_TIMEOUT_SEC}s timeout"
        STATUS="FAILED"
        TEST_STATUS="TIMEOUT"
        FINAL_SCREENSHOT="${ARTIFACT_DIR}/screenshot-$(date +%s)-ac-${AC_ID}-timeout.png"
        # 捕获截图到FINAL_SCREENSHOT
        break
    fi
    # 执行下一个工作流步骤;如果所有步骤完成,设置TEST_STATUS="COMPLETED"
done
如果测试超时:
  • 将验收标准标记为
    FAILED
  • 捕获最终超时截图
  • 记录证据:“Test hung and was terminated after ${TEST_TIMEOUT_SEC}s timeout”
  • 在处理下一个验收标准之前,尝试重置应用状态(关闭并重新打开窗口)
8. 记录结果
对于每个桌面测试,记录:
  • 验收标准ID和文本(截断)
  • 执行的GUI自动化操作(步骤列表)
  • 捕获的截图(路径列表及步骤描述)
  • 通过/失败状态
  • 错误消息、超时原因或断言不匹配(失败时)
如果任何断言或操作失败,立即将验收标准标记为
FAILED
。不重试。
边缘情况——错误处理
  • 无computer-use或MCP工具可用:以清晰的错误消息终止验证。不静默跳过桌面测试。
  • 构建失败:构建输出捕获在
    $STARTUP_LOGS_FILE
    中并报告。验证在启动前终止。
  • 测试期间应用崩溃:如果桌面进程在测试期间退出,捕获日志中的崩溃输出,将当前验收标准标记为
    FAILED
    ,并尝试为下一个验收标准重新启动应用。
  • 未找到窗口/元素:如果预期窗口或元素在测试超时内未出现,将验收标准标记为
    FAILED
    并捕获当前桌面状态的截图。
回退方案:如果特定验收标准无法转换为自动化测试(例如:需要人工审美判断),则标记为
MANUAL CHECK REQUIRED
并继续。

Phase 5: Map Results to Acceptance Criteria

阶段5:将结果映射到验收标准

Goal: Compare every test execution result against the
[IMP]
acceptance criteria from the functional specification and produce a deterministic verdict for each.
目标:将每个测试执行结果与功能规范中的
[IMP]
验收标准进行比较,并为每个标准生成确定性结论。

5.1 Initialize the Results Table

5.1 初始化结果表

Create an associative results structure (e.g., shell associative array, JSON object, or temporary file) keyed by AC ID. For every
[IMP]
acceptance criterion extracted in Phase 1, pre-populate a row with:
FieldInitial Value
ac_id
The AC identifier (e.g.,
AC-012
)
ac_text
The full criterion text, truncated to 120 characters for display
status
PENDING
(updated in 5.2)
evidence
Empty string (updated in 5.2)
actual
Empty string (populated only on mismatch)
expected
Empty string (populated only on mismatch)
Iterate over the specification’s acceptance criteria table and include only rows whose taxonomy tag is
[IMP]
.
[SEF]
and
[EXT]
criteria are excluded from runtime verification; they may be listed in an appendix but do not require a status verdict.
创建关联结果结构(例如:shell关联数组、JSON对象或临时文件),以验收标准ID为键。对于阶段1中提取的每个
[IMP]
验收标准,预填充一行:
字段初始值
ac_id
验收标准标识符(例如:
AC-012
ac_text
完整标准文本,显示时截断为120个字符
status
PENDING
(在5.2中更新)
evidence
空字符串(在5.2中更新)
actual
空字符串(仅在不匹配时填充)
expected
空字符串(仅在不匹配时填充)
遍历规范的验收标准表,包含分类标签为
[IMP]
的行。
[SEF]
[EXT]
标准不包含在运行时验证中;它们可能列在附录中,但不需要状态结论。

5.2 Apply Verdict Rules

5.2 应用结论规则

For each AC that was targeted by a test in Phase 4, apply the following rules in order:
  1. VERIFIED (REQ-016):
    • IF the test completed without assertion failures, timeouts, or tool errors
    • THEN set
      status = "VERIFIED"
    • Set
      evidence
      to a concise description:
      • REST API:
        curl returned HTTP <code> in <N>ms
      • Web SPA:
        Playwright assertion passed: <selector> <condition>
      • Desktop:
        GUI automation confirmed: <window/element> present
  2. FAILED (REQ-018):
    • IF any assertion failed, the test timed out, the tool crashed, or the dev server was unreachable
    • THEN set
      status = "FAILED"
    • Set
      actual
      to the observed value (e.g.,
      HTTP 400
      ,
      element not found
      ,
      timeout after 30s
      )
    • Set
      expected
      to the value demanded by the AC (e.g.,
      HTTP 201
      ,
      element visible
      )
    • Set
      evidence
      to a human-readable sentence combining
      actual
      vs
      expected
      , plus the path to any captured artifact (screenshot, response dump) if available.
  3. MANUAL CHECK REQUIRED (REQ-017):
    • IF no test was generated for the AC because:
      • The AC text lacked parseable endpoint / UI / desktop behavior hints
      • A required tool was missing (e.g., Playwright not installed)
      • The AC describes human judgment (e.g., "UI looks correct")
    • THEN set
      status = "MANUAL CHECK REQUIRED"
    • Set
      evidence
      to the reason:
      No automated test could be derived: <reason>
IMPORTANT: Every
[IMP]
AC MUST have a final status of
VERIFIED
,
FAILED
, or
MANUAL CHECK REQUIRED
. No AC may remain in
PENDING
when Phase 5 ends.
对于阶段4中测试覆盖的每个验收标准,按顺序应用以下规则:
  1. VERIFIED(REQ-016):
    • 如果测试完成且无断言失败、超时或工具错误
    • 则设置
      status = "VERIFIED"
    • 设置
      evidence
      为简洁描述:
      • REST API:
        curl returned HTTP <code> in <N>ms
      • Web SPA:
        Playwright assertion passed: <selector> <condition>
      • 桌面:
        GUI automation confirmed: <window/element> present
  2. FAILED(REQ-018):
    • 如果任何断言失败、测试超时、工具崩溃或开发服务器不可达
    • 则设置
      status = "FAILED"
    • 设置
      actual
      为观察到的值(例如:
      HTTP 400
      element not found
      timeout after 30s
    • 设置
      expected
      为验收标准要求的值(例如:
      HTTP 201
      element visible
    • 设置
      evidence
      为结合
      actual
      expected
      的人类可读句子,如果有可用的捕获工件(截图、响应转储),则添加其路径。
  3. MANUAL CHECK REQUIRED(REQ-017):
    • 如果未为验收标准生成测试,原因包括:
      • 验收标准文本缺少可解析的端点/UI/桌面行为提示
      • 缺少所需工具(例如:未安装Playwright)
      • 验收标准描述了人工判断(例如:“UI looks correct”)
    • 则设置
      status = "MANUAL CHECK REQUIRED"
    • 设置
      evidence
      为原因:
      No automated test could be derived: <reason>
重要提示:每个
[IMP]
验收标准必须具有
VERIFIED
FAILED
MANUAL CHECK REQUIRED
的最终状态。阶段5结束时,不得有验收标准处于
PENDING
状态。

5.3 Compute Summary Counts

5.3 计算汇总统计

After all rows are populated, compute:
bash
TOTAL_IMP=$(count_implementation_acs)
VERIFIED_COUNT=$(grep -c '"status":"VERIFIED"' "$RESULTS_FILE")
FAILED_COUNT=$(grep -c '"status":"FAILED"' "$RESULTS_FILE")
MANUAL_COUNT=$(grep -c '"status":"MANUAL CHECK REQUIRED"' "$RESULTS_FILE")
Store these counts; they are required in the report Summary section (REQ-020).
填充所有行后,计算:
bash
TOTAL_IMP=$(count_implementation_acs)
VERIFIED_COUNT=$(grep -c '"status":"VERIFIED"' "$RESULTS_FILE")
FAILED_COUNT=$(grep -c '"status":"FAILED"' "$RESULTS_FILE")
MANUAL_COUNT=$(grep -c '"status":"MANUAL CHECK REQUIRED"' "$RESULTS_FILE")
存储这些统计数据;它们是报告摘要部分(REQ-020)所需的。

Phase 6: Generate Report

阶段6:生成报告

Goal: Produce a deterministic, human-readable markdown report that follows the format defined in Phase 6.3 and contains no secrets.
目标:生成符合阶段6.3定义格式的确定性、人类可读Markdown报告,且不包含机密信息。

6.1 Prepare Report Paths and Directories

6.1 准备报告路径和目录

bash
REPORT_TIMESTAMP=$(date +%Y-%m-%d-%H%M%S)
REPORT_DIR="${SPEC_FOLDER}"
ARTIFACT_DIR="${REPORT_DIR}/e2e-artifacts"
REPORT_FILE="${REPORT_DIR}/e2e-report-${REPORT_TIMESTAMP}.md"

mkdir -p "$ARTIFACT_DIR"
  • SPEC_FOLDER
    is the value of the
    --spec
    argument (e.g.,
    docs/specs/001-real-e2e-verification/
    ).
  • The report filename MUST use the exact pattern
    e2e-report-YYYY-MM-DD-HHMMSS.md
    (AC-026).
  • The artifact directory MUST exist before any artifact paths are written into the report.
bash
REPORT_TIMESTAMP=$(date +%Y-%m-%d-%H%M%S)
REPORT_DIR="${SPEC_FOLDER}"
ARTIFACT_DIR="${REPORT_DIR}/e2e-artifacts"
REPORT_FILE="${REPORT_DIR}/e2e-report-${REPORT_TIMESTAMP}.md"

mkdir -p "$ARTIFACT_DIR"
  • SPEC_FOLDER
    --spec
    参数的值(例如:
    docs/specs/001-real-e2e-verification/
    )。
  • 报告文件名必须使用精确模式
    e2e-report-YYYY-MM-DD-HHMMSS.md
    (AC-026)。
  • 在将任何工件路径写入报告之前,必须确保工件目录存在。

6.2 Redact Secrets from Raw Output

6.2 从原始输出中编辑机密信息

Before writing any command output into the report, run the raw logs through a redaction pass (REQ-NR002):
bash
undefined
在将任何命令输出写入报告之前,对原始日志进行编辑处理(REQ-NR002):
bash
undefined

Redact Authorization header values

编辑Authorization标头值

sed -E 's/(Authorization:[[:space:]][Bb]earer[[:space:]]+)[^[:space:]]+/\1REDACTED*/g' "$STARTUP_LOGS_FILE" > "$STARTUP_LOGS_FILE.redacted"
sed -E 's/(Authorization:[[:space:]][Bb]earer[[:space:]]+)[^[:space:]]+/\1REDACTED*/g' "$STARTUP_LOGS_FILE" > "$STARTUP_LOGS_FILE.redacted"

Redact tokens in JSON bodies

编辑JSON体中的令牌

sed -E 's/("token"[[:space:]]:[[:space:]]")[^"]+/\1REDACTED/g' "$STARTUP_LOGS_FILE.redacted" > "$STARTUP_LOGS_FILE.redacted2"
sed -E 's/("token"[[:space:]]:[[:space:]]")[^"]+/\1REDACTED/g' "$STARTUP_LOGS_FILE.redacted" > "$STARTUP_LOGS_FILE.redacted2"

Redact passwords in curl -u arguments

编辑curl -u参数中的密码

sed -E 's/(-u[[:space:]]+[^:]:)[^[:space:]]+/\1REDACTED*/g' "$STARTUP_LOGS_FILE.redacted2" > "$STARTUP_LOGS_FILE.redacted3"
sed -E 's/(-u[[:space:]]+[^:]:)[^[:space:]]+/\1REDACTED*/g' "$STARTUP_LOGS_FILE.redacted2" > "$STARTUP_LOGS_FILE.redacted3"

Redact E2E_* environment variables (REQ-NR002)

编辑E2E_*环境变量(REQ-NR002)

sed -E 's/(E2E_[A-Z_]+=)[^[:space:]]+/\1REDACTED/g' "$STARTUP_LOGS_FILE.redacted3" > "$STARTUP_LOGS_FILE.redacted_final"

Only the redacted version (`$STARTUP_LOGS_FILE.redacted_final`) is included in the report. The original temp file is discarded.
sed -E 's/(E2E_[A-Z_]+=)[^[:space:]]+/\1REDACTED/g' "$STARTUP_LOGS_FILE.redacted3" > "$STARTUP_LOGS_FILE.redacted_final"

仅将编辑后的版本(`$STARTUP_LOGS_FILE.redacted_final`)包含在报告中。原始临时文件将被丢弃。

6.3 Write Report Sections

6.3 编写报告章节

Generate the report by appending each section in the exact order below. All paths inside the report MUST be relative to the report file location.
Section 1 — Summary
markdown
undefined
按以下精确顺序追加每个章节以生成报告。报告中的所有路径必须相对于报告文件位置。
章节1 — 摘要
markdown
undefined

Summary

摘要

MetricCount
Total AC Evaluated${TOTAL_IMP}
✅ VERIFIED${VERIFIED_COUNT}
❌ FAILED${FAILED_COUNT}
⚠️ MANUAL CHECK REQUIRED${MANUAL_COUNT}

**Section 2 — Environment**

```markdown
指标数量
评估的验收标准总数${TOTAL_IMP}
✅ VERIFIED${VERIFIED_COUNT}
❌ FAILED${FAILED_COUNT}
⚠️ MANUAL CHECK REQUIRED${MANUAL_COUNT}

**章节2 — 环境**

```markdown

Environment

环境

PropertyValue
Application Type${APP_TYPE}
Startup Command`${STARTUP_COMMAND}`
Target Port${TARGET_PORT}
Health Endpoint${HEALTH_CHECK_METHOD}
Runtime Version${RUNTIME_VERSION}
Verification Started${ISO8601_START_TIME}

- `RUNTIME_VERSION`: capture the runtime version detected during startup:
  - Docker: `docker --version`
  - Node.js: `node --version`
  - Java: `java -version 2>&1 | head -1`
  - Python: `python --version` or `python3 --version`
- `ISO8601_START_TIME`: the timestamp when Phase 3 began, in ISO-8601 format.

**Section 3 — Test Results**

```markdown
属性
应用类型${APP_TYPE}
启动命令`${STARTUP_COMMAND}`
目标端口${TARGET_PORT}
健康检查方式${HEALTH_CHECK_METHOD}
运行时版本${RUNTIME_VERSION}
验证开始时间${ISO8601_START_TIME}

- `RUNTIME_VERSION`:捕获启动期间检测到的运行时版本:
  - Docker:`docker --version`
  - Node.js:`node --version`
  - Java:`java -version 2>&1 | head -1`
  - Python:`python --version`或`python3 --version`
- `ISO8601_START_TIME`:阶段3开始的时间戳,采用ISO-8601格式。

**章节3 — 测试结果**

```markdown

Test Results

测试结果

AC IDCriterion (truncated)StatusEvidence
AC-012WHEN the app is classified...VERIFIED`curl` returned 200
AC-013WHEN a curl test executes...FAILEDExpected 201, got 400
AC-014WHEN authentication is required...MANUAL CHECK REQUIREDNo credentials found

- The "Criterion (truncated)" column MUST be truncated to 120 characters, appended with `...` if longer.
- The "Status" column MUST be one of `VERIFIED`, `FAILED`, or `MANUAL CHECK REQUIRED`.
- The "Evidence" column MUST reference artifacts using relative paths when applicable (e.g., `./e2e-artifacts/ac-019-screenshot.png`).

**Section 4 — Raw Output**

```markdown
验收标准ID标准(截断)状态证据
AC-012WHEN the app is classified...VERIFIED`curl` returned 200
AC-013WHEN a curl test executes...FAILEDExpected 201, got 400
AC-014WHEN authentication is required...MANUAL CHECK REQUIREDNo credentials found

- “标准(截断)”列必须截断为120个字符,如果更长则追加`...`。
- “状态”列必须是`VERIFIED`、`FAILED`或`MANUAL CHECK REQUIRED`之一。
- “证据”列必须在适用时使用相对路径引用工件(例如:`./e2e-artifacts/ac-019-screenshot.png`)。

**章节4 — 原始输出**

```markdown

Raw Output

原始输出

Startup Logs

启动日志

``` [contents of $STARTUP_LOGS_FILE.redacted_final] ```
``` [contents of $STARTUP_LOGS_FILE.redacted_final] ```

Test Commands

测试命令

``` [For each test: the exact command executed (auth redacted) and its first 2000 chars of stdout/stderr] ```

- If a test produced no output, write `(no output)`.
- Limit each code block to 2000 lines to avoid overwhelming the report; if logs are larger, append a note: ``(truncated; full logs available in <path>)``.

**Section 5 — Artifacts**

```markdown
``` [每个测试:执行的确切命令(已编辑认证信息)及其stdout/stderr的前2000个字符] ```

- 如果测试未产生输出,写入`(no output)`。
- 将每个代码块限制为2000行,避免报告过大;如果日志更大,追加注释:``(truncated; full logs available in <path>)``。

**章节5 — 工件**

```markdown

Artifacts

工件

AC IDTypePath
AC-019Screenshot./e2e-artifacts/screenshot-1717189200-ac-019.png
AC-014Response Dump./e2e-artifacts/ac-014-response.json

- List every artifact file present in `$ARTIFACT_DIR`.
- If no artifacts were generated, write ``No artifacts captured for this run.``

**Section 6 — Teardown Status**

```markdown
验收标准ID类型路径
AC-019截图./e2e-artifacts/screenshot-1717189200-ac-019.png
AC-014响应转储./e2e-artifacts/ac-014-response.json

- 列出`$ARTIFACT_DIR`中存在的每个工件文件。
- 如果未生成任何工件,写入``No artifacts captured for this run.``

**章节6 — 清理状态**

```markdown

Teardown Status

清理状态

PropertyValue
Teardown Executed${TEARDOWN_EXECUTED}
Port Released${PORT_RELEASED}
Remaining Processes${REMAINING_PROCESSES}

- `TEARDOWN_EXECUTED`: `true` / `false` / `skipped (--keep-alive)`
- `PORT_RELEASED`: `true` / `false` / `unknown`
- `REMAINING_PROCESSES`: A comma-separated list of PIDs or container names still active after teardown, or `none`.
属性
已执行清理${TEARDOWN_EXECUTED}
端口已释放${PORT_RELEASED}
剩余进程${REMAINING_PROCESSES}

- `TEARDOWN_EXECUTED`:`true` / `false` / `skipped (--keep-alive)`
- `PORT_RELEASED`:`true` / `false` / `unknown`
- `REMAINING_PROCESSES`:清理后仍活跃的PID或容器名称的逗号分隔列表,或`none`。

6.4 Finalize and Persist Report

6.4 最终确定并保存报告

bash
cat > "$REPORT_FILE" << 'REPORT_EOF'
[all sections assembled above]
REPORT_EOF

echo "E2E report saved to: $REPORT_FILE"
  • The report MUST be valid Markdown (passes basic
    markdownlint
    rules): proper heading levels, no trailing spaces, consistent pipe table delimiters.
  • The report MUST NOT be modified after it is written (REQ-NR005).
  • Set a variable
    REPORT_PATH="$REPORT_FILE"
    for use in Phase 8.
bash
cat > "$REPORT_FILE" << 'REPORT_EOF'
[上述所有章节组合内容]
REPORT_EOF

echo "E2E report saved to: $REPORT_FILE"
  • 报告必须是有效的Markdown(通过基本
    markdownlint
    规则):正确的标题级别、无尾随空格、一致的管道表分隔符。
  • 报告写入后不得修改(REQ-NR005)。
  • 设置变量
    REPORT_PATH="$REPORT_FILE"
    供阶段8使用。

Phase 7: Teardown Environment

阶段7:清理环境

Goal: Gracefully stop all runtime resources started in Phase 3, verify that ports and processes are fully released, and record the outcome. If
--keep-alive
is passed, skip teardown entirely and warn the user.
目标:优雅停止阶段3启动的所有运行时资源,验证端口和进程已完全释放,并记录结果。如果传递了
--keep-alive
,则完全跳过清理步骤并警告用户。

7.1
--keep-alive
Guard

7.1
--keep-alive
防护

bash
if [ "$KEEP_ALIVE" = "true" ]; then
    TEARDOWN_EXECUTED="skipped (--keep-alive)"
    PORT_RELEASED="skipped"
    REMAINING_PROCESSES="skipped"
    echo "WARNING: --keep-alive was passed. The runtime environment is still running."
    echo "You are responsible for cleaning up:"
    if [ "$APP_TYPE" = "Docker-managed" ]; then
        echo "  docker compose -f '${PROJECT_ROOT}/docker-compose.yml' down"
    else
        echo "  kill $(lsof -ti:${TARGET_PORT})"
    fi
    echo "Report was saved before teardown; artifacts are in ${ARTIFACT_DIR}."
    # Skip remaining teardown steps and proceed to Phase 8
    # (Phase 7 cleanup is bypassed when --keep-alive is active)
fi
bash
if [ "$KEEP_ALIVE" = "true" ]; then
    TEARDOWN_EXECUTED="skipped (--keep-alive)"
    PORT_RELEASED="skipped"
    REMAINING_PROCESSES="skipped"
    echo "WARNING: --keep-alive was passed. The runtime environment is still running."
    echo "You are responsible for cleaning up:"
    if [ "$APP_TYPE" = "Docker-managed" ]; then
        echo "  docker compose -f '${PROJECT_ROOT}/docker-compose.yml' down"
    else
        echo "  kill $(lsof -ti:${TARGET_PORT})"
    fi
    echo "Report was saved before teardown; artifacts are in ${ARTIFACT_DIR}."
    # 跳过剩余清理步骤并进入阶段8
    # (当--keep-alive激活时,跳过阶段7清理)
fi

7.2 Graceful Teardown by Application Type

7.2 按应用类型优雅清理

Docker-managed (AC-029):
bash
TEARDOWN_EXECUTED="true"
DOCKER_TEARDOWN_FAILED=false

if [ "$APP_TYPE" = "Docker-managed" ]; then
    cd "$PROJECT_ROOT"
    if docker compose down >> "$STARTUP_LOGS_FILE" 2>&1; then
        echo "Docker Compose stack stopped successfully."
    else
        echo "WARNING: docker compose down exited with a non-zero code."
        DOCKER_TEARDOWN_FAILED=true
    fi
fi
Local processes (JVM, Node.js/NestJS, Python, Desktop) (AC-029):
bash
if [ -n "$STARTUP_PID" ] && kill -0 "$STARTUP_PID" 2>/dev/null; then
    echo "Sending SIGTERM to process $STARTUP_PID ..."
    kill -TERM "$STARTUP_PID" 2>/dev/null || true
    # Wait up to 10 seconds for graceful exit
    GRACEFUL_WAIT=0
    while [ "$GRACEFUL_WAIT" -lt 10 ] && kill -0 "$STARTUP_PID" 2>/dev/null; do
        sleep 1
        GRACEFUL_WAIT=$((GRACEFUL_WAIT + 1))
    done
    # If still alive, send SIGKILL
    if kill -0 "$STARTUP_PID" 2>/dev/null; then
        echo "Process $STARTUP_PID did not exit after 10s; sending SIGKILL."
        kill -KILL "$STARTUP_PID" 2>/dev/null || true
        sleep 2
    fi
fi
Docker管理(AC-029):
bash
TEARDOWN_EXECUTED="true"
DOCKER_TEARDOWN_FAILED=false

if [ "$APP_TYPE" = "Docker-managed" ]; then
    cd "$PROJECT_ROOT"
    if docker compose down >> "$STARTUP_LOGS_FILE" 2>&1; then
        echo "Docker Compose stack stopped successfully."
    else
        echo "WARNING: docker compose down exited with a non-zero code."
        DOCKER_TEARDOWN_FAILED=true
    fi
fi
本地进程(JVM、Node.js/NestJS、Python、桌面)(AC-029):
bash
if [ -n "$STARTUP_PID" ] && kill -0 "$STARTUP_PID" 2>/dev/null; then
    echo "Sending SIGTERM to process $STARTUP_PID ..."
    kill -TERM "$STARTUP_PID" 2>/dev/null || true
    # 等待最多10秒优雅退出
    GRACEFUL_WAIT=0
    while [ "$GRACEFUL_WAIT" -lt 10 ] && kill -0 "$STARTUP_PID" 2>/dev/null; do
        sleep 1
        GRACEFUL_WAIT=$((GRACEFUL_WAIT + 1))
    done
    # 如果仍活跃,发送SIGKILL
    if kill -0 "$STARTUP_PID" 2>/dev/null; then
        echo "Process $STARTUP_PID did not exit after 10s; sending SIGKILL."
        kill -KILL "$STARTUP_PID" 2>/dev/null || true
        sleep 2
    fi
fi

Also terminate any child processes that may still hold the port

同时终止任何可能仍占用端口的子进程

PORT_PIDS=$(lsof -ti:"$TARGET_PORT" 2>/dev/null || true) if [ -n "$PORT_PIDS" ]; then echo "Additional PIDs holding port $TARGET_PORT: $PORT_PIDS" for pid in $PORT_PIDS; do kill -TERM "$pid" 2>/dev/null || true done sleep 3 for pid in $PORT_PIDS; do if kill -0 "$pid" 2>/dev/null; then kill -KILL "$pid" 2>/dev/null || true fi done fi

**Emergency cleanup — orphan processes from a crashed test**:

```bash
PORT_PIDS=$(lsof -ti:"$TARGET_PORT" 2>/dev/null || true) if [ -n "$PORT_PIDS" ]; then echo "Additional PIDs holding port $TARGET_PORT: $PORT_PIDS" for pid in $PORT_PIDS; do kill -TERM "$pid" 2>/dev/null || true done sleep 3 for pid in $PORT_PIDS; do if kill -0 "$pid" 2>/dev/null; then kill -KILL "$pid" 2>/dev/null || true fi done fi

**紧急清理——崩溃测试中的遗留进程**:

```bash

If the original STARTUP_PID is empty or already dead, but the port is still occupied,

如果原始STARTUP_PID为空或已终止,但端口仍被占用,

this indicates an orphan process from a crash or a detached child.

这表明存在崩溃或分离子进程产生的遗留进程。

if [ -z "$STARTUP_PID" ] || ! kill -0 "$STARTUP_PID" 2>/dev/null; then ORPHAN_PIDS=$(lsof -ti:"$TARGET_PORT" 2>/dev/null || true) if [ -n "$ORPHAN_PIDS" ]; then echo "WARNING: Orphan process(es) detected on port $TARGET_PORT: $ORPHAN_PIDS" for pid in $ORPHAN_PIDS; do kill -TERM "$pid" 2>/dev/null || true done sleep 3 for pid in $ORPHAN_PIDS; do if kill -0 "$pid" 2>/dev/null; then kill -KILL "$pid" 2>/dev/null || true fi done fi fi
undefined
if [ -z "$STARTUP_PID" ] || ! kill -0 "$STARTUP_PID" 2>/dev/null; then ORPHAN_PIDS=$(lsof -ti:"$TARGET_PORT" 2>/dev/null || true) if [ -n "$ORPHAN_PIDS" ]; then echo "WARNING: Orphan process(es) detected on port $TARGET_PORT: $ORPHAN_PIDS" for pid in $ORPHAN_PIDS; do kill -TERM "$pid" 2>/dev/null || true done sleep 3 for pid in $ORPHAN_PIDS; do if kill -0 "$pid" 2>/dev/null; then kill -KILL "$pid" 2>/dev/null || true fi done fi fi
undefined

7.3 Verify Port Release (AC-030)

7.3 验证端口释放(AC-030)

bash
PORT_RELEASED="false"
PORT_CHECK_WAIT=0
MAX_PORT_CHECK_WAIT=15

while [ "$PORT_CHECK_WAIT" -lt "$MAX_PORT_CHECK_WAIT" ]; do
    if ! lsof -i :"$TARGET_PORT" >/dev/null 2>&1 && ! nc -z localhost "$TARGET_PORT" 2>/dev/null; then
        PORT_RELEASED="true"
        echo "Port $TARGET_PORT is free."
        break
    fi
    sleep 1
    PORT_CHECK_WAIT=$((PORT_CHECK_WAIT + 1))
done

if [ "$PORT_RELEASED" != "true" ]; then
    echo "WARNING: Port $TARGET_PORT is still occupied after teardown."
fi
bash
PORT_RELEASED="false"
PORT_CHECK_WAIT=0
MAX_PORT_CHECK_WAIT=15

while [ "$PORT_CHECK_WAIT" -lt "$MAX_PORT_CHECK_WAIT" ]; do
    if ! lsof -i :"$TARGET_PORT" >/dev/null 2>&1 && ! nc -z localhost "$TARGET_PORT" 2>/dev/null; then
        PORT_RELEASED="true"
        echo "Port $TARGET_PORT is free."
        break
    fi
    sleep 1
    PORT_CHECK_WAIT=$((PORT_CHECK_WAIT + 1))
done

if [ "$PORT_RELEASED" != "true" ]; then
    echo "WARNING: Port $TARGET_PORT is still occupied after teardown."
fi

7.4 Identify Remaining Resources (REQ-023)

7.4 识别剩余资源(REQ-023)

bash
REMAINING_PROCESSES="none"

if [ "$APP_TYPE" = "Docker-managed" ]; then
    REMAINING_CONTAINERS=$(docker compose ps --format '{{.Name}}' 2>/dev/null | tr '\n' ',' | sed 's/,$//')
    if [ -n "$REMAINING_CONTAINERS" ]; then
        REMAINING_PROCESSES="containers: $REMAINING_CONTAINERS"
    fi
else
    REMAINING_PIDS=$(lsof -ti:"$TARGET_PORT" 2>/dev/null | tr '\n' ' ' | sed 's/ $//')
    if [ -n "$REMAINING_PIDS" ]; then
        REMAINING_PROCESSES="PIDs: $REMAINING_PIDS"
    fi
fi
bash
REMAINING_PROCESSES="none"

if [ "$APP_TYPE" = "Docker-managed" ]; then
    REMAINING_CONTAINERS=$(docker compose ps --format '{{.Name}}' 2>/dev/null | tr '\n' ',' | sed 's/,$//')
    if [ -n "$REMAINING_CONTAINERS" ]; then
        REMAINING_PROCESSES="containers: $REMAINING_CONTAINERS"
    fi
else
    REMAINING_PIDS=$(lsof -ti:"$TARGET_PORT" 2>/dev/null | tr '\n' ' ' | sed 's/ $//')
    if [ -n "$REMAINING_PIDS" ]; then
        REMAINING_PROCESSES="PIDs: $REMAINING_PIDS"
    fi
fi

7.5 Warn User on Incomplete Teardown

7.5 清理不完整时警告用户

bash
if [ "$PORT_RELEASED" != "true" ] || [ "$DOCKER_TEARDOWN_FAILED" = "true" ]; then
    echo "WARNING: Teardown did not complete cleanly. Manual cleanup may be required."
    if [ "$APP_TYPE" = "Docker-managed" ]; then
        echo "  Remaining containers: ${REMAINING_PROCESSES}"
        echo "  Manual command: docker compose -f '${PROJECT_ROOT}/docker-compose.yml' down"
    else
        echo "  Remaining PIDs on port ${TARGET_PORT}: ${REMAINING_PIDS}"
        echo "  Manual command: kill ${REMAINING_PIDS}"
    fi
fi
bash
if [ "$PORT_RELEASED" != "true" ] || [ "$DOCKER_TEARDOWN_FAILED" = "true" ]; then
    echo "WARNING: Teardown did not complete cleanly. Manual cleanup may be required."
    if [ "$APP_TYPE" = "Docker-managed" ]; then
        echo "  Remaining containers: ${REMAINING_PROCESSES}"
        echo "  Manual command: docker compose -f '${PROJECT_ROOT}/docker-compose.yml' down"
    else
        echo "  Remaining PIDs on port ${TARGET_PORT}: ${REMAINING_PIDS}"
        echo "  Manual command: kill ${REMAINING_PIDS}"
    fi
fi

7.6 Record Teardown Status

7.6 记录清理状态

Update the report file in-place (append the Teardown Status section if it was not written in Phase 6, or ensure the variables are correctly set before Phase 6 finalizes). The preferred order is:
  1. Phase 6 writes the report with placeholder teardown variables.
  2. Phase 7 executes teardown.
  3. Phase 7 updates the placeholders with actual values using an in-place edit (e.g.,
    sed
    ):
bash
sed -i.bak \
    -e "s/\${TEARDOWN_EXECUTED}/${TEARDOWN_EXECUTED}/g" \
    -e "s/\${PORT_RELEASED}/${PORT_RELEASED}/g" \
    -e "s/\${REMAINING_PROCESSES}/${REMAINING_PROCESSES}/g" \
    "$REPORT_FILE" && rm -f "$REPORT_FILE.bak"
If the report was already finalized before teardown (a valid alternative), append a Post-Teardown Update section at the end of the report:
markdown
undefined
就地更新报告文件(如果阶段6未写入清理状态部分,则追加该部分;或在阶段6最终确定前确保变量已正确设置)。首选顺序是:
  1. 阶段6写入带有清理变量占位符的报告。
  2. 阶段7执行清理。
  3. 阶段7使用就地编辑(例如:
    sed
    )将占位符更新为实际值:
bash
sed -i.bak \
    -e "s/\${TEARDOWN_EXECUTED}/${TEARDOWN_EXECUTED}/g" \
    -e "s/\${PORT_RELEASED}/${PORT_RELEASED}/g" \
    -e "s/\${REMAINING_PROCESSES}/${REMAINING_PROCESSES}/g" \
    "$REPORT_FILE" && rm -f "$REPORT_FILE.bak"
如果报告在清理前已最终确定(有效替代方案),则在报告末尾追加清理后更新章节:
markdown
undefined

Post-Teardown Update

清理后更新

PropertyValue
Teardown Executed${TEARDOWN_EXECUTED}
Port Released${PORT_RELEASED}
Remaining Processes${REMAINING_PROCESSES}

**IMPORTANT**: Do NOT delete the report or modify any spec/task files during teardown (REQ-NR005).
属性
已执行清理${TEARDOWN_EXECUTED}
端口已释放${PORT_RELEASED}
剩余进程${REMAINING_PROCESSES}

**重要提示**:清理期间不得删除报告或修改任何规范/任务文件(REQ-NR005)。

Phase 8: Present Results

阶段8:呈现结果

  1. Print a concise summary to the user:
    • ✅ VERIFIED: N | ❌ FAILED: N | ⚠️ MANUAL: N
    • Report path
    • Teardown status (or
      --keep-alive
      notice)
  2. If any AC failed:
    • Highlight the failed AC and evidence
    • Recommend running
      /developer-kit-specs:specs.task-implementation
      again for the failing task
  3. Mark all todos complete.
  1. 向用户打印简洁摘要:
    • ✅ VERIFIED: N | ❌ FAILED: N | ⚠️ MANUAL: N
    • 报告路径
    • 清理状态(或
      --keep-alive
      通知)
  2. 如果任何验收标准失败:
    • 突出显示失败的验收标准和证据
    • 建议针对失败任务重新运行
      /developer-kit-specs:specs.task-implementation
  3. 标记所有待办事项为完成。

Examples

示例

Spring Boot REST API Verification

Spring Boot REST API验证

bash
/developer-kit-specs:specs.e2e-verify --spec=docs/specs/001-user-auth/
Actions:
  1. Detect
    pom.xml
    +
    docker-compose.yml
    → Docker-managed Spring Boot
  2. Run
    docker compose up -d
  3. Wait for
    localhost:8080/actuator/health
  4. Read AC from spec; find
    [IMP]
    AC describing login endpoint
  5. Execute:
    curl -s -w "\n%{http_code}" -X POST http://localhost:8080/api/login -d '{"email":"test@example.com","password":"secret"}'
  6. Assert status
    200
    and response contains
    token
  7. Generate report:
    docs/specs/001-user-auth/e2e-report-2026-05-31-143022.md
  8. Run
    docker compose down
bash
/developer-kit-specs:specs.e2e-verify --spec=docs/specs/001-user-auth/
操作:
  1. 检测到
    pom.xml
    +
    docker-compose.yml
    → Docker管理的Spring Boot
  2. 运行
    docker compose up -d
  3. 等待
    localhost:8080/actuator/health
  4. 从规范中读取验收标准;找到描述登录端点的
    [IMP]
    验收标准
  5. 执行:
    curl -s -w "\n%{http_code}" -X POST http://localhost:8080/api/login -d '{"email":"test@example.com","password":"secret"}'
  6. 断言状态
    200
    且响应包含
    token
  7. 生成报告:
    docs/specs/001-user-auth/e2e-report-2026-05-31-143022.md
  8. 运行
    docker compose down

React SPA Verification

React SPA验证

bash
/developer-kit-specs:specs.e2e-verify --spec=docs/specs/002-dashboard/
Actions:
  1. Detect
    package.json
    with
    react
    dependency
  2. Run
    npm run dev
    ; wait for
    localhost:5173
  3. Launch Playwright, navigate to
    /dashboard
  4. AC says "User can click 'Refresh' to reload data" →
    page.click('[data-testid=refresh]')
    , assert table rows update
  5. Screenshot on success; capture on failure
  6. Generate report with screenshot artifacts
  7. Kill dev server process
bash
/developer-kit-specs:specs.e2e-verify --spec=docs/specs/002-dashboard/
操作:
  1. 检测到包含
    react
    依赖的
    package.json
  2. 运行
    npm run dev
    ;等待
    localhost:5173
  3. 启动Playwright,导航到
    /dashboard
  4. 验收标准要求“用户可点击‘Refresh’重新加载数据” →
    page.click('[data-testid=refresh]')
    ,断言表格行更新
  5. 成功时截图;失败时捕获截图
  6. 生成包含截图工件的报告
  7. 终止开发服务器进程

Desktop App Verification

桌面应用验证

bash
/developer-kit-specs:specs.e2e-verify --spec=docs/specs/003-settings-app/
Actions:
  1. Detect
    src-tauri/Cargo.toml
    → Tauri desktop app
  2. Run
    cargo tauri build --debug
    then launch the
    .app
    /
    .exe
  3. Use computer-use to verify Settings window opens
  4. AC says "User can toggle dark mode" → click toggle, capture screenshot, verify theme change
  5. Generate report with step screenshots
  6. Close app and verify process terminated
bash
/developer-kit-specs:specs.e2e-verify --spec=docs/specs/003-settings-app/
操作:
  1. 检测到
    src-tauri/Cargo.toml
    → Tauri桌面应用
  2. 运行
    cargo tauri build --debug
    然后启动
    .app
    /
    .exe
  3. 使用computer-use验证设置窗口打开
  4. 验收标准要求“用户可切换深色模式” → 点击切换开关,捕获截图,验证主题变化
  5. 生成包含步骤截图的报告
  6. 关闭应用并验证进程已终止

Command Whitelist

命令白名单

Only commands documented in
references/test-execution-patterns.md
may be executed automatically. The whitelist covers:
CategoryAllowed Commands
Docker
docker compose up -d --build
,
docker compose down
,
docker compose ps
,
docker compose logs
JVM / Spring Boot
./mvnw spring-boot:run
,
./gradlew bootRun
Node.js / NestJS
npm run dev
,
npm run start:dev
,
npm start
,
node server.js
Python
uvicorn main:app --reload
,
python manage.py runserver
,
flask run
Desktop (Tauri)
cargo tauri build --debug
,
cargo tauri dev
,
open *.app
, direct binary execution
Desktop (Electron)
npm run electron:dev
,
npx electron .
Testing
curl
(without
-k
/
--insecure
unless
--insecure
passed),
npx playwright
,
node
(for inline Playwright scripts)
Teardown
docker compose down
,
kill -TERM
,
kill -KILL
(for stuck processes only),
lsof
,
nc
Non-whitelisted command policy: IF a derived command is NOT in the whitelist → use
AskUserQuestion
to request explicit user confirmation before execution. The user MUST approve the command; otherwise, abort.
仅允许自动执行
references/test-execution-patterns.md
中记录的命令。白名单包括:
类别允许的命令
Docker
docker compose up -d --build
docker compose down
docker compose ps
docker compose logs
JVM / Spring Boot
./mvnw spring-boot:run
./gradlew bootRun
Node.js / NestJS
npm run dev
npm run start:dev
npm start
node server.js
Python
uvicorn main:app --reload
python manage.py runserver
flask run
桌面(Tauri)
cargo tauri build --debug
cargo tauri dev
open *.app
、直接二进制执行
桌面(Electron)
npm run electron:dev
npx electron .
测试
curl
(默认不使用
-k
/
--insecure
,除非传递
--insecure
)、
npx playwright
node
(用于内联Playwright脚本)
清理
docker compose down
kill -TERM
kill -KILL
(仅用于卡住的进程)、
lsof
nc
非白名单命令策略:如果推导的命令不在白名单中→使用
AskUserQuestion
请求用户明确确认后再执行。用户必须批准该命令;否则终止。

Security Guards and Negative Requirements

安全防护和负面要求

The following negative requirements (REQ-NR001 through REQ-NR008) are enforced at multiple points in the skill workflow:
在技能工作流的多个点强制执行以下负面要求(REQ-NR001至REQ-NR008):

REQ-NR001: No Destructive Commands

REQ-NR001:禁止破坏性命令

The system SHALL NOT run
sudo
,
rm -rf
,
docker system prune
,
mkfs
,
dd
, or any other destructive system command during startup, test, or teardown.
Enforcement:
  • Phase 1.5 scans all commands for forbidden patterns before execution.
  • Phase 3 data integrity pre-flight rejects commands that would destroy data.
  • Teardown commands are restricted to
    docker compose down
    and graceful process termination (
    kill -TERM
    /
    kill -KILL
    for stuck orphans only).
系统在启动、测试或清理期间不得运行
sudo
rm -rf
docker system prune
mkfs
dd
或任何其他破坏性系统命令。
执行方式
  • 阶段1.5在执行前扫描所有命令以查找禁止模式。
  • 阶段3数据完整性预检拒绝会破坏数据的命令。
  • 清理命令仅限于
    docker compose down
    和优雅进程终止(
    kill -TERM
    /
    kill -KILL
    仅用于卡住的遗留进程)。

REQ-NR002: No Secrets in Report

REQ-NR002:报告中不得包含机密信息

The system SHALL NOT expose secrets, API keys, or passwords in the E2E report.
Enforcement:
  • Phase 6 redaction pass removes:
    • Authorization: Bearer <token>
      Authorization: Bearer ***REDACTED***
    • E2E_AUTH_TOKEN=...
      ,
      E2E_USERNAME=...
      ,
      E2E_PASSWORD=...
      E2E_xxx=***REDACTED***
    • "token": "..."
      "token": "***REDACTED***"
    • Passwords in
      curl -u user:password
      curl -u user:***REDACTED***
  • Only the redacted version of logs is written to the report.
  • The original unredacted temp file is discarded immediately after redaction.
系统不得在E2E报告中暴露机密信息、API密钥或密码。
执行方式
  • 阶段6编辑处理移除:
    • Authorization: Bearer <token>
      Authorization: Bearer ***REDACTED***
    • E2E_AUTH_TOKEN=...
      E2E_USERNAME=...
      E2E_PASSWORD=...
      E2E_xxx=***REDACTED***
    • "token": "..."
      "token": "***REDACTED***"
    • curl -u user:password
      中的密码 →
      curl -u user:***REDACTED***
  • 仅将编辑后的日志版本写入报告。
  • 编辑后立即丢弃原始未编辑的临时文件。

REQ-NR003: No TLS Bypass by Default

REQ-NR003:默认禁止TLS绕过

The system SHALL NOT disable TLS certificate verification (
-k
/
--insecure
) in curl by default.
Enforcement:
  • Phase 1.5 rejects curl commands containing
    -k
    or
    --insecure
    unless the
    --insecure
    flag was explicitly passed.
  • Phase 4 curl command generation rules explicitly forbid adding
    -k
    /
    --insecure
    .
  • IF
    --insecure
    is passed → a warning is logged: "TLS certificate verification is disabled. Use only for local development."
系统默认不得在curl中禁用TLS证书验证(
-k
/
--insecure
)。
执行方式
  • 阶段1.5拒绝包含
    -k
    --insecure
    的curl命令,除非明确传递了
    --insecure
    标志。
  • 阶段4 curl命令生成规则明确禁止添加
    -k
    /
    --insecure
  • 如果传递了
    --insecure
    →记录警告:“TLS certificate verification is disabled. Use only for local development.”

REQ-NR004: No Data Overwrite on Startup

REQ-NR004:启动时不得覆盖数据

The system SHALL NOT overwrite or delete existing databases, volumes, or local data unless the startup command itself does so.
Enforcement:
  • Phase 1.5 data integrity pre-check rejects commands containing
    rm
    ,
    --volumes
    ,
    prune
    ,
    drop
    , or similar destructive data patterns.
  • Phase 3 data integrity pre-flight aborts startup if the command would destroy existing data.
系统不得覆盖或删除现有数据库、卷或本地数据,除非启动命令本身会执行此操作。
执行方式
  • 阶段1.5数据完整性预检拒绝包含
    rm
    --volumes
    prune
    drop
    或类似破坏性数据模式的命令。
  • 阶段3数据完整性预检如果命令会破坏现有数据,则终止启动。

REQ-NR005: No Spec or Task File Modification

REQ-NR005:不得修改规范或任务文件

The system SHALL NOT modify the functional specification or task files during report generation or teardown.
Enforcement:
  • Phase 6 writes the report as a new file (
    e2e-report-YYYY-MM-DD-HHMMSS.md
    ) only.
  • Phase 7 teardown appends only a Post-Teardown Update section to the report; it never modifies spec or task files.
  • The skill is read-only regarding source code, specs, and tasks.
系统在报告生成或清理期间不得修改功能规范或任务文件。
执行方式
  • 阶段6仅将报告写入新文件(
    e2e-report-YYYY-MM-DD-HHMMSS.md
    )。
  • 阶段7清理仅向报告追加清理后更新章节;绝不修改规范或任务文件。
  • 技能对源代码、规范和任务是只读的。

REQ-NR006: No Orphan Processes on Failure

REQ-NR006:失败时不得遗留进程

The system SHALL NOT leave orphan processes or containers running when startup fails.
Enforcement:
  • Phase 3 Docker startup: on timeout,
    docker compose down
    is executed before exiting.
  • Phase 3 Spring Boot / Node.js startup: on timeout,
    kill -TERM $STARTUP_PID
    is executed before exiting.
  • Phase 7 teardown includes emergency orphan cleanup for processes still holding the target port.
启动失败时,系统不得遗留进程或容器运行。
执行方式
  • 阶段3 Docker启动:超时后,在退出前执行
    docker compose down
  • 阶段3 Spring Boot / Node.js启动:超时后,在退出前执行
    kill -TERM $STARTUP_PID
  • 阶段7清理包括对仍占用目标端口的进程进行紧急遗留进程清理。

REQ-NR007: Per-Test Timeout Enforcement

REQ-NR007:强制执行每个测试的超时

The system SHALL NOT block indefinitely on a hanging test.
Enforcement:
  • Phase 4 Playwright tests: wrapped with
    timeout --signal=TERM <N>
    ; if timeout occurs, the AC is marked
    FAILED
    .
  • Phase 4 Desktop tests: a manual polling loop checks elapsed time against
    TEST_TIMEOUT_SEC
    ; if exceeded, the AC is marked
    FAILED
    .
  • If a test hangs, it is terminated and marked
    FAILED
    , never
    VERIFIED
    or
    PENDING
    .
系统不得无限期挂起在卡住的测试上。
执行方式
  • 阶段4 Playwright测试:使用
    timeout --signal=TERM <N>
    包装;如果超时,验收标准标记为
    FAILED
  • 阶段4桌面测试:手动轮询循环检查经过时间是否超过
    TEST_TIMEOUT_SEC
    ;如果超过,验收标准标记为
    FAILED
  • 如果测试卡住,将其终止并标记为
    FAILED
    ,绝不标记为
    VERIFIED
    PENDING

REQ-NR008: No Retry on Failed curl Tests

REQ-NR008:失败的curl测试不得重试

The system SHALL NOT retry failed curl requests.
Enforcement:
  • Phase 4 curl execution: each command is executed exactly once.
  • NO
    --retry
    flag, NO retry loops, NO fallback re-execution.
  • The first result is recorded immediately; if it fails, the AC is marked
    FAILED
    .
系统不得重试失败的curl请求。
执行方式
  • 阶段4 curl执行:每个命令仅执行一次。
  • 不使用
    --retry
    标志,不使用重试循环,不使用回退重新执行。
  • 立即记录第一个结果;如果失败,验收标准标记为
    FAILED

Constraints and Warnings

约束和警告

  • Read-only on source: This skill never edits application source code, test files, configuration, functional specifications, or task files.
  • No auto-install: If Playwright or Docker is missing, the skill reports the gap and suggests install commands. It does NOT run
    npm install -g
    or
    brew install
    automatically.
  • Local only: Tests run against
    localhost
    . Remote URLs, staging, or production endpoints are out of scope.
  • Destructive guard: Startup, test, and teardown commands are whitelisted per
    references/test-execution-patterns.md
    . Any command outside the whitelist requires user confirmation via
    AskUserQuestion
    .
  • Secret hygiene: Authorization headers, bearer tokens,
    E2E_*
    environment variables, and passwords are redacted from the report. Only header names and redacted values appear.
  • TLS default: curl commands use strict TLS verification by default. Pass
    --insecure
    only for local development with self-signed certificates.
  • Cleanup responsibility: If
    --keep-alive
    is passed, the user is responsible for manual teardown.
  • No persistent test suite: The skill generates ad-hoc tests for verification. It does not create permanent test files in the project.
  • 源代码只读:该技能绝不编辑应用源代码、测试文件、配置、功能规范或任务文件。
  • 不自动安装:如果缺少Playwright或Docker,技能报告该问题并建议安装命令。它不会自动运行
    npm install -g
    brew install
  • 仅本地:测试针对
    localhost
    运行。远程URL、 staging或生产端点不在范围内。
  • 破坏性防护:启动、测试和清理命令根据
    references/test-execution-patterns.md
    列入白名单。任何白名单之外的命令需要通过
    AskUserQuestion
    获得用户确认。
  • 机密信息卫生:报告中编辑Authorization标头、Bearer令牌、
    E2E_*
    环境变量和密码。仅显示标头名称和编辑后的值。
  • TLS默认设置:curl命令默认使用严格TLS验证。仅在使用自签名证书的本地开发场景中传递
    --insecure
  • 清理责任:如果传递了
    --keep-alive
    ,用户负责手动清理。
  • 无持久测试套件:该技能为验证生成临时测试。它不会在项目中创建永久测试文件。