Back to Details

running-release-tests

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Release Testing

发布测试

AgentSpace routing (SigV4 only): If 
list_agent_spaces
is available in your tool list and the multi-space orchestration skill has NOT been invoked yet this session, invoke it first to determine which 
agent_space_id
to use. Then pass 
agent_space_id
on all tool calls below. For bearer token auth this is unnecessary — the token is already scoped to one space.
Run automated release testing in the cloud via the AWS DevOps Agent's Release Testing Agent. Supports UI testing (browser-based) and API testing (OpenAPI spec-based). Uses pre-existing test profiles that define target URL, agent type, personas, and credentials.
Input is a test profile — the test profile already contains the target URL, agent type (UI or API), test personas, and credentials. Do NOT ask the user for a URL directly; the URL is defined in the test profile.
AgentSpace 路由（仅支持SigV4）： 如果你的工具列表中包含
list_agent_spaces
，且本次会话尚未调用多空间编排技能，请先调用该技能确定要使用的
agent_space_id
。之后在以下所有工具调用中传入
agent_space_id
。对于Bearer令牌认证，此步骤无需执行——令牌已限定在单个空间范围内。
通过AWS DevOps Agent的Release Testing Agent在云端运行自动化发布测试。支持UI测试（基于浏览器）和API测试（基于OpenAPI规范）。使用预先创建的测试配置文件，其中定义了目标URL、Agent类型、角色和凭据。
输入为测试配置文件——测试配置文件已包含目标URL、Agent类型（UI或API）、测试角色和凭据。请勿直接向用户索要URL；URL已在测试配置文件中定义。

Prerequisites

前提条件

  • A pre-existing test profile (Knowledge Item ID like 
    ki-12345
    ) created from the AWS DevOps Agent console
  • 已通过AWS DevOps Agent控制台创建预先存在的测试配置文件（知识项ID格式如
    ki-12345
    ）

Gathering test parameters

收集测试参数

Before starting any workflow, you MUST gather the following parameters. Do NOT proceed to job creation until answered.
在启动任何工作流之前，你必须收集以下参数。在得到答复前，请勿继续创建任务。

Step 1 — Test profile (required)

步骤1 — 测试配置文件（必填）

Ask the user which test profile to use. The test profile already contains the target URL, agent type (UI or API), test personas, and credentials configuration — these do NOT need to be gathered separately.
Note: A pre-existing test profile is a prerequisite. Test profiles are created using the AWS DevOps Agent console or API, not through this tool. If the user asks whether one can be created here, inform them it must already exist.
询问用户要使用哪个测试配置文件。测试配置文件已包含目标URL、Agent类型（UI或API）、测试角色和凭据配置——这些无需单独收集。
注意： 预先存在的测试配置文件是前提条件。测试配置文件需通过AWS DevOps Agent控制台或API创建，无法通过本工具创建。如果用户询问是否可以在此创建，请告知他们配置文件必须已存在。

Step 2 — Test requirement (optional)

步骤2 — 测试需求（可选）

If the user has not already mentioned a test focus, ask:
"Do you have a specific test requirement or focus area? If not, I'll run a full exploratory test."
Wait for the user's response. If they provide one, use it as the 
test_requirement
. If they say no or skip, proceed without it.
IMPORTANT: You MUST wait for the user to respond before proceeding to job creation.
如果用户尚未提及测试重点，请询问：
"你是否有特定的测试需求或重点领域？如果没有，我将运行完整的探索性测试。"
等待用户回复。如果用户提供了需求，将其作为
test_requirement
。如果用户表示没有或跳过，则无需该参数继续执行。
重要提示： 在继续创建任务之前，你必须等待用户回复。

Core workflow

核心工作流

1. Select Agent Space

1. 选择Agent空间

List available agent spaces:
aws devops-agent list-agent-spaces --region us-east-1
Present the list to the user and ask which agent space they'd like to use. Do NOT proceed until the user has selected one. Use the selected 
agentSpaceId
as 
SPACE_ID
in all subsequent calls.
列出可用的Agent空间：
aws devops-agent list-agent-spaces --region us-east-1
将列表展示给用户，并询问他们想要使用哪个Agent空间。在用户选择完成前，请勿继续执行。 在后续所有调用中，将选中的
agentSpaceId
作为
SPACE_ID
使用。

2. Check tool availability

2. 检查工具可用性

Verify that the following tools are available: 
aws_devops_agent__create_release_testing_job
, 
aws_devops_agent__get_task
, 
aws_devops_agent__list_journal_records
, 
aws_devops_agent__get_release_ui_testing_report
, 
aws_devops_agent__get_release_api_testing_report
. These tools are NOT deferred/lazy-loaded — if they do not appear in your tool list, they are unavailable. Do NOT search for them via ToolSearch. If any are missing, skip the remaining steps in this section and use the "Fallback (aws-mcp)" path below instead.
验证以下工具是否可用：
aws_devops_agent__create_release_testing_job
、
aws_devops_agent__get_task
、
aws_devops_agent__list_journal_records
、
aws_devops_agent__get_release_ui_testing_report
、
aws_devops_agent__get_release_api_testing_report
。这些工具不会延迟/懒加载——如果它们未出现在你的工具列表中，则表示不可用。请勿通过ToolSearch搜索它们。如果缺少任何工具，请跳过本节剩余步骤，改用下方的"备选方案（aws-mcp）"路径。

3. Start the Job

3. 启动任务

aws_devops_agent__create_release_testing_job(
    test_profile_id="ki-12345",
    webhook_event_message="<optional test requirement>"
)
→ {"taskId": "...", "executionId": "...", "status": "started"}
Record the taskId and executionId from the response.
aws_devops_agent__create_release_testing_job(
    test_profile_id="ki-12345",
    webhook_event_message="<可选测试需求>"
)
→ {"taskId": "...", "executionId": "...", "status": "started"}
记录响应中的taskId和executionId。

4. Poll for Status

4. 轮询状态

Call 
aws_devops_agent__get_task(task_id=TASK_ID)
every 30 seconds until the status transitions to 
IN_PROGRESS
or a terminal state.
每隔30秒调用一次
aws_devops_agent__get_task(task_id=TASK_ID)
，直到状态变为
IN_PROGRESS
或终止状态。

5. Monitor Until Completion

5. 监控直至完成

Once 
IN_PROGRESS
, poll for progress in a loop:
  1. Call 
    aws_devops_agent__list_journal_records(execution_id=EXEC_ID, order="ASC")
    to fetch new findings.
  2. Present each record to the user with a friendly progress update.
  3. Use 
    next_token
    from the response to fetch only new records on subsequent polls.
  4. Wait 20 seconds between each poll iteration.
  5. Check 
    aws_devops_agent__get_task(task_id=TASK_ID)
    periodically — stop when terminal status (
    COMPLETED
    , 
    FAILED
    , 
    CANCELED
    , 
    TIMED_OUT
    ).
状态变为
IN_PROGRESS
后，循环轮询进度：
  1. 调用
    aws_devops_agent__list_journal_records(execution_id=EXEC_ID, order="ASC")
    获取新的测试结果。
  2. 向用户展示每条记录并提供友好的进度更新。
  3. 使用响应中的
    next_token
    在后续轮询中仅获取新记录。
  4. 每次轮询间隔等待20秒。
  5. 定期调用
    aws_devops_agent__get_task(task_id=TASK_ID)
    ——当状态变为终止状态（
    COMPLETED
    、
    FAILED
    、
    CANCELED
    、
    TIMED_OUT
    ）时停止。

6. Present Results

6. 展示结果

Once the job reaches a terminal status:
  • If 
    COMPLETED
    :
    1. Determine the report type from the test profile's agent type (UI or API). Call 
      aws_devops_agent__get_release_ui_testing_report(execution_id=EXEC_ID)
      for UI profiles or 
      aws_devops_agent__get_release_api_testing_report(execution_id=EXEC_ID)
      for API profiles.
    2. Write the report contents to a markdown file:
      release-testing-report-<YYYY-MM-DD-HHmmss>.md
    3. Inform the user that the report was saved, including the file path.
  • If 
    FAILED
    or 
    TIMED_OUT
    : Present the error information and suggest next steps.
  • If 
    CANCELED
    : Inform the user the job was canceled and no report is available.
任务进入终止状态后：
  • 如果状态为
    COMPLETED
    ：
    1. 根据测试配置文件的Agent类型（UI或API）确定报告类型。对于UI配置文件，调用
      aws_devops_agent__get_release_ui_testing_report(execution_id=EXEC_ID)
      ；对于API配置文件，调用
      aws_devops_agent__get_release_api_testing_report(execution_id=EXEC_ID)
      。
    2. 将报告内容写入markdown文件：
      release-testing-report-<YYYY-MM-DD-HHmmss>.md
    3. 告知用户报告已保存，并提供文件路径。
  • 如果状态为
    FAILED
    或
    TIMED_OUT
    ：展示错误信息并建议后续步骤。
  • 如果状态为
    CANCELED
    ：告知用户任务已取消，无可用报告。

Cancelling a job

取消任务

aws_devops_agent__cancel_release_testing_job(task_id=TASK_ID)
aws_devops_agent__cancel_release_testing_job(task_id=TASK_ID)

Error handling

错误处理

  1. If the task status changes to 
    FAILED
    , stop the workflow and report the error.
  2. If the task does not reach 
    IN_PROGRESS
    within 5 minutes, cancel it using 
    cancel_release_testing_job
    .
  3. If any output contains "NoCredentialsError", "ExpiredTokenException", or auth failures, suggest the user refresh their credentials or check the bearer token.
  4. If throttled (
    429
    or 
    ThrottlingException
    ), wait 30 seconds before retrying. After 3 retries, inform the user.
  1. 如果任务状态变为
    FAILED
    ，停止工作流并报告错误。
  2. 如果任务在5分钟内未进入
    IN_PROGRESS
    状态，使用
    cancel_release_testing_job
    取消任务。
  3. 如果任何输出包含"NoCredentialsError"、"ExpiredTokenException"或认证失败信息，建议用户刷新凭据或检查Bearer令牌。
  4. 如果遇到限流（
    429
    或
    ThrottlingException
    ），等待30秒后重试。重试3次后，告知用户。

Fallback (aws-mcp)

备选方案（aws-mcp）

If the 
aws-devops-agent
remote server is unavailable, use the AWS CLI directly:
Tell the user: "Remote server unavailable — using direct AWS API fallback."
如果
aws-devops-agent
远程服务器不可用，直接使用AWS CLI：
告知用户："远程服务器不可用——使用直接AWS API备选方案。"

1. Select Agent Space

1. 选择Agent空间

List available agent spaces:
aws devops-agent list-agent-spaces --region us-east-1
Present the list to the user and ask which agent space they'd like to use. Do NOT proceed until the user has selected one. Use the selected 
agentSpaceId
as 
SPACE_ID
in all subsequent calls.
列出可用的Agent空间：
aws devops-agent list-agent-spaces --region us-east-1
将列表展示给用户，并询问他们想要使用哪个Agent空间。在用户选择完成前，请勿继续执行。 在后续所有调用中，将选中的
agentSpaceId
作为
SPACE_ID
使用。

2. Start the Job

2. 启动任务

aws devops-agent create-backlog-task \
  --agent-space-id SPACE_ID \
  --task-type RELEASE_TESTING \
  --title 'Release Testing' \
  --priority MEDIUM \
  --description '{\"testProfileId\": \"<PROFILE_ID>\", \"webhookEventMessage\": \"<REQUIREMENT>\"}' \
  --region us-east-1
If the user provided a test requirement, include it as 
webhookEventMessage
. If not, omit the field or leave it empty.
aws devops-agent create-backlog-task \
  --agent-space-id SPACE_ID \
  --task-type RELEASE_TESTING \
  --title 'Release Testing' \
  --priority MEDIUM \
  --description '{\"testProfileId\": \"<PROFILE_ID>\", \"webhookEventMessage\": \"<REQUIREMENT>\"}' \
  --region us-east-1
如果用户提供了测试需求，将其包含在
webhookEventMessage
中。如果没有，则省略该字段或留空。

3. Poll for Status

3. 轮询状态

aws devops-agent get-backlog-task \
  --agent-space-id SPACE_ID \
  --task-id TASK_ID \
  --region us-east-1
Poll every 30 seconds until the status transitions to 
IN_PROGRESS
or a terminal state (
COMPLETED
, 
FAILED
, 
CANCELED
, 
TIMED_OUT
).
aws devops-agent get-backlog-task \
  --agent-space-id SPACE_ID \
  --task-id TASK_ID \
  --region us-east-1
每隔30秒轮询一次，直到状态变为
IN_PROGRESS
或终止状态（
COMPLETED
、
FAILED
、
CANCELED
、
TIMED_OUT
）。

4. Monitor Until Completion

4. 监控直至完成

Once 
IN_PROGRESS
, poll for progress in a loop:
aws devops-agent list-journal-records \
  --agent-space-id SPACE_ID \
  --execution-id EXEC_ID \
  --order ASC \
  --region us-east-1
  1. Present each record to the user with a friendly progress update.
  2. Use 
    next_token
    from the response to fetch only new records on subsequent polls.
  3. Wait 20 seconds between each poll iteration.
  4. Check 
    get-backlog-task
    periodically — stop when terminal status (
    COMPLETED
    , 
    FAILED
    , 
    CANCELED
    , 
    TIMED_OUT
    ).
状态变为
IN_PROGRESS
后，循环轮询进度：
aws devops-agent list-journal-records \
  --agent-space-id SPACE_ID \
  --execution-id EXEC_ID \
  --order ASC \
  --region us-east-1
  1. 向用户展示每条记录并提供友好的进度更新。
  2. 使用响应中的
    next_token
    在后续轮询中仅获取新记录。
  3. 每次轮询间隔等待20秒。
  4. 定期调用
    get-backlog-task
    ——当状态变为终止状态（
    COMPLETED
    、
    FAILED
    、
    CANCELED
    、
    TIMED_OUT
    ）时停止。

5. Present Results

5. 展示结果

Once the job reaches a terminal status:
  • If 
    COMPLETED
    :
    1. Retrieve the report using the appropriate record type:
      • UI testing: 
        --record-type qa_ui_testing_report
      • API testing: 
        --record-type qa_api_testing_report
      aws devops-agent list-journal-records \
  --agent-space-id SPACE_ID \
  --execution-id EXEC_ID \
  --record-type qa_ui_testing_report \
  --order ASC \
  --region us-east-1
    2. Write the report contents to a markdown file:
      release-testing-report-<YYYY-MM-DD-HHmmss>.md
    3. Inform the user that the report was saved, including the file path.
  • If 
    FAILED
    or 
    TIMED_OUT
    : Present the error information and suggest next steps.
  • If 
    CANCELED
    : Inform the user the job was canceled and no report is available.
任务进入终止状态后：
  • 如果状态为
    COMPLETED
    ：
    1. 根据记录类型获取报告：
      • UI测试：
        --record-type qa_ui_testing_report
      • API测试：
        --record-type qa_api_testing_report
      aws devops-agent list-journal-records \
  --agent-space-id SPACE_ID \
  --execution-id EXEC_ID \
  --record-type qa_ui_testing_report \
  --order ASC \
  --region us-east-1
    2. 将报告内容写入markdown文件：
      release-testing-report-<YYYY-MM-DD-HHmmss>.md
    3. 告知用户报告已保存，并提供文件路径。
  • 如果状态为
    FAILED
    或
    TIMED_OUT
    ：展示错误信息并建议后续步骤。
  • 如果状态为
    CANCELED
    ：告知用户任务已取消，无可用报告。

Cancelling (fallback)

取消任务（备选方案）

aws devops-agent update-backlog-task \
  --agent-space-id SPACE_ID \
  --task-id TASK_ID \
  --task-status CANCELED \
  --region us-east-1
aws devops-agent update-backlog-task \
  --agent-space-id SPACE_ID \
  --task-id TASK_ID \
  --task-status CANCELED \
  --region us-east-1