Release Testing

AgentSpace routing (SigV4 only): If
list_agent_spaces
is available in your tool list and the multi-space orchestration skill has NOT been invoked yet this session, invoke it first to determine which
agent_space_id
to use. Then pass
agent_space_id
on all tool calls below. For bearer token auth this is unnecessary — the token is already scoped to one space.

Run automated release testing in the cloud via the AWS DevOps Agent's Release Testing Agent. Supports UI testing (browser-based) and API testing (OpenAPI spec-based). Uses pre-existing test profiles that define target URL, agent type, personas, and credentials.

Input is a test profile — the test profile already contains the target URL, agent type (UI or API), test personas, and credentials. Do NOT ask the user for a URL directly; the URL is defined in the test profile.

Prerequisites

A pre-existing test profile (Knowledge Item ID like
```
ki-12345
```
) created from the AWS DevOps Agent console

Gathering test parameters

Before starting any workflow, you MUST gather the following parameters. Do NOT proceed to job creation until answered.

Step 1 — Test profile (required)

Ask the user which test profile to use. The test profile already contains the target URL, agent type (UI or API), test personas, and credentials configuration — these do NOT need to be gathered separately.

Note: A pre-existing test profile is a prerequisite. Test profiles are created using the AWS DevOps Agent console or API, not through this tool. If the user asks whether one can be created here, inform them it must already exist.

Step 2 — Test requirement (optional)

If the user has not already mentioned a test focus, ask:

"Do you have a specific test requirement or focus area? If not, I'll run a full exploratory test."

Wait for the user's response. If they provide one, use it as the

test_requirement

. If they say no or skip, proceed without it.

IMPORTANT: You MUST wait for the user to respond before proceeding to job creation.

Core workflow

1. Select Agent Space

List available agent spaces:

aws devops-agent list-agent-spaces --region us-east-1

Present the list to the user and ask which agent space they'd like to use. Do NOT proceed until the user has selected one. Use the selected

agentSpaceId

SPACE_ID

in all subsequent calls.

2. Check tool availability

Verify that the following tools are available:

aws_devops_agent__create_release_testing_job

aws_devops_agent__get_task

aws_devops_agent__list_journal_records

aws_devops_agent__get_release_ui_testing_report

aws_devops_agent__get_release_api_testing_report

. These tools are NOT deferred/lazy-loaded — if they do not appear in your tool list, they are unavailable. Do NOT search for them via ToolSearch. If any are missing, skip the remaining steps in this section and use the "Fallback (aws-mcp)" path below instead.

3. Start the Job

aws_devops_agent__create_release_testing_job(
    test_profile_id="ki-12345",
    webhook_event_message="<optional test requirement>"
)
→ {"taskId": "...", "executionId": "...", "status": "started"}

Record the taskId and executionId from the response.

4. Poll for Status

Call

aws_devops_agent__get_task(task_id=TASK_ID)

every 30 seconds until the status transitions to

IN_PROGRESS

or a terminal state.

5. Monitor Until Completion

Once

IN_PROGRESS

, poll for progress in a loop:

Call

aws_devops_agent__list_journal_records(execution_id=EXEC_ID, order="ASC")

to fetch new findings.

Present each record to the user with a friendly progress update.
Use
```
next_token
```
from the response to fetch only new records on subsequent polls.
Wait 20 seconds between each poll iteration.

Check

aws_devops_agent__get_task(task_id=TASK_ID)

periodically — stop when terminal status (

COMPLETED

FAILED

CANCELED

TIMED_OUT

6. Present Results

Once the job reaches a terminal status:

If
```
COMPLETED
```
:
1. Determine the report type from the test profile's agent type (UI or API). Call
  aws_devops_agent__get_release_ui_testing_report(execution_id=EXEC_ID)
  for UI profiles or
  aws_devops_agent__get_release_api_testing_report(execution_id=EXEC_ID)
  for API profiles.
2. Write the report contents to a markdown file:
```
release-testing-report-<YYYY-MM-DD-HHmmss>.md
```
3. Inform the user that the report was saved, including the file path.
If
```
FAILED
```
or
```
TIMED_OUT
```
: Present the error information and suggest next steps.
If
```
CANCELED
```
: Inform the user the job was canceled and no report is available.

Cancelling a job

aws_devops_agent__cancel_release_testing_job(task_id=TASK_ID)

Error handling

If the task status changes to
```
FAILED
```
, stop the workflow and report the error.
If the task does not reach
```
IN_PROGRESS
```
within 5 minutes, cancel it using
```
cancel_release_testing_job
```
.
If any output contains "NoCredentialsError", "ExpiredTokenException", or auth failures, suggest the user refresh their credentials or check the bearer token.
If throttled (
```
429
```
or
```
ThrottlingException
```
), wait 30 seconds before retrying. After 3 retries, inform the user.

Fallback (aws-mcp)

If the

aws-devops-agent

remote server is unavailable, use the AWS CLI directly:

Tell the user: "Remote server unavailable — using direct AWS API fallback."

1. Select Agent Space

List available agent spaces:

aws devops-agent list-agent-spaces --region us-east-1

Present the list to the user and ask which agent space they'd like to use. Do NOT proceed until the user has selected one. Use the selected

agentSpaceId

SPACE_ID

in all subsequent calls.

2. Start the Job

aws devops-agent create-backlog-task \
  --agent-space-id SPACE_ID \
  --task-type RELEASE_TESTING \
  --title 'Release Testing' \
  --priority MEDIUM \
  --description '{\"testProfileId\": \"<PROFILE_ID>\", \"webhookEventMessage\": \"<REQUIREMENT>\"}' \
  --region us-east-1

If the user provided a test requirement, include it as

webhookEventMessage

. If not, omit the field or leave it empty.

3. Poll for Status

aws devops-agent get-backlog-task \
  --agent-space-id SPACE_ID \
  --task-id TASK_ID \
  --region us-east-1

Poll every 30 seconds until the status transitions to

IN_PROGRESS

or a terminal state (

COMPLETED

FAILED

CANCELED

TIMED_OUT

4. Monitor Until Completion

Once

IN_PROGRESS

, poll for progress in a loop:

aws devops-agent list-journal-records \
  --agent-space-id SPACE_ID \
  --execution-id EXEC_ID \
  --order ASC \
  --region us-east-1

Present each record to the user with a friendly progress update.
Use
```
next_token
```
from the response to fetch only new records on subsequent polls.
Wait 20 seconds between each poll iteration.
Check
```
get-backlog-task
```
periodically — stop when terminal status (
```
COMPLETED
```
,
```
FAILED
```
,
```
CANCELED
```
,
```
TIMED_OUT
```
).

5. Present Results

Once the job reaches a terminal status:

COMPLETED

Retrieve the report using the appropriate record type:

UI testing:
```
--record-type qa_ui_testing_report
```
API testing:
```
--record-type qa_api_testing_report
```

aws devops-agent list-journal-records \
  --agent-space-id SPACE_ID \
  --execution-id EXEC_ID \
  --record-type qa_ui_testing_report \
  --order ASC \
  --region us-east-1

Write the report contents to a markdown file:

release-testing-report-<YYYY-MM-DD-HHmmss>.md

Inform the user that the report was saved, including the file path.

If
```
FAILED
```
or
```
TIMED_OUT
```
: Present the error information and suggest next steps.
If
```
CANCELED
```
: Inform the user the job was canceled and no report is available.

Cancelling (fallback)

aws devops-agent update-backlog-task \
  --agent-space-id SPACE_ID \
  --task-id TASK_ID \
  --task-status CANCELED \
  --region us-east-1

running-release-tests

NPX Install

Tags

SKILL.md Content

Release Testing

Prerequisites

Gathering test parameters

Step 1 — Test profile (required)

Step 2 — Test requirement (optional)

Core workflow

1. Select Agent Space

2. Check tool availability

3. Start the Job

4. Poll for Status

5. Monitor Until Completion

6. Present Results

Cancelling a job

Error handling

Fallback (aws-mcp)

1. Select Agent Space

2. Start the Job

3. Poll for Status

4. Monitor Until Completion

5. Present Results

Cancelling (fallback)