momentic-maintain

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Maintain Momentic tests

维护Momentic测试

Find the earliest point where a run entered a bad state, classify the cause from evidence, and repair the test when a durable test change is appropriate. Use MCP for targeted investigation,
momentic ai classify
for a fresh classifier verdict, and
momentic ai triage
for automated repair and verification.
Do not weaken a test to hide an application bug, missing fixture, invalid credentials, outage, or Momentic defect. Report those causes instead of changing the test to pass.
找到运行进入异常状态的最早节点,根据证据分类故障原因,并在需要持久化测试变更时修复测试。使用MCP进行针对性调查,使用
momentic ai classify
获取全新的分类器结论,使用
momentic ai triage
进行自动化修复和验证。
不得通过弱化测试来掩盖应用程序bug、缺失的测试夹具、无效凭证、服务中断或Momentic缺陷。对于这些情况,应上报原因而非修改测试使其通过。

Common workflows

常见工作流

  • What happened on my branch? Call
    momentic_list_runs
    with the exact
    gitBranchName
    and bounded
    start
    /
    end
    . Start without a status filter, then narrow to
    FAILED
    ,
    CANCELLED
    , or
    recovered=true
    . Open exact candidates and compare with main only for a concrete cross-branch question.
  • Why is main red? List
    FAILED
    runs on the actual main branch over the incident window and paginate through every page needed for the claim. Narrow with
    failureCategory
    when investigating a known category, but group symptoms only after inspecting representative runs and step evidence. Read each exact run's saved classification before optionally re-classifying it.
  • De-flake and reduce recovery latency. Use
    momentic_quarantine_list
    for quarantined tests,
    recovered=true
    for successful runs that needed recovery, and
    minAttempts
    for retry-heavy runs (
    2
    means at least one retry;
    3
    means at least two). Inspect earlier attempts, prioritize repeated same-intent failures, fix the earliest unstable postcondition, and measure again.
Know the discovery limits:
  • momentic_list_runs
    filters by test, branch, status, quarantine, recovery, effective failure category, minimum total attempts, and time.
    failureCategory
    uses a manual classification when present, otherwise the automated classification.
  • momentic_quarantine_list
    filters by quarantine date and returns each active quarantine's latest retained failed CLI run. It does not filter by branch or classification.
  • A category or retry threshold can still span multiple pages. Paginate through all
    pageCount
    pages needed before calling an inventory exhaustive.
  • 我的分支上发生了什么? 使用精确的
    gitBranchName
    和限定的
    start
    /
    end
    参数调用
    momentic_list_runs
    。初始时不设置状态过滤器,之后再缩小范围到
    FAILED
    CANCELLED
    recovered=true
    。仅在需要明确跨分支对比时,才打开候选记录并与主分支进行比较。
  • 主分支为何报错? 列出事件窗口内主分支上的
    FAILED
    运行记录,遍历所有必要的分页结果。在调查已知类别时可通过
    failureCategory
    缩小范围,但需先检查代表性运行记录和步骤证据后再归纳症状。在选择重新分类前,先查看每条运行记录已保存的分类结果。
  • 修复抖动问题并缩短恢复延迟 使用
    momentic_quarantine_list
    查看隔离测试,使用
    recovered=true
    查看需要恢复才能成功的运行记录,使用
    minAttempts
    查看重试频繁的运行记录(
    2
    表示至少重试1次;
    3
    表示至少重试2次)。检查早期尝试记录,优先处理重复出现的同类型故障,修复最早出现的不稳定后置条件,然后再次验证效果。
了解查询限制:
  • momentic_list_runs
    可按测试、分支、状态、隔离状态、恢复状态、实际故障类别、最小尝试次数和时间进行过滤。
    failureCategory
    优先使用手动分类结果,若无则使用自动分类结果。
  • momentic_quarantine_list
    可按隔离日期过滤,返回每个有效隔离测试的最新保留失败CLI运行记录。该工具不支持按分支或分类过滤。
  • 按类别或重试阈值过滤时仍可能涉及多页结果,需遍历所有
    pageCount
    页才能确保查询全面。

Maintenance workflow

维护工作流

  1. Start from the exact run, branch, run group, commit, or local results target.
  2. Inspect existing automated/manual classifications and heal state first.
  3. Reconstruct exact failures with MCP and only the necessary run history.
  4. Explain the earliest divergence and broken postcondition before editing.
  5. Classify afresh only when useful; persist a verdict only when requested.
  6. Prepare the exact app and fixtures, then triage when repair is appropriate.
  7. Review the diff and replay evidence; re-run inconclusive repairs.
  8. Report diagnosis, repair, verification, delivery, and remaining blockers.
  1. 从指定的运行记录、分支、运行组、提交记录或本地结果目标开始。
  2. 先检查现有的自动/手动分类结果并修复异常状态。
  3. 使用MCP和必要的运行历史重现精确故障。
  4. 在编辑测试前,先解释最早出现的偏差和损坏的后置条件。
  5. 仅在有用时重新分类,仅在用户要求时保存结论。
  6. 准备好精确的应用程序和测试夹具,在适合修复时执行分流操作。
  7. 审查差异并回放证据,重新执行不确定的修复操作。
  8. 上报诊断结果、修复方案、验证情况、交付进度及剩余障碍。

Inspect the existing classification

检查现有分类结果

Fetch raw run metadata before MCP investigation when an API key is available:
bash
RUN_ID="<run-id>"
MOMENTIC_SERVER_URL="${MOMENTIC_SERVER:-https://api.momentic.ai}"
RUN_INSPECTION_DIR="$(mktemp -d)"

curl --fail --silent --show-error \
  -H "Authorization: Bearer ${MOMENTIC_API_KEY:?Set MOMENTIC_API_KEY}" \
  "${MOMENTIC_SERVER_URL}/v1/runs/${RUN_ID}/metadata" \
  -o "${RUN_INSPECTION_DIR}/metadata.json"

jq '{
  runId: .id,
  status,
  failureReason,
  classification: .failureDetails.classification,
  manualClassification: .failureDetails.manualClassification,
  healStatus,
  healDetails
}' "${RUN_INSPECTION_DIR}/metadata.json"
Do not treat a missing field as proof that no human reviewed the failure until the metadata request succeeds for the exact run and organization. Do not echo, log, or commit the API key. Treat downloaded metadata and signed artifact URLs as sensitive run data.
momentic_get_run
intentionally removes prior classification and heal fields from the evidence it returns so a fresh investigation is not anchored by an old verdict. Use the metadata endpoint or dashboard for the existing verdict, then use MCP for the underlying evidence.
当有API密钥时,在使用MCP调查前先获取原始运行元数据:
bash
RUN_ID="<run-id>"
MOMENTIC_SERVER_URL="${MOMENTIC_SERVER:-https://api.momentic.ai}"
RUN_INSPECTION_DIR="$(mktemp -d)"

curl --fail --silent --show-error \
  -H "Authorization: Bearer ${MOMENTIC_API_KEY:?Set MOMENTIC_API_KEY}" \
  "${MOMENTIC_SERVER_URL}/v1/runs/${RUN_ID}/metadata" \
  -o "${RUN_INSPECTION_DIR}/metadata.json"

jq '{
  runId: .id,
  status,
  failureReason,
  classification: .failureDetails.classification,
  manualClassification: .failureDetails.manualClassification,
  healStatus,
  healDetails
}' "${RUN_INSPECTION_DIR}/metadata.json"
在确认指定运行记录和组织的元数据请求成功前,不要将缺失字段视为无人审核故障的证据。不要回显、记录或提交API密钥,将下载的元数据和签名工件URL视为敏感运行数据。
momentic_get_run
会故意从返回的证据中移除之前的分类和修复字段,以便全新调查不受旧结论影响。如需查看现有结论,请使用元数据端点或控制台,使用MCP获取底层证据。

Helpful MCP tools

实用MCP工具

momentic_get_run
- Return run metadata and a summary of the full result tree. Use it to identify the attempt, failed section, failing step, and nested parent chain. If the current run is already loaded, call it again only for a different attempt or missing data.
momentic_list_runs
- List recent runs for the test. Always pass
gitBranchName
when it exists on the target run so comparisons are more likely to use the same test version. Omit it only when cross-branch history answers a specific question. Pass
recovered=true
to inspect recovered runs. Pass
failureCategory
to select effective manual/automated classifications and
minAttempts: 2
or higher to find retried runs.
momentic_quarantine_list
- List active local test quarantines and each test's latest retained failed CLI run. Use a bounded quarantine-date range.
momentic_get_step_result
- Return command details, before/after screenshots, and optionally the full trace for one step. Pass
parentStepIdChain
for nested steps. Request
includeTrace=true
only when screenshots and normal fields do not answer the question.
momentic_get_test_steps_for_run
- Return the simplified test definition recorded on a run. Use it when test intent remains unclear or when comparing whether the authored test changed between runs.
momentic_submit_result_classification
- Persist a manual MCP classification. Call it only after the investigation is complete and only when the user asked to record the verdict. Do not call it in addition to
ai classify --save
for the same conclusion.
momentic_get_run
- 返回运行元数据和完整结果树的摘要。用于识别尝试次数、失败区段、失败步骤及嵌套父级链。若当前运行记录已加载,仅在需要获取不同尝试次数或缺失数据时再次调用。
momentic_list_runs
- 列出测试的近期运行记录。当目标运行记录存在
gitBranchName
时,务必传入该参数,以便对比时更可能使用相同测试版本。仅在跨分支历史能解答特定问题时才省略该参数。传入
recovered=true
可查看已恢复的运行记录,传入
failureCategory
可筛选手动/自动分类结果,传入
minAttempts: 2
或更高值可找到重试过的运行记录。
momentic_quarantine_list
- 列出本地有效隔离测试及每个测试的最新保留失败CLI运行记录。可使用限定的隔离日期范围。
momentic_get_step_result
- 返回单个步骤的命令详情、前后截图及可选的完整跟踪信息。对于嵌套步骤,需传入
parentStepIdChain
。仅在截图和常规字段无法解答问题时,才请求
includeTrace=true
momentic_get_test_steps_for_run
- 返回运行记录中保存的简化测试定义。当测试意图不明确或需要对比运行记录间测试是否变更时使用。
momentic_submit_result_classification
- 保存手动MCP分类结果。仅在调查完成且用户要求记录结论时调用,不要针对同一结论同时调用该工具和
ai classify --save

Investigation workflow

调查工作流

Start with the current run before relying on history.
  1. Call
    momentic_get_run
    and identify the failing attempt, section (
    beforeSteps
    , main steps, or
    afterSteps
    ), failing step, and any
    parentStepIdChain
    .
  2. Pull the failing step result with screenshots and trace. If the step is nested, also pull the nearest parent container or module result.
  3. Decide whether the failing step's before-screenshot is the correct baseline for that action. If it is already wrong, walk backward through the current run until you find the step/container that produced that bad state.
  4. For repeated modules or repeated workflows, compare invocations inside the same current run before comparing older runs. The later failure is often caused by an earlier invocation that succeeded, recovered, or left an invalid postcondition.
  5. Treat successful containers with failed or recovered child steps as partial failures until you inspect the container's final after-screenshot and URL.
  6. Use past runs only for specific comparison questions once the current-run behavior is understood.
Before classifying, be able to answer:
  • What is the test's intended behavior?
  • What is the earliest divergent step/container?
  • What did that step intend to do?
  • Which element/state did it actually interact with or observe?
  • What changed in the screenshot, URL, DOM, trace, or recovery log after the step?
  • Why is the later failure a consequence of that earlier divergence?
Avoid vague root causes such as "setup was unreliable" or "the page was in the wrong state." Name the broken postcondition directly: for example, "the row-level plus button was clicked, but the app stayed on the parent page instead of opening the child-page editor; the following global
Add to
assertion passed against unrelated page text, so the untargeted type step never entered the child title."
先从当前运行记录入手,再依赖历史记录。
  1. 调用
    momentic_get_run
    ,识别失败尝试次数、失败区段(
    beforeSteps
    、主步骤或
    afterSteps
    )、失败步骤及任何
    parentStepIdChain
  2. 获取失败步骤的结果(含截图和跟踪信息)。若步骤为嵌套结构,还需获取最近的父容器或模块结果。
  3. 判断失败步骤的前置截图是否为该操作的正确基准状态。若已异常,则回溯当前运行记录,直到找到产生该异常状态的步骤/容器。
  4. 对于重复模块或重复工作流,先对比当前运行记录内的多次调用,再对比旧运行记录。后续故障通常由之前成功、恢复或留下无效后置条件的调用导致。
  5. 将包含失败或已恢复子步骤的成功容器视为部分故障,需检查容器的最终后置截图和URL。
  6. 仅在理解当前运行记录行为后,才使用历史记录进行特定对比。
分类前需明确以下问题:
  • 测试的预期行为是什么?
  • 最早出现偏差的步骤/容器是什么?
  • 该步骤的预期操作是什么?
  • 实际交互或观察到的元素/状态是什么?
  • 步骤执行后,截图、URL、DOM、跟踪信息或恢复日志有何变化?
  • 后续故障为何是早期偏差的结果?
避免模糊的根本原因描述,如“设置不可靠”或“页面状态错误”。直接明确损坏的后置条件,例如:“点击行级加号按钮后,应用未跳转到子页面编辑器,仍停留在父页面;后续全局‘添加到’断言匹配了无关页面文本,导致无目标的输入步骤从未进入子页面标题栏。”

Evidence standards

证据标准

  • Screenshots are the default truth source for page state. Use trace fields and DOM/HTML to explain why the screenshot changed or did not change.
  • Verify every causal claim. Do not say an overlay, side peek, modal, or menu was present unless the relevant before/after screenshot, URL, or DOM proves it.
  • Separate "the target is missing now" from "the browser is in the state where that target should exist." A missing target is often a symptom of an earlier failed action.
  • For click/type/action steps, record the intended action, actual interacted element when available, before/after URL, and whether the expected UI state appeared.
  • For assertions, check whether the assertion is scoped enough to prove the intended state. A broad page-content assertion can pass for unrelated text.
  • For recovery, inspect both the failed child step and the recovered container final state. Recovery can pass a retried assertion while leaving state that later steps did not expect.
  • 截图是页面状态的默认可信来源。使用跟踪字段和DOM/HTML解释截图变化或未变化的原因。
  • 验证所有因果主张。除非相关前后截图、URL或DOM能证明,否则不要断言存在覆盖层、侧边弹窗、模态框或菜单。
  • 区分“目标元素当前缺失”和“浏览器处于目标元素应存在的状态”。目标元素缺失通常是早期操作失败的症状。
  • 对于点击/输入/操作步骤,记录预期操作、实际交互元素(若可用)、前后URL及预期UI状态是否出现。
  • 对于断言,检查断言范围是否足够证明预期状态。宽泛的页面内容断言可能因无关文本而通过。
  • 对于恢复操作,检查失败子步骤和恢复后容器的最终状态。恢复操作可能通过重试断言,但留下后续步骤未预期的状态。

Run AI classification

执行AI分类

momentic ai classify
runs Momentic's result-classification agent over failed runs. It reads the run artifacts, uses relevant run history and repository context, and returns a category, recoverability, confidence, reasoning, and citations. It explains the failure; it does not edit the test.
Use one selector per command:
bash
npx momentic ai classify --run-id "$RUN_ID" --output-format json
npx momentic ai classify --run-group-id "<run-group-id>"
npx momentic ai classify --git-commit "<commit-sha>"
  • Omit
    --save
    for a diagnostic re-classification. Add
    --save
    only when the new verdict should replace the saved classification on the local archive or cloud run.
  • Use
    --no-cache
    when the existing cached classifier result is suspected to be stale and the user explicitly needs a fresh analysis.
  • Use
    --skip-classified
    when resuming a batch without revisiting runs that already have saved classifications.
  • Use
    --output-format json
    for automation and multi-run bookkeeping.
  • Classify multiple related run IDs together when useful, but keep each result mapped to its exact run.
For one run in a real terminal, keep the classifier conversation open:
bash
npx momentic ai classify --run-id "$RUN_ID" --interactive
Use the chat to ask why it chose the category, what evidence it relied on, how the current run differs from a prior run, or what a durable fix would require. Interactive classification is single-run only, requires stdin to be a TTY, and cannot be combined with JSON output.
momentic ai classify
通过Momentic的结果分类Agent分析失败运行记录。它读取运行工件,结合相关运行历史和仓库上下文,返回类别、可恢复性、置信度、推理过程及引用来源。该工具仅解释故障,不编辑测试。
每个命令使用一个选择器:
bash
npx momentic ai classify --run-id "$RUN_ID" --output-format json
npx momentic ai classify --run-group-id "<run-group-id>"
npx momentic ai classify --git-commit "<commit-sha>"
  • 诊断性重新分类时省略
    --save
    。仅当新结论应替换本地归档或云端运行记录中的已保存分类结果时,才添加
    --save
  • 当怀疑现有缓存分类结果过时且用户明确需要全新分析时,使用
    --no-cache
  • 恢复批量处理时,使用
    --skip-classified
    避免重新处理已保存分类结果的运行记录。
  • 自动化处理和多运行记录管理时,使用
    --output-format json
  • 必要时可同时分类多个相关运行ID,但需确保每个结果对应其精确运行记录。
在真实终端中处理单个运行记录时,保持分类会话开启:
bash
npx momentic ai classify --run-id "$RUN_ID" --interactive
通过聊天询问分类器为何选择该类别、依赖哪些证据、当前运行记录与之前运行记录的差异,以及持久化修复需要什么条件。交互式分类仅支持单个运行记录,要求标准输入为TTY,且无法与JSON输出结合使用。

Prepare triage

准备分流操作

Treat application readiness as a hard precondition. Triage replays the test in a real browser; it cannot repair a test against an unavailable app, missing fixture, wrong environment, unseeded database, invalid account, or cold compiler.
Before triage:
  1. Read
    momentic.config.yaml
    , the target test and its setup modules, repository scripts, and the CI workflow that normally runs the test. Use the same URL, environment, credentials, headers, files, services, and data fixtures.
  2. Confirm
    MOMENTIC_API_KEY
    , project configuration, dependencies, and the requested browser are available.
  3. Inspect
    git status
    and preserve unrelated work. Decide the repair delivery before running the command; do not let an unknown dashboard default create a pull request or commit unexpectedly.
  4. Build the application before starting triage. Prefer a production or otherwise precompiled build and a stable server over Next.js, webpack, Vite, or another development server that compiles routes incrementally.
  5. Start every required service and wait for the actual readiness URL. Warm any unavoidable dev-only compilation path before triage and verify that the relevant route no longer shows a compile/loading screen or incurs a long first request.
  6. Run a focused smoke or the failing test once when necessary to prove the app, fixture, authentication, and browser are ready. Fix setup failures before blaming triage.
  7. Start with
    --parallel 1
    . Raise it only when the machine has enough CPU and memory for one browser and application workload per worker.
If the repository exposes a stable server command, either start it separately or let triage own it with
--start
and
--wait-on
:
bash
undefined
将应用程序就绪视为硬性前提条件。分流操作会在真实浏览器中重放测试,无法针对不可用的应用程序、缺失的测试夹具、错误环境、未初始化数据库、无效账户或冷编译状态修复测试。
分流前需完成:
  1. 阅读
    momentic.config.yaml
    、目标测试及其设置模块、仓库脚本和通常运行测试的CI工作流。使用相同的URL、环境、凭证、请求头、文件、服务和数据夹具。
  2. 确认
    MOMENTIC_API_KEY
    、项目配置、依赖项及请求的浏览器可用。
  3. 检查
    git status
    并保留无关工作。运行命令前确定修复交付方式,避免因未知控制台默认设置意外创建拉取请求或提交记录。
  4. 执行分流前先构建应用程序。优先使用生产环境或预编译构建及稳定服务器,而非Next.js、webpack、Vite等会增量编译路由的开发服务器。
  5. 启动所有必需服务并等待就绪URL。若无法避免仅开发环境可用的编译路径,需提前预热并验证相关路由不再显示编译/加载界面或出现首次请求延迟过长的情况。
  6. 必要时先运行一次聚焦的冒烟测试或失败测试,证明应用程序、测试夹具、认证及浏览器已就绪。在归咎于分流操作前先修复设置故障。
  7. 初始使用
    --parallel 1
    。仅当机器有足够CPU和内存支持每个工作进程处理一个浏览器和应用程序工作负载时,再提高并行数。
若仓库提供稳定的服务器启动命令,可单独启动或通过
--start
--wait-on
让分流操作管理:
bash
undefined

Build first; use the repository's actual commands and readiness URL.

先构建;使用仓库实际的命令和就绪URL。

pnpm build npx momentic ai triage --run-id "$RUN_ID"
--start "pnpm start"
--wait-on "http://localhost:3000/health"
--parallel 1
--yes
--on-heal-success nothing

Do not copy these example commands blindly. Discover the real build, start, and
readiness contract from the repository and CI configuration.
pnpm build npx momentic ai triage --run-id "$RUN_ID"
--start "pnpm start"
--wait-on "http://localhost:3000/health"
--parallel 1
--yes
--on-heal-success nothing

不要盲目复制示例命令。从仓库和CI配置中了解真实的构建、启动和就绪约定。

Run AI triage

执行AI分流

momentic ai triage
groups failures by shared root cause, decides which ones are repairable, edits the relevant tests, replays them in a browser, and accepts only repairs that satisfy its verification. It can leave changes locally, produce a patch, commit, or open a pull request. It cannot fix an application bug, external outage, or missing fixture by changing a test.
Run classification first when the user wants an explicit diagnosis, then pass the same target to triage:
bash
npx momentic ai classify --run-id "$RUN_ID" --output-format json
npx momentic ai triage --run-id "$RUN_ID" \
  --parallel 1 \
  --yes \
  --on-heal-success nothing
For a local run group, pass the results directory instead:
bash
npx momentic ai triage ./test-results \
  --parallel 1 \
  --yes \
  --on-heal-success nothing
Best practices:
  • Pass multiple related run IDs after one
    --run-id
    so the bucketing agent can group shared causes. Do not launch one competing triage command per run.
  • Set
    --on-heal-success
    explicitly: use
    nothing
    for local review,
    patch
    for a portable diff, or a pull-request/commit behavior only when the user asked for that delivery.
  • Pass
    --yes
    only after the environment and delivery choice are confirmed so dependency prompts do not stall an autonomous run.
  • Use
    --dry-run --no-save
    to inspect bucketing without editing tests or saving triage metadata. Dry-run is not a repair attempt.
  • Use
    --regenerate-heal
    only when a cached heal is stale or failed and a new repair should be generated from scratch.
  • Keep the command attached and monitor reasoning, tool calls, browser progress, CPU, and memory. Several minutes without browser progress can indicate app startup, cold compilation, resource pressure, or a missing fixture.
  • Inspect every resulting diff. Do not accept deleted assertions, broadened checks, arbitrary sleeps, or longer timeouts that merely conceal the cause.
  • Do not remove quarantine solely because triage produced a patch. Re-run the repaired test first.
Use interactive triage when the user wants to question the repair agent:
bash
npx momentic ai triage --run-id "$RUN_ID" \
  --parallel 1 \
  --yes \
  --on-heal-success nothing \
  --interactive
Interactive triage requires a TTY, cannot be combined with
--json
, and is available only when the command runs the bucketing agent. Use it to ask why a fix was accepted or rejected, what setup it observed, and what evidence remains unresolved.
momentic ai triage
按共同根本原因分组故障,判断哪些故障可修复,编辑相关测试,在浏览器中重放测试,仅接受通过验证的修复。它可将变更保留在本地、生成补丁、提交或打开拉取请求。该工具无法通过修改测试修复应用程序bug、外部服务中断或缺失的测试夹具。
当用户需要明确诊断时,先执行分类,再将同一目标传入分流操作:
bash
npx momentic ai classify --run-id "$RUN_ID" --output-format json
npx momentic ai triage --run-id "$RUN_ID" \
  --parallel 1 \
  --yes \
  --on-heal-success nothing
对于本地运行组,传入结果目录即可:
bash
npx momentic ai triage ./test-results \
  --parallel 1 \
  --yes \
  --on-heal-success nothing
最佳实践:
  • 在一个
    --run-id
    后传入多个相关运行ID,以便分组Agent按共同原因分组。不要为每个运行记录启动独立的分流命令。
  • 显式设置
    --on-heal-success
    :本地审核使用
    nothing
    ,可移植差异使用
    patch
    ,仅当用户要求时才选择拉取请求/提交行为。
  • 确认环境和交付方式后再传入
    --yes
    ,避免依赖提示中断自动化运行。
  • 使用
    --dry-run --no-save
    检查分组情况,不编辑测试或保存分流元数据。试运行不执行修复尝试。
  • 当缓存修复过时或失败且需要从头生成新修复时,使用
    --regenerate-heal
  • 保持命令运行并监控推理过程、工具调用、浏览器进度、CPU和内存使用情况。数分钟无浏览器进度可能表明应用程序启动、冷编译、资源压力或缺失测试夹具。
  • 检查所有生成的差异。不要接受删除断言、扩大检查范围、任意添加等待或延长超时时间等仅掩盖问题的修改。
  • 不要仅因分流操作生成补丁就解除隔离,需先重新运行修复后的测试。
当用户希望向修复Agent提问时,使用交互式分流:
bash
npx momentic ai triage --run-id "$RUN_ID" \
  --parallel 1 \
  --yes \
  --on-heal-success nothing \
  --interactive
交互式分流需要TTY,无法与
--json
结合使用,仅当命令运行分组Agent时可用。可通过它询问修复被接受或拒绝的原因、观察到的设置情况及未解决的证据。

Download run artifacts manually

手动下载运行工件

Prefer MCP for ordinary inspection because
momentic_get_run
downloads and extracts cloud attempts automatically. Use the API when MCP output is missing, when raw files must be grepped, or when independently verifying the agent.
The export endpoint returns run metadata plus one short-lived signed ZIP URL per attempt; it does not return one combined run ZIP. Download and extract them without printing the URLs:
bash
RUN_ID="<run-id>"
MOMENTIC_SERVER_URL="${MOMENTIC_SERVER:-https://api.momentic.ai}"
RUN_INSPECTION_DIR="$(mktemp -d)"

curl --fail --silent --show-error \
  -H "Authorization: Bearer ${MOMENTIC_API_KEY:?Set MOMENTIC_API_KEY}" \
  "${MOMENTIC_SERVER_URL}/v1/runs/${RUN_ID}/export" \
  -o "${RUN_INSPECTION_DIR}/export.json"

jq '.run' "${RUN_INSPECTION_DIR}/export.json" \
  > "${RUN_INSPECTION_DIR}/metadata.json"

jq -r '.attemptZipUrls | to_entries[] | select(.value != null) |
  [(.key + 1), .value] | @tsv' "${RUN_INSPECTION_DIR}/export.json" |
  while IFS=$'\t' read -r attempt url; do
    mkdir -p "${RUN_INSPECTION_DIR}/attempts/${attempt}"
    curl --fail --silent --show-error "$url" \
      -o "${RUN_INSPECTION_DIR}/attempt-${attempt}.zip"
    unzip -q "${RUN_INSPECTION_DIR}/attempt-${attempt}.zip" \
      -d "${RUN_INSPECTION_DIR}/attempts/${attempt}"
  done
Inspect
metadata.json
, each attempt's
metadata.json
, screenshots, DOM snapshots, console logs, HAR logs, video, resource usage, and crash archives. Keep attempt numbers intact. Do not commit the export response, run artifacts, signed URLs, credentials, or customer data. Remove the temporary directory when the investigation is complete.
常规检查优先使用MCP,因为
momentic_get_run
会自动下载并提取云端尝试记录。当MCP输出缺失、需要搜索原始文件或独立验证Agent时,使用API。
导出端点返回运行元数据及每个尝试记录的短期签名ZIP URL;不返回合并的运行ZIP文件。下载并提取文件时不要打印URL:
bash
RUN_ID="<run-id>"
MOMENTIC_SERVER_URL="${MOMENTIC_SERVER:-https://api.momentic.ai}"
RUN_INSPECTION_DIR="$(mktemp -d)"

curl --fail --silent --show-error \
  -H "Authorization: Bearer ${MOMENTIC_API_KEY:?Set MOMENTIC_API_KEY}" \
  "${MOMENTIC_SERVER_URL}/v1/runs/${RUN_ID}/export" \
  -o "${RUN_INSPECTION_DIR}/export.json"

jq '.run' "${RUN_INSPECTION_DIR}/export.json" \
  > "${RUN_INSPECTION_DIR}/metadata.json"

jq -r '.attemptZipUrls | to_entries[] | select(.value != null) |
  [(.key + 1), .value] | @tsv' "${RUN_INSPECTION_DIR}/export.json" |
  while IFS=$'\t' read -r attempt url; do
    mkdir -p "${RUN_INSPECTION_DIR}/attempts/${attempt}"
    curl --fail --silent --show-error "$url" \
      -o "${RUN_INSPECTION_DIR}/attempt-${attempt}.zip"
    unzip -q "${RUN_INSPECTION_DIR}/attempt-${attempt}.zip" \
      -d "${RUN_INSPECTION_DIR}/attempts/${attempt}"
  done
检查
metadata.json
、每个尝试记录的
metadata.json
、截图、DOM快照、控制台日志、HAR日志、视频、资源使用情况及崩溃归档。保留尝试编号不变。不要提交导出响应、运行工件、签名URL、凭证或客户数据。调查完成后删除临时目录。

Background

背景知识

Test run result structure

测试运行结果结构

When momentic tests are run via the CLI, the results are stored in a "run group". The data for this run group is stored in a single directory within the momentic project. By default, the directory is called
test-results
, but can be changed in momentic project settings or on a single run of a run group. The run group results folder has the following structure:
test-results/
├── metadata.json         data about the run group, including git metadata and timing info.
└── runs/                 On zip for each test run in the run group.
    ├── <runId_1>.zip         a zipped run directory containing data about this specific test run.  Follows the structure described below.
    └── <runId_2>.zip
When unzipped, run directories have the following structure:
<runId>/
├── metadata.json           run-level metadata.
└── attempts/<n>/           one folder per attempt (1-based n).
    ├── metadata.json       attempt outcome and step results.
    ├── console.json        optional browser console output.
    └── assets/
        ├── <snapshotId>.jpeg     before/after screenshot for each step (see attempt metadata.json for snapshot ID).
        ├── <snapshotId>.html     before/after DOM snapshot for each step (see attempt metadata.json for snapshot ID).
        ├── har-pages.log         HAR pages (ndjson).
        ├── har-entries.log       HAR network entries (ndjson).
        ├── resource-usage.ndjson CPU/memory samples taken during the attempt.
        ├── <videoName>           video recording (when video recording is enabled).
        └── browser-crash.zip     browser crash dump (only present on crash).
When getting run results via the momentic MCP, tools such as
momentic_get_run
will return links to the MCP working directory (default
.momentic-mcp
). This directory will contain unzipped run result folders, following the structure above, named
run-result-<runId>
.
通过CLI运行Momentic测试时,结果存储在“运行组”中。运行组的数据存储在Momentic项目内的单个目录中。默认目录名为
test-results
,可在Momentic项目设置或单次运行组时修改。运行组结果目录结构如下:
test-results/
├── metadata.json         运行组相关数据,包括Git元数据和时间信息。
└── runs/                 运行组中每个测试运行的ZIP文件。
    ├── <runId_1>.zip         包含该测试运行具体数据的压缩目录,结构如下所述。
    └── <runId_2>.zip
解压后,运行目录结构如下:
<runId>/
├── metadata.json           运行级元数据。
└── attempts/<n>/           每个尝试记录对应一个文件夹(n从1开始)。
    ├── metadata.json       尝试结果和步骤结果。
    ├── console.json        可选的浏览器控制台输出。
    └── assets/
        ├── <snapshotId>.jpeg     每个步骤的前后截图(查看尝试记录metadata.json获取快照ID)。
        ├── <snapshotId>.html     每个步骤的前后DOM快照(查看尝试记录metadata.json获取快照ID)。
        ├── har-pages.log         HAR页面(ndjson格式)。
        ├── har-entries.log       HAR网络请求记录(ndjson格式)。
        ├── resource-usage.ndjson 尝试过程中的CPU/内存采样数据。
        ├── <videoName>           视频录制(启用视频录制时存在)。
        └── browser-crash.zip     浏览器崩溃转储(仅崩溃时存在)。
通过Momentic MCP获取运行结果时,
momentic_get_run
等工具会返回指向MCP工作目录(默认
.momentic-mcp
)的链接。该目录包含解压后的运行结果文件夹,命名为
run-result-<runId>
,结构如上所述。

Element locators

元素定位器

Certain step types that interact with elements have a "target" property, or locator, that specifies which element the step should interact with.
某些与元素交互的步骤类型具有“target”属性(即定位器),用于指定步骤应交互的元素。

Locator caches

定位器缓存

Locators identify elements by sending the page state html/xml to an llm as well as a screenshot. The llm identifies which element on the page the user is referring to. Momentic will attempt to "cache" the answer from the llm so that future runs don't require AI calls. On future runs, the page state is checked against the cached element to determine whether the element is still usable, or the page has changed enough such that another AI call is required.
A locator cache can bust for a variety of reasons:
  • the element description has changed, in which case we'll always bust the cache
  • the cached element could not be located in the current page state
  • the cached element was located in the page state, but fails certain checks specified on the cache entry, such as requiring a certain position, shape, or content.
You can find the
cacheBustReason
on the
trace
property in the results for a given step, but only when you explicitly request
includeTrace=true
. The
cache
property is also listed on the results, showing the full cache saved for that element.
定位器通过将页面状态HTML/XML及截图发送给LLM来识别元素,LLM会识别用户指向的页面元素。Momentic会尝试缓存LLM的响应,以便后续运行无需调用AI。后续运行时,会检查当前页面状态与缓存元素是否匹配,判断元素是否仍可用,或页面变化是否需要再次调用AI。
定位器缓存失效的原因包括:
  • 元素描述已变更,此时缓存必然失效
  • 缓存元素无法在当前页面状态中找到
  • 缓存元素可在当前页面状态中找到,但未通过缓存条目指定的某些检查,如位置、形状或内容要求
可在步骤结果的
trace
属性中找到
cacheBustReason
,但需显式请求
includeTrace=true
。结果中还会列出
cache
属性,显示该元素的完整缓存内容。

Identifying bad caches

识别无效缓存

Sometimes the element that was cached is not the element that the user intended to target. This can cause failures or unexpected behaviors in tests. In these cases, it helps to verify exactly why the wrong cache was saved in the first place. Only request
includeTrace=true
for these cache-debugging cases or when you suspect incorrect Momentic execution data. Use the
runId
property of the
targetUpdateLoggerTags
on the incorrect cache to get the details of the original run, calling
momentic_get_run
with this runId. This will return the run where the cache target was updated.
有时缓存的元素并非用户预期的目标元素,这会导致测试失败或行为异常。此时需验证错误缓存最初被保存的原因。仅在调试缓存或怀疑Momentic执行数据错误时,才请求
includeTrace=true
。使用错误缓存的
targetUpdateLoggerTags
中的
runId
属性获取原始运行记录详情,调用
momentic_get_run
传入该runId,即可返回缓存目标被更新的运行记录。

Module caching

模块缓存

Cached modules skip executing their steps when the module cache key and resolved inputs are unchanged, and reuse the cached return value from the module's last step.
Authentication modules can also save and restore browser auth state from the module cache, including cookies, localStorage, and IndexedDB. They may use a page-content check after restoring auth state to decide whether the cache is still valid.
当模块缓存键和解析输入未变更时,缓存模块会跳过执行步骤,重用模块最后一步的缓存返回值。
认证模块还可从模块缓存中保存和恢复浏览器认证状态,包括Cookie、localStorage和IndexedDB。恢复认证状态后,模块可能通过页面内容检查判断缓存是否仍有效。

File uploads

文件上传

A file upload step prepares one file for the next native file picker, so it must run before the action that opens the picker.
Sources can be remote URLs,
file://
references to earlier downloads, CLI-local paths, or uploaded user files. The step can also override the presented filename, and Momentic wires the prepared file into the browser's file chooser handling.
文件上传步骤会为下一个原生文件选择器准备一个文件,因此必须在打开选择器的操作之前运行。
文件来源可以是远程URL、
file://
协议引用的早期下载文件、CLI本地路径或用户上传的文件。该步骤还可覆盖显示的文件名,Momentic会将准备好的文件关联到浏览器的文件选择器处理逻辑中。

Using past runs

使用历史运行记录

Past runs are comparison evidence, not a substitute for reconstructing the current run. Use them when the current run does not answer:
  • When did this test start failing?
  • What differed vs the last passing run?
  • Did the same action behave differently on an earlier run?
  • Is this a test weakness, an application change, a real application bug, or a temporary slowdown?
Use step results and screenshots on past runs to answer these questions. Do NOT rely only on summaries from
momentic_get_run
or
momentic_list_runs
to understand what happened in a test run. Look at the specific run details, including step results and screenshots, before citing a past run as evidence.
When looking at past runs, use the following workflow:
  1. Call the
    momentic_list_runs
    tool to identify the runs you want more detail on. Always pass
    gitBranchName
    when it exists on the run in question. Omit it when you need runs from other branches.
  2. Call
    momentic_get_run
    for that specific run to get the run details.
  3. Call
    momentic_get_step_result
    for the same step/container or closest equivalent you are comparing, especially for screenshots.
When past runs are irrelevant because the current run already proves the root cause, say that briefly instead of forcing historical evidence.
历史运行记录仅作为对比证据,不能替代对当前运行记录的重构。仅在当前运行记录无法解答以下问题时使用:
  • 该测试何时开始失败?
  • 与上次成功运行记录有何差异?
  • 同一操作在早期运行记录中的表现是否不同?
  • 这是测试缺陷、应用程序变更、真实应用程序bug还是临时性能下降?
使用历史运行记录的步骤结果和截图解答这些问题。不要仅依赖
momentic_get_run
momentic_list_runs
的摘要来理解测试运行情况。在引用历史运行记录作为证据前,需查看具体的运行详情,包括步骤结果和截图。
查看历史运行记录时,遵循以下工作流:
  1. 调用
    momentic_list_runs
    工具确定需要查看详情的运行记录。当运行记录存在
    gitBranchName
    时,务必传入该参数。仅在需要查看其他分支的运行记录时才省略。
  2. 调用
    momentic_get_run
    获取该运行记录的详情。
  3. 调用
    momentic_get_step_result
    获取要对比的相同步骤/容器或最接近的等效项,尤其是截图。
当当前运行记录已能证明根本原因,历史运行记录无关时,可简要说明,无需强行提供历史证据。

Multi-attempt runs

多尝试运行记录

When
momentic_list_runs
shows a passing run with
attempts > 1
, treat it as a partial failure worth investigating, not a clean passing run. Use the
attemptNumber
parameter to retrieve earlier failed attempt results for that run to understand what was going wrong before the retry succeeded.
momentic_list_runs
显示
attempts > 1
的成功运行记录时,需将其视为值得调查的部分失败,而非完全成功的运行记录。使用
attemptNumber
参数获取该运行记录的早期失败尝试结果,了解重试成功前存在的问题。

Flakiness and intermittent failures

抖动和间歇性故障

  • Call a test flaky only when the same app and test behavior fails intermittently. One failure is insufficient; rule out application and test changes using run evidence before attributing the pattern to flakiness.
  • 仅当相同应用程序和测试行为间歇性失败时,才称测试存在抖动。单次失败不足以判定;需通过运行证据排除应用程序和测试变更后,再将该模式归因于抖动。

Test temporality

测试时效性

  • Any past results may not necessarily match today’s test file. The test may have changed, meaning the result was on a different version of the test.
  • You can call
    get_test_steps_for_run
    to help you determine if the test itself changed between runs, although note that this tool returns a summary of each test step. If you suspect that specific details on certain steps have changed between test runs, full step details are included in the response from
    momentic_get_step_result
    ; only request
    includeTrace=true
    when those fields and screenshots still are not enough.
  • 任何历史结果可能与当前测试文件不匹配。测试可能已变更,意味着历史结果基于不同版本的测试。
  • 可调用
    get_test_steps_for_run
    帮助判断运行记录间测试是否变更,但需注意该工具返回的是每个测试步骤的摘要。若怀疑特定步骤的具体细节在运行记录间发生变化,
    momentic_get_step_result
    的响应中包含完整步骤详情;仅当这些字段和截图仍不足以解答时,才请求
    includeTrace=true

Common failure modes to watch for

需要关注的常见故障模式

  • A setup module appears to pass but leaves the wrong page, overlay, filter, search, selected row, or side peek open. Classify from the step/container that left the bad postcondition, not only from the next step that failed.
  • A click reports success and targets the intended element, but the application does not transition to the intended state. Verify the post-state; do not assume the click worked because the locator was correct.
  • A weak global assertion such as "page contains X" passes because unrelated text on the page matches. The next step may then type or click in the wrong context.
  • A type step without a specific target can silently type nowhere useful if the preceding action failed to focus the intended field.
  • A locator or cache can be technically valid but semantically wrong. Check the interacted element and, for bad caches, inspect the original cache-update run from
    targetUpdateLoggerTags.runId
    .
  • A recovered step can hide the first failure. Inspect failed child steps inside recovered modules and compare the recovered final state to the next step's expected baseline.
  • A timeout is not automatically
    INFRA
    . First rule out missing data, wrong page state, changed app flow, bad locator/assertion, and setup failure.
  • 设置模块看似通过,但留下错误页面、覆盖层、过滤器、搜索条件、选中行或侧边弹窗未关闭。需从产生异常后置条件的步骤/容器进行分类,而非仅从后续失败步骤分类。
  • 点击操作报告成功且目标为预期元素,但应用程序未切换到预期状态。需验证后置状态;不要因定位器正确就假设点击操作成功。
  • 宽泛的全局断言(如“页面包含X”)因页面上的无关文本而通过,导致后续步骤在错误上下文中输入或点击。
  • 无特定目标的输入步骤,若前置操作未成功聚焦预期字段,可能会在无效位置静默输入。
  • 定位器或缓存技术上有效,但语义错误。检查交互元素;对于无效缓存,从
    targetUpdateLoggerTags.runId
    查看原始缓存更新运行记录。
  • 已恢复步骤可能掩盖首次失败。检查已恢复模块内的失败子步骤,对比恢复后的最终状态与后续步骤的预期基准状态。
  • 超时并不自动归类为
    INFRA
    。需先排除缺失数据、错误页面状态、应用流程变更、无效定位器/断言及设置失败等原因。

Identifying related vs unrelated issues

识别相关与无关问题

  • Determine intent from the test name, description, and simplified steps. Treat failures outside that intent, including most setup/teardown failures, as unrelated. Relatedness qualifies bugs and changes; it does not change an
    INFRA
    failure into another category.
  • 从测试名称、描述和简化步骤判断意图。将超出该意图的故障(包括大多数设置/清理故障)视为无关问题。相关性用于区分bug和变更,但不会将
    INFRA
    故障转换为其他类别。

Bug vs change

Bug与变更

  • Bug: something very clearly went wrong when it should not have, such as an error message appearing. It is obvious just by looking at a single step or two that this is a bug.
  • Change: a clear change in the application behavior that you can prove through screenshots.
  • Bug:明显不应发生的错误,如出现错误提示。仅查看一两个步骤即可明显判断为bug。
  • 变更:可通过截图证明的应用程序行为明确变化。

Recoverability

可恢复性

Along with the category, determine one recoverability value:
  • RECOVERABLE
    — The failure can be automatically fixed by updating the test itself so that future runs pass.
    • Examples: an application change that requires a test update; vague locators or assertions that can be rewritten to pass stably.
  • ONE_TIME_RECOVERABLE
    — The failure can be recovered for this specific run without persisting a test change.
    • Examples: a random modal that can be dismissed without affecting test purpose; a temporary delay where waiting or retrying would likely succeed.
  • NON_RECOVERABLE
    — The failure cannot be automatically addressed and requires manual intervention.
    • Examples: missing credentials; missing local files required for upload; outages likely caused by third-party systems where test steps cannot fix the issue.
除类别外,需确定一个可恢复性值:
  • RECOVERABLE
    — 可通过更新测试本身自动修复故障,使后续运行通过。
    • 示例:应用程序变更需要更新测试;模糊定位器或断言可重写为稳定通过的形式。
  • ONE_TIME_RECOVERABLE
    — 可针对该特定运行记录恢复故障,无需持久化测试变更。
    • 示例:可在不影响测试目的的情况下关闭随机模态框;临时延迟可通过等待或重试解决。
  • NON_RECOVERABLE
    — 无法自动处理故障,需手动干预。
    • 示例:缺失凭证;文件上传步骤所需的本地文件缺失;第三方系统中断导致的故障,无法通过测试步骤修复。

Formal classification output

正式分类输出

  • Exactly one category id — no new labels, no multi-label.
  • Ground your decision in data. Be sure that you've fully investigated the run before assigning the category.
  • Prefer human-readable references over UUIDs when the step/module can be identified colloquially:
    module create-subpage-under-parent-page
    ,
    the last invocation of module <name>
    ,
    substep 4 (0-indexed)
    ,
    the failed setup assertion
    , etc. Tool calls still require exact IDs, but final reasoning should be readable.
  • When referencing past runs in final output, use clickable Momentic URLs rather than bare UUIDs:
    https://app.momentic.ai/runs/<runId>
    . Do not shorten UUIDs inside those URLs.
  • The reasoning must include the earliest divergent step/container and the broken postcondition it produced, not just the final failing step.
text
Reasoning: <a few sentences tied to the earliest divergence, screenshots/traces, past runs if used, and test intent>
Category: <one id from the list>
Recoverable: <RECOVERABLE | ONE_TIME_RECOVERABLE | NON_RECOVERABLE>
Confidence: <high | medium | low>
Confidence levels:
  • high
    — direct evidence, such as a clear screenshot of a label change or crash
  • medium
    — strong inference from multiple signals but no single conclusive screenshot or data point
  • low
    — ambiguous evidence; the classification required significant inference or the root cause is unclear
  • 精确指定一个类别ID — 不要新增标签,不要多标签。
  • 基于数据做出决策。在分配类别前,务必完成对运行记录的全面调查。
  • 当步骤/模块可通俗识别时,优先使用人类可读的引用而非UUID:
    module create-subpage-under-parent-page
    模块<name>的最后一次调用
    子步骤4(从0开始计数)
    失败的设置断言
    等。工具调用仍需精确ID,但最终推理过程应易于阅读。
  • 在最终输出中引用历史运行记录时,使用可点击的Momentic URL而非纯UUID:
    https://app.momentic.ai/runs/<runId>
    。不要缩短URL中的UUID。
  • 推理过程必须包含最早出现偏差的步骤/容器及其产生的损坏后置条件,而非仅最终失败步骤。
text
Reasoning: <几句与最早偏差、截图/跟踪信息、所用历史运行记录及测试意图相关的描述>
Category: <列表中的一个ID>
Recoverable: <RECOVERABLE | ONE_TIME_RECOVERABLE | NON_RECOVERABLE>
Confidence: <high | medium | low>
置信度级别:
  • high
    — 直接证据,如标签变更或崩溃的清晰截图
  • medium
    — 多个信号的强推论,但无单一决定性截图或数据点
  • low
    — 证据模糊;分类需大量推论或根本原因不明确

Category ids

类别ID

Use these strings verbatim:
  • NO_FAILURE
    — The run had no failures; all attempts passed.
  • APPLICATION_CHANGE
    — The test is out of date because the application's flow or UI has changed; updating the test to match the new behavior would permanently fix the failure.
  • BUG
    — Something clearly went wrong in the application that shouldn't have, such as an error message appearing or expected content failing to render.
  • TEST_AUTHORSHIP
    — The test can be permanently updated to prevent the failure while still validating its original intent, and you can recommend a specific authorship change such as adding or modifying a step, rewriting a vague assertion, or making a locator description more specific. If you cannot name a concrete change, choose a different category. Timeouts, slow page loads, and any failure whose recommended fix is to "wait longer" or to increase a timeout are NOT authorship issues — those are
    INFRA
    , even when the test could technically be edited to wait longer.
    • Examples: race conditions that can be fixed by adding or modifying steps other than waits/timeouts (e.g. replacing a "type with pressEnter" step with an explicit "select from list" step so the test no longer races the application); vague assertions or locator descriptions that can be rewritten to be more specific.
  • TEST_SETUP
    — Missing test data or files necessary to run the test, where the fix requires user action outside of the test itself.
    • Examples: missing file for a file upload step; missing or incorrect credentials needed by the test.
  • INFRA
    — The failure was unrelated to the application or application code and was caused by an infrastructure outage, long load times, or some other issue due to outside factors.
    • Examples: browser crash; high resource usage; rate limiting; a step or assertion that timed out waiting for the page or application to reach a slow-but-eventual state.
  • MOMENTIC_ISSUE
    — Some issue occurred with the execution of the test or Momentic data was incorrect (e.g. cache is wrong, global locator redirect did something weird, AI hallucinations).
    • Examples: unexpected behavior when viewing the run trace; the AI clearly misread or hallucinated data that is unambiguous in the screenshot, and no reasonable test alternative exists to avoid the AI step.
  • OTHER
    — The failure doesn't fit any of the other categories.
请严格使用以下字符串:
  • NO_FAILURE
    — 运行记录无故障;所有尝试均通过。
  • APPLICATION_CHANGE
    — 测试已过时,因应用程序流程或UI已变更;更新测试以匹配新行为可永久修复故障。
  • BUG
    — 应用程序出现明显不应发生的错误,如出现错误提示或预期内容未渲染。
  • TEST_AUTHORSHIP
    — 可通过永久更新测试防止故障,同时仍验证其原始意图,且可推荐具体的编写变更,如添加或修改步骤、重写模糊断言、使定位器描述更具体。若无法明确具体变更,请选择其他类别。超时、页面加载缓慢及建议修复为“等待更长时间”或增加超时的故障不属于编写问题 — 这些属于
    INFRA
    ,即使测试技术上可编辑为延长等待时间。
    • 示例:可通过添加或修改步骤(而非等待/超时)修复的竞争条件(如将“输入并按回车”步骤替换为显式“从列表选择”步骤,使测试不再与应用程序竞争);可重写为更具体的模糊断言或定位器描述。
  • TEST_SETUP
    — 运行测试所需的测试数据或文件缺失,修复需在测试本身之外进行用户操作。
    • 示例:文件上传步骤所需文件缺失;测试所需凭证缺失或错误。
  • INFRA
    — 故障与应用程序或应用程序代码无关,由基础设施中断、加载时间过长或其他外部因素导致。
    • 示例:浏览器崩溃;资源使用率过高;速率限制;步骤或断言因等待页面或应用程序达到缓慢但最终的状态而超时。
  • MOMENTIC_ISSUE
    — 测试执行出现问题或Momentic数据错误(如缓存错误、全局定位器重定向异常、AI幻觉)。
    • 示例:查看运行跟踪时出现意外行为;AI明显误读或幻觉出截图中明确的数据,且无合理的测试替代方案可避免该AI步骤。
  • OTHER
    — 故障不符合上述任何类别。