Back to Details

qa-dashboard

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese
<objective> Build dashboards that drive decisions, not dashboards that display data. The failure mode this prevents: a wall of panels showing "5 failures" with no link to the failures, no target, and no trend — a notification dressed up as a tool that everyone stops opening by week three. This skill delivers audience-specific QA reporting — rich HTML reports (Allure), real-time trend panels with regression alerts (Grafana/InfluxDB), self-hosted AI-assisted aggregation (ReportPortal), and automated stakeholder summaries — each panel mapped to a question someone actually asks and every red indicator drilling down to the failing test. </objective>
<目标> 构建能够驱动决策的仪表盘，而非仅展示数据的仪表盘。这能避免一种失效模式：一整墙的面板显示「5个失败用例」，但未关联具体失败用例、无目标值、无趋势——这种工具本质上只是包装过的通知，不出三周就没人会再打开。本技能提供面向特定受众的QA报告功能——丰富的HTML报告（Allure）、带回归告警的实时趋势面板（Grafana/InfluxDB）、自托管式AI辅助聚合平台（ReportPortal），以及自动化的利益相关者摘要——每个面板都对应实际有人提出的问题，每个红色指标都可深入查看具体失败的测试用例。 </目标>

Quick Route

快速选型路径

SituationGo to
Already on Grafana for infra metricsGrafana Dashboards — add test metrics alongside prod
Test runner has a hosted dashboard (Cypress/Playwright)SaaS-Native Dashboards — least plumbing
Need rich HTML report, no infra to runAllure Report — generate in CI, publish artifact
Need self-hosted aggregation across frameworks + AI triageReportPortal
Need a single view across multiple runnersGrafana or Allure (cross-runner aggregation)
Need a weekly summary / release verdict for stakeholdersStakeholder Reports
场景选择方案
已在使用Grafana监控基础设施指标Grafana仪表盘——将测试指标与生产指标放在一起展示
测试运行器自带托管仪表盘（Cypress/Playwright）SaaS原生仪表盘——所需配置最少
需要丰富的HTML报告，无基础设施可运行Allure Report——在CI中生成，发布为制品
需要跨框架的自托管聚合平台 + AI分类ReportPortal
需要跨多个运行器的统一视图Grafana 或 Allure（跨运行器聚合）
需要面向利益相关者的每周总结/发布判定利益相关者报告

Discovery Questions

探索问题

Check 
.agents/qa-project-context.md
first — if it exists, use it and skip anything answered there. Then:
  1. What tool do you use for test reporting today? Console output, JUnit XML, HTML reports, or a dedicated platform? Identifies the starting point and how much plumbing is left.
  2. Who will look at the dashboard? Developers need failure details and traces; QA leads need trends and flakiness; leadership needs release confidence and defect rates. Each audience needs a different view.
  3. What decisions should the dashboard drive? "Is this build safe to release?" "Which tests need fixing?" "Is quality improving sprint over sprint?" Dashboards without a decision context become shelfware.
  4. Where do test results live? GitHub Actions artifacts, S3, a database? The storage location determines which dashboard tool is practical.
  5. What CI platform? GitHub Actions, GitLab CI, Jenkins? Each has different artifact and reporting integrations.
首先查看
.agents/qa-project-context.md
——如果存在，使用该文件内容并跳过已解答的问题。然后询问：
  1. 当前使用什么工具进行测试报告？ 控制台输出、JUnit XML、HTML报告，还是专用平台？用于确定起点和所需配置工作量。
  2. 谁会查看仪表盘？ 开发人员需要失败详情和追踪信息；QA负责人需要趋势和不稳定测试情况；管理层需要发布信心和缺陷率。不同受众需要不同视图。
  3. 仪表盘应驱动哪些决策？ 「此构建是否可安全发布？」「哪些测试需要修复？」「质量是否在迭代中持续提升？」缺乏决策场景的仪表盘会沦为摆设。
  4. 测试结果存储在哪里？ GitHub Actions制品、S3、数据库？存储位置决定了适用的仪表盘工具。
  5. 使用什么CI平台？ GitHub Actions、GitLab CI、Jenkins？每个平台的制品和报告集成方式不同。

Core Principles

核心原则

1. Dashboards answer questions, they do not display numbers. Every panel must map to a question someone actually asks. "What is the flakiness rate?" is a question. "Total test count" is trivia.
2. Different audiences need different views. A developer debugging a CI failure needs stack traces, screenshots, and traces. A VP needs a single number: "Are we ready to release?" Do not force both through the same dashboard.
3. Real-time for CI, trends for leadership. CI dashboards update on every pipeline run. Leadership dashboards aggregate weekly or per-sprint. Mixing cadences confuses both audiences.
4. Drill-down to action. Every red indicator must link to the specific failing test, the specific flaky test, or the specific coverage gap. A dashboard that shows "5 failures" but does not link to the failures is a notification, not a tool.
5. Automate report generation. Reports that require manual effort (running scripts, copying data, formatting slides) will not survive the first busy sprint. Generate reports from CI pipelines automatically.
1. 仪表盘要解答问题，而非仅展示数字。 每个面板必须对应实际有人提出的问题。「不稳定测试率是多少？」是问题，「测试总数」则是无关信息。
2. 不同受众需要不同视图。 调试CI失败的开发人员需要堆栈跟踪、截图和追踪信息；副总裁只需要一个数字：「我们是否准备好发布？」不要强迫两类受众使用同一仪表盘。
3. CI场景用实时数据，管理层用趋势数据。 CI仪表盘在每次流水线运行时更新；管理层仪表盘按周或迭代聚合数据。混合更新频率会混淆两类受众。
4. 可深入至操作层面。 每个红色指标必须链接到具体失败的测试用例、不稳定测试用例或覆盖率缺口。仅显示「5个失败用例」但不关联具体内容的仪表盘只是通知，而非工具。
5. 自动化报告生成。 需要手动操作（运行脚本、复制数据、格式化幻灯片）的报告在第一个忙碌迭代后就会停止使用。需从CI流水线自动生成报告。

Allure Report

Allure Report

Allure generates rich HTML reports from test results with history, categories, and retries built in. It works with Playwright, Jest, Vitest, pytest, and most frameworks.
Allure 2 vs Allure 3. Two paths, and they are easy to mix up because the framework adapters (
allure-playwright
, 
allure-vitest
, 
allure-jest
) emit the same Allure 2 result files; only the reader differs:
  • Allure 2 path — 
    allure-commandline
    (2.42.1, Jun 2026). 
    brew install allure
    installs this, not v3. Commands: 
    allure generate
    / 
    allure open
    / 
    allure serve
    ; categories via a 
    categories.json
    dropped into 
    allure-results/
    . Stable, frozen, dependency-bump-only now.
  • Allure 3 path — 
    allure
    npm package + 
    allurerc.mjs
    (3.9.0, May 2026). TypeScript rewrite: plugins, single-file config, real-time 
    allure watch
    , project-wide quality gates, multi-environment reports, and Allure Service for server-side history. Commands: 
    npx allure run
    / 
    npx allure generate
    / 
    npx allure watch
    ; categories move into 
    allurerc.mjs
    (the dropped-in 
    categories.json
    is a v2 concept).
Choose Allure 3 for new projects. If you follow only the 
brew install allure
/ 
allure generate
commands you are on the v2 path — that is fine and fully supported; just know which one you are running.
Minimal Playwright reporter wiring — register the adapter as 
reporter: ["allure-playwright", { outputFolder, environmentInfo }]
so every run drops results into 
allure-results/
with the environment captured:
typescript
// playwright.config.ts
reporter: [
  ["list"],
  ["allure-playwright", {
    outputFolder: "allure-results",
    environmentInfo: {
      Environment: process.env.TEST_ENV ?? "local",
      BaseURL: process.env.BASE_URL ?? "http://localhost:3000",
    },
  }],
],
Add per-test metadata (
allure.severity
/ 
feature
/ 
story
/ 
tag
) to drive grouping, define failure 
categories
to split product bugs from infra breakage, and preserve 
history/
across CI runs so trends exist at all. Without history an Allure report is a single snapshot — no trends, no intermittent-failure detection. See 
references/allure.md
for the full Playwright/Vitest configs, the v2 
categories.json
, the v3 
allurerc.mjs
+ 
allure run
runnable path, and the GitHub Actions history-preservation steps.
Allure可从测试结果生成包含历史记录、分类和重试信息的丰富HTML报告。它支持Playwright、Jest、Vitest、pytest及大多数测试框架。
Allure 2 vs Allure 3. 有两种使用路径，容易混淆，因为框架适配器（
allure-playwright
、
allure-vitest
、
allure-jest
）输出的是相同的Allure 2结果文件，仅阅读器不同：
  • Allure 2路径 — 
    allure-commandline
    （版本2.42.1，2026年6月）。
    brew install allure
    安装的是此版本，而非v3。命令：
    allure generate
    / 
    allure open
    / 
    allure serve
    ；通过将
    categories.json
    放入
    allure-results/
    目录定义分类。目前已稳定，仅更新依赖。
  • Allure 3路径 — 
    allure
    npm包 + 
    allurerc.mjs
    （版本3.9.0，2026年5月）。基于TypeScript重写：支持插件、单文件配置、实时
    allure watch
    、项目级质量门、多环境报告，以及用于服务端历史记录的Allure Service。命令：
    npx allure run
    / 
    npx allure generate
    / 
    npx allure watch
    ；分类配置移至
    allurerc.mjs
    （放入目录的
    categories.json
    是v2的概念）。
新项目建议选择Allure 3。如果仅执行
brew install allure
/ 
allure generate
命令，则使用的是v2路径——这也完全受支持，只需明确自己使用的版本。
Playwright报告器最简配置——注册适配器为
reporter: ["allure-playwright", { outputFolder, environmentInfo }]
，以便每次运行都将结果存入
allure-results/
并捕获环境信息：
typescript
// playwright.config.ts
reporter: [
  ["list"],
  ["allure-playwright", {
    outputFolder: "allure-results",
    environmentInfo: {
      Environment: process.env.TEST_ENV ?? "local",
      BaseURL: process.env.BASE_URL ?? "http://localhost:3000",
    },
  }],
],
添加测试元数据（
allure.severity
/ 
feature
/ 
story
/ 
tag
）用于分组，定义失败
categories
以区分产品缺陷与基础设施故障，并在CI运行间保留
history/
目录，这样才能显示趋势。没有历史记录的Allure报告只是单一快照——无趋势、无间歇性故障检测。 完整的Playwright/Vitest配置、v2的
categories.json
、v3的
allurerc.mjs
+ 
allure run
可运行路径，以及GitHub Actions历史记录保留步骤，请查看
references/allure.md
。

Grafana Dashboards

Grafana仪表盘

Grafana gives real-time dashboards with alerting. Best when the team already runs Grafana for infrastructure and wants test metrics next to production metrics.
Data pipeline: a post-test CI step parses results (JUnit XML, coverage JSON, timing) and pushes points to a time-series DB (InfluxDB or a Prometheus pushgateway); Grafana queries those. The push script writes two measurements — 
test_execution
(one point per test, tagged by 
suite
/
test_name
/
status
/
branch
/
run_id
) and 
test_run_summary
(one point per run with 
pass_rate
, 
total
, 
failed
, 
avg_duration_ms
). Tag every point with 
branch
and 
run_id
so panels can filter to 
main
and link back to a specific run.
The script runs under 
if: always()
, so wrap the write loop in try/flush/close — a throw mid-loop otherwise loses every buffered point. See 
references/grafana.md
for the full 
push-test-metrics.ts
(with the flush guard) and the GitHub Actions step.
Grafana提供带告警功能的实时仪表盘。最适合已使用Grafana监控基础设施、希望将测试指标与生产指标放在一起展示的团队。
数据流水线： 测试后的CI步骤解析结果（JUnit XML、覆盖率JSON、耗时数据）并将数据点推送到时序数据库（InfluxDB或Prometheus推送网关）；Grafana查询这些数据。推送脚本写入两类测量数据——
test_execution
（每个测试对应一个数据点，标记
suite
/
test_name
/
status
/
branch
/
run_id
）和
test_run_summary
（每次运行对应一个数据点，包含
pass_rate
、
total
、
failed
、
avg_duration_ms
）。为每个数据点标记
branch
和
run_id
，以便面板可过滤到
main
分支并链接到具体运行记录。
脚本需在
if: always()
条件下运行，因此需将写入循环包裹在try/flush/close中——否则中途抛出异常会丢失所有缓冲的数据点。完整的
push-test-metrics.ts
脚本（包含刷新保护）和GitHub Actions步骤，请查看
references/grafana.md
。

Recommended panels

推荐面板

PanelQuestionQuery shape
Pass Rate Trend (time series)Is quality improving?
SELECT mean("pass_rate") FROM "test_run_summary" WHERE "branch"='main' GROUP BY time(1d)
, thresholds at 95% (yellow) / 99% (green)
Release Readiness (stat)Is main ready to release?
SELECT last("pass_rate") FROM "test_run_summary" WHERE "branch"='main'
, red <95 / yellow 95–99 / green ≥99 — pair with a coverage stat ≥80
Flakiness Top 10 (table)Which tests waste the most time?
SELECT "test_name", count("retries") AS retry_count FROM "test_execution" WHERE "retries">0 AND time>now()-14d GROUP BY "test_name" ORDER BY retry_count DESC LIMIT 10
CI Duration TrendIs the pipeline getting slower?
avg_duration_ms
over time with a target line at 600s
Full queries plus Coverage Trend and Duration Distribution panels are in 
references/grafana.md
.
面板解答的问题查询格式
通过率趋势（时序图）质量是否在提升？
SELECT mean("pass_rate") FROM "test_run_summary" WHERE "branch"='main' GROUP BY time(1d)
，阈值设为95%（黄色）/ 99%（绿色）
发布就绪状态（统计面板）main分支是否可发布？
SELECT last("pass_rate") FROM "test_run_summary" WHERE "branch"='main'
，红色<95 / 黄色95–99 / 绿色≥99 — 搭配覆盖率≥80%的统计面板
Top 10不稳定测试（表格）哪些测试最耗时？
SELECT "test_name", count("retries") AS retry_count FROM "test_execution" WHERE "retries">0 AND time>now()-14d GROUP BY "test_name" ORDER BY retry_count DESC LIMIT 10
CI耗时趋势流水线是否在变慢？展示
avg_duration_ms
随时间变化的趋势，目标线设为600秒
完整查询语句以及覆盖率趋势和耗时分布面板，请查看
references/grafana.md
。

Alerting

告警功能

Provision alert rules as YAML under 
provisioning/alerting/
(Grafana 11+). The one the Done When requires: main pass rate drops more than 2 percentage points in a single day. Build it from two queries (mean 
pass_rate
over the last 
1d
vs the day before) feeding a math expression 
$yesterday - $today > 2
, then route via a Slack contact point (incoming-webhook URL). The full provisioned rule + contact point is in 
references/grafana.md
. Also alert on: pass rate below 95% (10m window), CI duration above 15 min, coverage drop >2% in a week. Dashboards are for investigation; alerts are for detection — a dashboard no one opens catches nothing.
在
provisioning/alerting/
目录下通过YAML配置告警规则（Grafana 11+）。必须配置的规则：main分支单日通过率下降超过2个百分点。通过两个查询（过去1天的平均
pass_rate
与前一天对比）生成数学表达式
$yesterday - $today > 2
，然后通过Slack联系人（传入Webhook URL）发送告警。完整的预配置规则+联系人设置，请查看
references/grafana.md
。还需配置以下告警：通过率低于95%（10分钟窗口）、CI耗时超过15分钟、周覆盖率下降>2%。仪表盘用于排查问题；告警用于发现问题——没人查看的仪表盘毫无用处。

ReportPortal

ReportPortal

Self-hosted test reporting platform with ML-powered failure analysis, cross-framework aggregation, and real-time dashboards.
bash
undefined
自托管式测试报告平台，具备ML驱动的故障分析、跨框架聚合和实时仪表盘功能。
bash
undefined

Pin to a tagged release (current 26.0.x line: 26.0.3). The 
master
branch may not

固定使用标记版本（当前26.0.x分支：26.0.3）。
master
分支可能与受支持的26.x分支不匹配。

match the supported 26.x line.

—
curl -LO https://raw.githubusercontent.com/reportportal/reportportal/26.0.3/docker-compose.yml docker compose up -d
curl -LO https://raw.githubusercontent.com/reportportal/reportportal/26.0.3/docker-compose.yml docker compose up -d

Access at http://localhost:8080 (default: superadmin/erebus)

访问地址：http://localhost:8080（默认账号：superadmin/erebus）

ML auto-analysis lives in the 
service-auto-analyzer
container — confirm it is running.

ML自动分析功能位于
service-auto-analyzer
容器中——请确认该容器正在运行。


As of 26.0.3 ReportPortal also ingests **agentic** test results (launches carry an `AGENTIC` vs
`AUTOMATION` execution-type badge) — relevant if part of your suite runs through Claude Code or another agent.

截至26.0.3版本，ReportPortal还支持导入**Agent驱动**的测试结果（启动记录带有`AGENTIC` vs `AUTOMATION`执行类型标识）——如果部分测试套件通过Claude Code或其他Agent运行，此功能非常实用。

Playwright integration

Playwright集成

typescript
// playwright.config.ts
reporter: [
  ["list"],
  ["@reportportal/agent-js-playwright", {
    apiKey: process.env.RP_API_KEY,
    endpoint: process.env.RP_ENDPOINT ?? "http://localhost:8080/api/v1",
    project: "my-project",
    launch: `E2E Tests - ${process.env.CI ? "CI" : "local"}`,
    attributes: [
      { key: "branch", value: process.env.GITHUB_HEAD_REF ?? "local" },
      { key: "build", value: process.env.GITHUB_RUN_ID ?? "dev" },
    ],
  }],
],
Install with 
npm i -D @reportportal/agent-js-playwright
.
FeatureWhat it does
Auto-analysisML failure classification: product bug, test bug, system issue, or to-investigate
Defect type mappingCustom defect categories with sub-types for your project
Flaky test detectionTests that flip pass/fail across launches
Merge launchesCombine sharded CI runs into one unified view
Quality gatesPass/fail criteria per launch (max failures, min pass rate)
ComparisonSide-by-side of two launches to spot regressions
Quality gates are queryable after a run (
GET /api/v1/$PROJECT/launch/$LAUNCH_ID/quality-gate
) — use the status as a CI gate and fail the pipeline if it is not 
PASSED
.
typescript
// playwright.config.ts
reporter: [
  ["list"],
  ["@reportportal/agent-js-playwright", {
    apiKey: process.env.RP_API_KEY,
    endpoint: process.env.RP_ENDPOINT ?? "http://localhost:8080/api/v1",
    project: "my-project",
    launch: `E2E Tests - ${process.env.CI ? "CI" : "local"}`,
    attributes: [
      { key: "branch", value: process.env.GITHUB_HEAD_REF ?? "local" },
      { key: "build", value: process.env.GITHUB_RUN_ID ?? "dev" },
    ],
  }],
],
安装命令：
npm i -D @reportportal/agent-js-playwright
。
功能作用
自动分析ML故障分类：产品缺陷、测试缺陷、系统问题或待排查
缺陷类型映射为项目自定义缺陷类别及子类别
不稳定测试检测跨启动记录中状态反复切换的测试
合并启动记录将分片的CI运行合并为统一视图
质量门每次启动的通过/失败标准（最大失败数、最小通过率）
对比功能两个启动记录的并排对比，用于发现回归问题
运行后可查询质量门状态（
GET /api/v1/$PROJECT/launch/$LAUNCH_ID/quality-gate
）——将该状态作为CI校验门，如果状态不是
PASSED
则使流水线失败。

Allure TestOps (managed alternative)

Allure TestOps（托管替代方案）

If self-hosting feels heavy, Allure TestOps (26.2.x line, 2026) is the SaaS path: Allure 3 quality gates, named environments, global attachments, and Allure 3-style flaky detection (flags a test once it shows ≥3 status transitions across its last 10 runs). Its MCP server is in public beta (26.1.1), letting AI agents query launches and quality gates directly — relevant when your QA workflow runs through Claude Code / Cursor.
如果自托管配置过于繁琐，Allure TestOps（26.2.x分支，2026年）是SaaS方案：支持Allure 3质量门、命名环境、全局附件，以及Allure 3风格的不稳定测试检测（当测试在最近10次运行中出现≥3次状态切换时标记）。其MCP服务器处于公开测试阶段（26.1.1），允许AI Agent直接查询启动记录和质量门——当QA工作流通过Claude Code / Cursor运行时非常实用。

SaaS-Native Test Dashboards

SaaS原生测试仪表盘

If your test runner has a first-class hosted dashboard, prefer it over Allure/Grafana for that runner's native data — less plumbing, more retention, built-in PR comments. Cross-pollinate with Allure/Grafana only for cross-runner aggregation.
PlatformTest runnerNative data + PR comments
Cypress CloudCypressTest replay, parallelization, flake detection; AI add-on (Auto Heal, Bug Triage)
Currents.devCypress, PlaywrightOSS-friendly Cypress Cloud alternative; lower price point
Playwright HTML + 
--reporter=blob
PlaywrightFree, self-hosted; combine shards with 
merge-reports
Datadog Test OptimizationAny (CI-side)Flaky Test Management (now with Bits AI auto-fix), TIA, native APM
Allure TestOpsAnyAllure 3 quality gates, named environments, MCP server beta
Combining sharded Playwright runs (free, native). Have each shard emit a blob report, then merge into one HTML report — the no-cost answer to "combine sharded CI runs":
bash
undefined
如果测试运行器自带一流的托管仪表盘，优先选择该方案而非Allure/Grafana，因为它能提供原生数据——配置更少、保留时间更长、内置PR评论功能。仅当需要跨运行器聚合时，再结合Allure/Grafana使用。
平台测试运行器原生数据 + PR评论
Cypress CloudCypress测试回放、并行化、不稳定测试检测；AI附加功能（自动修复、缺陷分类）
Currents.devCypress、Playwright开源友好的Cypress Cloud替代方案；价格更低
Playwright HTML + 
--reporter=blob
Playwright免费、自托管；通过
merge-reports
合并分片运行结果
Datadog Test Optimization任意（CI端）不稳定测试管理（现已支持Bits AI自动修复）、TIA、原生APM
Allure TestOps任意Allure 3质量门、命名环境、MCP服务器测试版
合并分片Playwright运行结果（免费、原生）。 每个分片输出blob报告，然后合并为一个HTML报告——这是「合并分片CI运行结果」的免费解决方案：
bash
undefined

each shard: npx playwright test --reporter=blob (uploads blob-report/ as an artifact)

每个分片运行：npx playwright test --reporter=blob （将blob-report/上传为制品）

npx playwright merge-reports --reporter html ./blob-reports

Use Allure or Grafana when you need one dashboard across multiple runners, or when a SaaS option's pricing/data-residency does not fit. Otherwise the SaaS-native dashboard is usually the cheapest path to PR-level signal.

---
npx playwright merge-reports --reporter html ./blob-reports

当需要跨多个运行器的统一仪表盘，或SaaS方案的价格/数据驻留政策不符合需求时，再使用Allure或Grafana。否则SaaS原生仪表盘通常是获取PR级信号成本最低的方案。

---

Stakeholder Reports

利益相关者报告

Weekly QA Summary — automate via a scheduled CI job. Include: pass rate + trend, new vs fixed failures, top 5 flaky tests, coverage delta, avg CI duration. Classify health: STABLE (>= 98%), NEEDS ATTENTION (>= 95%), CRITICAL (< 95%). Post to Slack automatically.
Release Quality Report — generate before each release. Gate on: E2E pass rate >= 99%, unit pass rate 100%, branch coverage >= 80%, zero critical bugs, major bugs <= 2, and the Core Web Vitals budget. Output a READY / NOT READY verdict with a per-gate pass/fail breakdown.
Core Web Vitals "good" thresholds for the perf gate (current as of 2026): LCP < 2500ms, INP < 200ms, CLS < 0.1. Gate on INP, not FID — INP replaced FID as a Core Web Vital on 2024-03-12 and FID was fully retired on 2024-09-09.
每周QA总结——通过定时CI任务自动生成。内容包括：通过率+趋势、新增与修复的失败用例、Top 5不稳定测试、覆盖率变化、平均CI耗时。健康状态分类：稳定（≥98%）、需关注（≥95%）、严重（<95%）。自动发布至Slack。
发布质量报告——每次发布前生成。校验标准：E2E通过率≥99%、单元测试通过率100%、分支覆盖率≥80%、零严重缺陷、主要缺陷≤2个，以及Core Web Vitals达标。输出就绪/未就绪判定，并附带每个校验项的通过/失败详情。
性能校验门的Core Web Vitals「良好」阈值（2026年当前标准）：LCP < 2500ms，INP < 200ms，CLS < 0.1。校验INP而非FID——INP于2024年3月12日取代FID成为Core Web Vital，FID于2024年9月9日完全停用。

Recommended Dashboard Panels

推荐仪表盘面板

A practical set covering the most common questions teams ask.
PanelQuestion It AnswersData SourceAudience
Pass/Fail TrendIs quality improving or degrading?CI test results over timeEveryone
Flakiness Top 10Which tests waste the most time?Tests with retries in last 14 daysDevelopers, QA
Coverage HeatmapWhere are we blind?Coverage by module/directoryDevelopers
Defect Escape TrendAre bugs reaching production?Incidents tagged as test escapesQA leads, Leadership
CI DurationIs the pipeline getting slower?Pipeline duration over timeDevOps, Developers
Test VelocityTests proportional to features?New tests added per sprintQA leads
Failure CategoriesProduct bugs or test infra?Categorized failure reasonsQA leads
Release ReadinessCan we ship?Composite score from all gatesLeadership
一套实用的面板集合，覆盖团队最常提出的问题。
面板解答的问题数据源受众
通过率/失败率趋势质量在提升还是下降？一段时间内的CI测试结果所有人
Top 10不稳定测试哪些测试最耗时？过去14天内有重试记录的测试开发人员、QA
覆盖率热力图哪些模块存在测试盲区？按模块/目录划分的覆盖率数据开发人员
缺陷逃逸趋势缺陷是否流入生产环境？标记为测试逃逸的事件QA负责人、管理层
CI耗时流水线是否在变慢？一段时间内的流水线耗时DevOps、开发人员
测试速度测试数量与功能增长是否匹配？每个迭代新增的测试数量QA负责人
失败分类是产品缺陷还是测试基础设施问题？分类后的失败原因QA负责人
发布就绪状态我们可以发布吗？所有校验门的综合得分管理层

Anti-Patterns

反模式

Dashboard with 30 panels. No one reads it. Start with 5–6 panels that answer the most urgent questions; add panels only when someone asks one the dashboard cannot answer.
Metrics without context. "Pass rate: 97%" means nothing without "target: 99%" and "last week: 98.5%." Every metric needs a target and a trend to be actionable.
Manual report generation. If the weekly summary needs someone to SSH in, run queries, and paste into slides, it stops happening by week 3. Automate it into CI.
Same dashboard for developers and leadership. Developers need failure details, stack traces, and repro steps; leadership needs one traffic light. Build separate views.
Reporting test counts as progress. "We added 200 tests" says nothing about quality. Report critical-path coverage, defect escape rate, and mean time to detect regressions instead.
No alerting on regressions. A dashboard no one checks is useless. Alert on pass-rate drops, coverage decreases, and CI-duration increases. Dashboards are for investigation; alerts are for detection.
Allure without history. A single Allure report is a snapshot — no trends, no intermittent-failure detection, no measure of improvement. Always preserve 
history/
across CI runs (or use Allure 3 / Allure Service for server-side history).
Mixing the Allure 2 and 3 code paths. Following the v3 callout in prose but copying 
brew install allure
+ 
allure generate
+ a dropped-in 
categories.json
lands you on v2 with v2 categories. Pick one path and use its commands end to end.
包含30个面板的仪表盘。 没人会看。从5-6个解答最紧迫问题的面板开始；仅当有人提出仪表盘无法解答的问题时，再添加新面板。
无上下文的指标。 「通过率：97%」在没有「目标：99%」和「上周：98.5%」的情况下毫无意义。每个指标都需要目标值和趋势才能具备可操作性。
手动生成报告。 如果每周总结需要有人SSH登录、运行查询、粘贴到幻灯片中，不出三周就会停止执行。需将其自动化到CI中。
开发人员和管理层使用同一仪表盘。 开发人员需要失败详情、堆栈跟踪和复现步骤；管理层只需要一个红绿灯标识。需构建独立视图。
将测试数量视为进度。 「我们新增了200个测试」无法说明质量情况。应报告关键路径覆盖率、缺陷逃逸率和回归检测平均耗时。
未配置回归告警。 没人查看的仪表盘毫无用处。应对通过率下降、覆盖率降低和CI耗时增加配置告警。仪表盘用于排查问题；告警用于发现问题。
无历史记录的Allure报告。 单一Allure报告只是快照——无趋势、无间歇性故障检测、无法衡量改进情况。必须在CI运行间保留
history/
目录（或使用Allure 3 / Allure Service存储服务端历史记录）。
混合使用Allure 2和3的代码路径。 文字说明中提到v3，但复制
brew install allure
+ 
allure generate
+ 放入目录的
categories.json
，最终会使用v2版本和v2分类。需选择一种路径并全程使用对应命令。

Verification

验证

Prove the report/dashboard actually renders before calling it done. Smallest checks first:
bash
undefined
在完成前需确认报告/仪表盘可正常渲染。先进行最小化检查：
bash
undefined

Allure: a report builds and the trend widget shows >1 run (history preserved)

Allure：报告可生成，且趋势组件显示>1次运行（历史记录已保留）

npx allure generate allure-results --clean -o allure-report && npx allure open allure-report
npx allure generate allure-results --clean -o allure-report && npx allure open allure-report

-> Overview page loads; the "Trend" widget shows more than one run.

-> 概览页面加载成功；「趋势」组件显示多于一次运行记录。

Grafana push: the summary point landed in InfluxDB

Grafana推送：汇总数据点已存入InfluxDB

influx query 'from(bucket:"test-results") |> range(start:-1d) |> filter(fn:(r)=> r._measurement=="test_run_summary")'
influx query 'from(bucket:"test-results") |> range(start:-1d) |> filter(fn:(r)=> r._measurement=="test_run_summary")'

-> returns at least one row with a pass_rate field for branch=main.

-> 返回至少一行包含main分支pass_rate字段的数据。

Grafana alert: the provisioned rule loaded

Grafana告警：预配置规则已加载

curl -s http://localhost:3000/api/v1/provisioning/alert-rules -u admin:admin | jq '.[].title'
curl -s http://localhost:3000/api/v1/provisioning/alert-rules -u admin:admin | jq '.[].title'

-> includes "Main pass rate dropped >2pp in a day".

-> 包含「Main pass rate dropped >2pp in a day」。


---

---

Done When

完成标准

  • Dashboard is published to a known location that returns HTTP 200 for the team (CI artifact URL, Grafana URL, or hosted SaaS link) — no local setup or manual report run required.
  • Test execution trends (pass rate, failure count, duration) are visible over at least 2 weeks of historical data.
  • Flakiness panel is configured showing the top flaky tests with retry counts over a rolling 14-day window.
  • A stakeholder report template is generating automatically — either a weekly summary with a STABLE/NEEDS ATTENTION/CRITICAL health label or a per-release quality report with a READY/NOT READY verdict and per-gate breakdown (one or both is acceptable).
  • An alert is configured and routed (Slack or equivalent) that fires when the main-branch pass rate drops by more than 2 percentage points in a single day.
  • 仪表盘已发布至团队可访问的已知位置（CI制品URL、Grafana URL或托管SaaS链接）——无需本地配置或手动运行报告。
  • 可查看至少2周历史数据的测试执行趋势（通过率、失败数、耗时）。
  • 已配置不稳定测试面板，显示过去14天内重试次数最多的Top不稳定测试。
  • 已自动生成利益相关者报告模板——要么是带有稳定/需关注/严重健康标签的每周总结，要么是带有就绪/未就绪判定及各校验项详情的发布质量报告（两者满足其一或全部均可）。
  • 已配置并路由（Slack或等效工具）告警：当main分支单日通过率下降超过2个百分点时触发。

Related Skills

相关技能

  • qa-metrics — Defining quality KPIs, measurement frameworks, and metric interpretation. Go there to decide what to measure; come here to visualize it.
  • ci-cd-integration — Pipeline configuration for automated report generation and artifact management.
  • ai-bug-triage — AI-powered failure classification that feeds into dashboard failure categories.
  • qa-metrics — 定义质量KPI、测量框架和指标解读。如需决定测量什么，请使用该技能；如需可视化测量结果，请使用本技能。
  • ci-cd-integration — 自动生成报告和制品管理的流水线配置。
  • ai-bug-triage — AI驱动的故障分类，可为仪表盘的失败分类提供数据。

Reference Files (in 
references/
)

参考文件（位于
references/
目录）

  • allure.md — full Playwright/Vitest adapter configs, per-test metadata, the v2 
    categories.json
    , the v3 
    allurerc.mjs
    + 
    allure run
    runnable path, and the CI history-preservation workflow.
  • grafana.md — the 
    push-test-metrics.ts
    script (with flush guard), all panel queries, and the provisioned ">2pp pass-rate drop" alert rule + Slack contact point.
  • allure.md — 完整的Playwright/Vitest适配器配置、测试元数据、v2的
    categories.json
    、v3的
    allurerc.mjs
    + 
    allure run
    可运行路径，以及CI历史记录保留工作流。
  • grafana.md — 
    push-test-metrics.ts
    脚本（包含刷新保护）、所有面板查询语句，以及预配置的「通过率下降>2个百分点」告警规则+Slack联系人设置。 ",