k6-trend-analysis

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

k6 Trend Analysis

k6趋势分析

Analyze metric trends across multiple runs of a Grafana Cloud k6 test to catch degradation early -- before thresholds breach and alerts fire. A P95 at 380ms against a 500ms threshold is "green" today, but if it was 250ms a month ago, something is quietly degrading; this skill surfaces that drift and recommends action.
分析Grafana Cloud k6测试多次运行的指标趋势,以便在阈值突破和警报触发之前尽早发现性能退化。当前P95延迟为380ms,阈值为500ms,虽然处于“正常”状态,但如果一个月前是250ms,说明存在隐性退化;此技能会发现这种漂移并给出行动建议。

What this skill does NOT do

此技能不具备的功能

  • Deep-dive into a single run's failure: load
    k6-cloud-investigate-test
  • Edit scripts or apply threshold changes: load
    k6-test-maintenance
  • Create new test scripts: use the appropriate test creation workflow
  • Query service-side metrics directly: this skill hands off to
    debug-with-grafana
    when observability correlation is needed
  • 深入分析单次运行的失败情况:加载
    k6-cloud-investigate-test
  • 编辑脚本或应用阈值变更:加载
    k6-test-maintenance
  • 创建新测试脚本:使用相应的测试创建工作流
  • 直接查询服务端指标:当需要可观测性关联时,此技能会转交至
    debug-with-grafana

Dependencies

依赖项

This skill delegates all GCk6 API mechanics to
k6-manage
. Read it before executing any API call -- it covers auth, path construction (the doubled
cloud/cloud/
prefix), pagination with
@nextLink
, the spill envelope, and metric query syntax. Do not duplicate that knowledge here.
Tools used:
gcx
(via k6-manage patterns).

此技能将所有GCk6 API操作委托给**
k6-manage
**。在执行任何API调用之前请阅读该文档——它涵盖了认证、路径构造(双重
cloud/cloud/
前缀)、使用
@nextLink
的分页、溢出信封以及指标查询语法。请勿在此处重复这些内容。
使用的工具:
gcx
(遵循k6-manage模式)。

Workflow

工作流程

Follow these steps in order. Present findings at the end -- do not apply changes.
按以下顺序执行步骤。在最后呈现结果——不要直接应用变更。

Step 1: Identify the test

步骤1:确定测试

The user provides one of:
  • A GCk6 URL (extract the load test ID from the path)
  • A test ID directly
  • A test name (search via
    gcx k6 tests list
    or the v6 API)
Confirm the test exists by fetching its metadata. Record the
id
,
name
,
project_id
, and
created
timestamp. You will need the load test ID (not a run ID) for the multi-run metric endpoints.
用户会提供以下信息之一:
  • GCk6 URL(从路径中提取负载测试ID)
  • 直接提供测试ID
  • 测试名称(通过
    gcx k6 tests list
    或v6 API搜索)
通过获取测试元数据确认测试存在。记录
id
name
project_id
created
时间戳。多运行指标端点需要负载测试ID(而非运行ID)。

Step 2: Determine the analysis window

步骤2:确定分析窗口

Default to last 30 days. Adjust if:
  • The user requests a specific window
  • The test has very few runs (<5 in 30 days) -- widen the window and note this
  • The test runs very frequently (hundreds of runs in 30 days) -- consider sampling or narrowing. Ask the user if the volume is extreme (>200 runs)
State the window explicitly: "Analyzing runs from {start_date} to {end_date}."
默认使用最近30天。在以下情况调整窗口:
  • 用户请求特定窗口
  • 测试运行次数极少(30天内少于5次)——扩大窗口并注明这一点
  • 测试运行非常频繁(30天内数百次)——考虑抽样或缩小窗口。如果数量极多(>200次),请询问用户
明确说明窗口:“分析{start_date}至{end_date}的运行情况。”

Step 3: Fetch runs in the window

步骤3:获取窗口内的运行记录

List all runs for the test using the v6 API with
$orderby=created desc
pagination (see k6-manage Section 3 for the
@nextLink
loop pattern). Filter to runs within the analysis window.
For each run, record:
  • id
    ,
    created
    ,
    ended
  • result
    (passed/failed/timed_out)
  • status
    (created/queued/initializing/running/finished/aborted)
  • note
    (if present -- users sometimes annotate runs)
Discard runs that did not reach
finished
status (aborted, timed_out, etc.) unless the user specifically asks about them -- incomplete runs produce unreliable metric aggregates.
Count the runs. If fewer than 3 usable runs exist, inform the user that trend analysis is not meaningful with this sample size and suggest widening the window or waiting for more runs.
使用v6 API并按
$orderby=created desc
分页(查看k6-manage第3节的
@nextLink
循环模式)列出测试的所有运行记录。筛选出分析窗口内的运行。
为每个运行记录:
  • id
    created
    ended
  • result
    (通过/失败/超时)
  • status
    (已创建/排队/初始化/运行中/已完成/已中止)
  • note
    (如果存在——用户有时会为运行添加注释)
除非用户特别要求,否则丢弃未达到
finished
状态的运行(已中止、超时等)——不完整的运行会产生不可靠的指标聚合结果。
统计运行次数。如果可用运行次数少于3次,告知用户此样本量下趋势分析无意义,并建议扩大窗口或等待更多运行记录。

Step 4: Fetch all metrics and their types

步骤4:获取所有指标及其类型

Before querying values, discover what metrics the test emits. Use the multi-run metric listing endpoint (k6-manage references/metrics.md Section 2):
GET /cloud/v5/load_tests/{loadTestId}/metrics(test_run_ids=[{id1},{id2},...])
Pass a representative subset of run IDs (the first and last few) to catch metrics that may have been added or removed over the window. Record each metric's
name
and
type
(counter, gauge, trend, rate).
Group metrics by type -- the query method must match the metric type (see metrics.md "Query methods"). Using the wrong method returns empty results.
在查询指标值之前,先发现测试会输出哪些指标。使用多运行指标列表端点(k6-manage参考文档/metrics.md第2节):
GET /cloud/v5/load_tests/{loadTestId}/metrics(test_run_ids=[{id1},{id2},...])
传入有代表性的部分运行ID(前几个和后几个),以捕捉窗口内可能新增或移除的指标。记录每个指标的
name
type
(计数器、仪表、趋势、比率)。
按类型分组指标——查询方法必须与指标类型匹配(查看metrics.md的“查询方法”)。使用错误的方法会返回空结果。

Step 5: Fetch per-run aggregate values

步骤5:获取每运行的聚合值

For each metric, query its aggregate value across all runs in the window using the multi-run aggregate endpoint (metrics.md Section 8):
GET /cloud/v5/load_tests/{loadTestId}/query_aggregate_k6(
  query='<method>',
  metric='<metric_name>',
  test_run_ids=[{id1},{id2},...]
)
Concrete
gcx
form (proxy prefix per k6-manage §2;
-o json
avoids the spill envelope):
bash
LT=<load_test_id>; IDS="123,124,125"
gcx --context <ctx> api "/api/plugins/k6-app/resources/cloud/cloud/v5/load_tests/$LT/query_aggregate_k6(query='histogram_quantile(0.95)',metric='http_req_duration',test_run_ids=[$IDS])" -o json
Choose the aggregate method based on metric type:
Metric typePrimary methodWhat it captures
trend
histogram_quantile(0.95)
P95 latency -- the most common SLO target
trend
histogram_quantile(0.5)
Median -- shows typical behavior
trend
histogram_avg
Mean -- sensitive to outliers
counter
increase
Total count per run
rate
ratio
Success/failure ratio
gauge
max
Peak value per run
For trend-type metrics (latencies), query multiple quantiles (P50, P90, P95, P99) to see if degradation is uniform or concentrated in the tail.
Always break down by request grouping for multi-target tests. A single aggregate p95 across a whole test obscures regressions confined to one endpoint or one page -- a 3x slowdown on one URL can be invisible at the test-level p95 if the test hits many URLs. Default to grouped queries:
Metric patternDefault groupingWhy
browser_web_vital_*
(LCP, FCP, CLS, TTFB, INP, FID)
by (url)
Each navigation emits its own web vital; per-URL trends pin regressions to a specific page.
http_req_duration
,
http_req_failed
,
http_reqs
by (name, status)
if requests are tagged with
name
; otherwise
by (url, status)
Different endpoints have different baselines; mixing them hides per-endpoint drift.
iteration_duration
(multi-scenario tests)
by (scenario)
Browser and protocol scenarios have very different durations; mixing them is meaningless.
Custom trends with tags
by (<the tag>)
Whatever the user tagged on is presumably what they care about.
Discover available labels first via the labels endpoint (metrics.md §4) and label-values endpoint (metrics.md §5) before constructing the grouped query. Don't assume labels from reading the script -- a tag rename or refactor can silently shift the label space. Example:
undefined
对于每个指标,使用多运行聚合端点(metrics.md第8节)查询窗口内所有运行的聚合值:
GET /cloud/v5/load_tests/{loadTestId}/query_aggregate_k6(
  query='<method>',
  metric='<metric_name>',
  test_run_ids=[{id1},{id2},...]
)
具体的
gcx
格式(遵循k6-manage第2节的代理前缀;
-o json
避免溢出信封):
bash
LT=<load_test_id>; IDS="123,124,125"
gcx --context <ctx> api "/api/plugins/k6-app/resources/cloud/cloud/v5/load_tests/$LT/query_aggregate_k6(query='histogram_quantile(0.95)',metric='http_req_duration',test_run_ids=[$IDS])" -o json
根据指标类型选择聚合方法:
指标类型主要方法捕获内容
趋势
histogram_quantile(0.95)
P95延迟——最常见的SLO目标
趋势
histogram_quantile(0.5)
中位数——展示典型行为
趋势
histogram_avg
平均值——对异常值敏感
计数器
increase
每次运行的总计数
比率
ratio
成功/失败比率
仪表
max
每次运行的峰值
对于趋势型指标(延迟),查询多个分位数(P50、P90、P95、P99),以了解退化是均匀的还是集中在尾部。
**对于多目标测试,始终按请求分组拆分。**整个测试的单一聚合P95会掩盖局限于某个端点或页面的回归——如果测试访问多个URL,某个URL的3倍变慢在测试级P95中可能不可见。默认使用分组查询:
指标模式默认分组原因
browser_web_vital_*
(LCP、FCP、CLS、TTFB、INP、FID)
by (url)
每次导航都会输出自己的Web Vital;按URL的趋势可将回归定位到特定页面。
http_req_duration
http_req_failed
http_reqs
如果请求标记了
name
by (name, status)
;否则
by (url, status)
不同端点有不同的基线;混合它们会隐藏端点级的漂移。
iteration_duration
(多场景测试)
by (scenario)
浏览器和协议场景的持续时间差异很大;混合它们没有意义。
带标签的自定义趋势
by (<the tag>)
用户标记的内容大概是他们关心的。
在构建分组查询之前,先通过标签端点(metrics.md第4节)和标签值端点(metrics.md第5节)发现可用的标签。不要假设通过阅读脚本就能知道标签——标签重命名或重构可能会悄然改变标签空间。示例:
undefined

List labels for a specific metric on a representative run

列出特定指标在代表性运行中的标签

GET /cloud/v5/test_runs/{id}/labels?match[]=browser_web_vital_lcp
GET /cloud/v5/test_runs/{id}/labels?match[]=browser_web_vital_lcp

Then enumerate values for a useful label

枚举有用标签的值

GET /cloud/v5/test_runs/{id}/label/url/values

Only fall back to the bare aggregate (no `by`) for metrics where grouping
adds no information -- e.g., `vus`, `load_generator_cpu_percent`,
single-target tests where every request hits the same URL.

The response includes `test_run_id` as a label -- use this to map each value
back to its run timestamp from Step 3. Grouped queries return one series per
(group, run) combination; flatten into a tidy "rid x group" table for the
trend computation in Step 7.

If there are too many run IDs to fit in a single URL (hundreds), batch the
queries into groups of 50 run IDs and merge the results.
GET /cloud/v5/test_runs/{id}/label/url/values

仅当分组无法增加信息时才回退到裸聚合(无`by`)——例如`vus`、`load_generator_cpu_percent`、所有请求都命中同一URL的单目标测试。

响应包含`test_run_id`作为标签——使用它将每个值映射回步骤3中的运行时间戳。分组查询会返回每个(组,运行)组合的一个序列;将其扁平化为整洁的“运行ID x 组”表格,用于步骤7的趋势计算。

如果运行ID太多无法放入单个URL(数百个),将查询分批为每组50个运行ID并合并结果。

Step 6: Extract thresholds

步骤6:提取阈值

Thresholds define what "passing" means. Fetch the test's current script (via k6-manage Section 5) and parse the
export const options = { thresholds: {...} }
block. Record each threshold's:
  • Metric name and selector (e.g.,
    http_req_duration{name:homepage}
    )
  • Condition (e.g.,
    p(95)<500
    )
  • Whether
    abortOnFail
    is set
Also check the most recent run's threshold results from the run data to see which thresholds are currently passing vs. failing.
Not all metrics will have thresholds -- that's fine. Metrics without thresholds still get trend analysis; they just won't have headroom calculations.
阈值定义了“通过”的标准。获取测试的当前脚本(通过k6-manage第5节)并解析
export const options = { thresholds: {...} }
块。记录每个阈值的:
  • 指标名称和选择器(例如
    http_req_duration{name:homepage}
  • 条件(例如
    p(95)<500
  • 是否设置
    abortOnFail
同时查看最近一次运行的阈值结果,了解哪些阈值当前是通过还是失败的。
并非所有指标都有阈值——这没关系。没有阈值的指标仍会进行趋势分析,只是不会计算余量。

Step 7: Compute trends

步骤7:计算趋势

Detect inflection points and rule out script changes deterministically. Before drawing conclusions about a regression, look for discontinuities in the metric values -- sudden jumps or drops that align across multiple metrics on the same date. When you spot one, do not guess whether the script changed. The run-bundled script endpoint gives a deterministic answer in seconds:
  1. sha256-diff the bundled scripts at the boundary. Fetch the snapshot from the last "before" run and the first "after" run via
    GET /cloud/v6/test_runs/{id}/script
    (k6-manage §5, "Two distinct script endpoints" -- use the run-scoped endpoint, not the load-test one, since the latter only shows the current version). Compare with
    shasum -a 256
    .
    bash
    gcx --context <ctx> api /api/plugins/k6-app/resources/cloud/cloud/v6/test_runs/<before_id>/script > /tmp/before.bin
    gcx --context <ctx> api /api/plugins/k6-app/resources/cloud/cloud/v6/test_runs/<after_id>/script > /tmp/after.bin
    shasum -a 256 /tmp/before.bin /tmp/after.bin
    For thoroughness on multi-run inflections, hash every run across the transition -- if N consecutive runs share one hash and N more share another, you have a clean before/after boundary. If hashes change mid-stream, the test was edited multiple times.
  2. If the sha256 differs, the test script changed -- the inflection may be a test-side artifact, not a service regression. Split the analysis into distinct eras at the boundary and compute trends within each era separately. Comparing metrics across script changes produces misleading trends -- a P95 drop from 3,000ms to 150ms is not an "improvement" if the script simply stopped hitting a slow endpoint. State the eras explicitly in the report and focus recommendations on the most recent era.
  3. If the sha256 matches, the script is byte-identical and the regression is external to the test (service-side, infrastructure, or load-zone). This is high-confidence information -- carry it into Step 9 (service-side correlation) instead of leaving "did the test change?" as an open question.
This sha256 diff is a 5-second deterministic check that rules out a huge class of causes. Run it at every detected inflection, not just when the user asks.
For each metric, build a time-ordered series of (run_timestamp, value) pairs. Then compute:
Basic statistics:
  • Mean, standard deviation, min, max across all runs
  • Current value (most recent run)
  • Baseline value (oldest run in window, or mean of first 3 runs for stability)
Trend direction: Split the runs into two halves (first half and second half of the time window). Compare the mean of each half:
  • Degrading: second-half mean is worse by >10% (higher for latency/errors, lower for success rates)
  • Improving: second-half mean is better by >10%
  • Stable: change is within 10%
  • Volatile: standard deviation exceeds 25% of the mean, regardless of direction
The 10% and 25% thresholds are starting points. If the user's test has very tight tolerances or very noisy metrics, adjust and explain the reasoning.
Rate of change: Express as percentage change per week:
((recent_mean - baseline_mean) / baseline_mean) * 100 / weeks_in_window
. This normalizes across different window sizes.
Anomaly detection: Flag any run where the metric value is more than 2 standard deviations from the overall mean. These are potential inflection points worth investigating individually.
**检测拐点并确定性地排除脚本变更。**在得出回归结论之前,查看指标值中的不连续点——同一日期多个指标出现的突然上升或下降。当发现这种情况时,不要猜测脚本是否变更。运行绑定的脚本端点可在几秒内给出确定性答案:
  1. **对边界处的绑定脚本进行sha256差异对比。**通过
    GET /cloud/v6/test_runs/{id}/script
    (k6-manage第5节,“两个不同的脚本端点”——使用运行范围的端点,而非负载测试端点,因为后者仅显示当前版本)获取最后一个“变更前”运行和第一个“变更后”运行的快照。使用
    shasum -a 256
    进行对比。
    bash
    gcx --context <ctx> api /api/plugins/k6-app/resources/cloud/cloud/v6/test_runs/<before_id>/script > /tmp/before.bin
    gcx --context <ctx> api /api/plugins/k6-app/resources/cloud/cloud/v6/test_runs/<after_id>/script > /tmp/after.bin
    shasum -a 256 /tmp/before.bin /tmp/after.bin
    为了全面分析多运行拐点,对过渡期间的每个运行进行哈希——如果N个连续运行共享一个哈希,接下来N个运行共享另一个哈希,那么你就得到了清晰的变更前后边界。如果哈希在中途变化,说明测试被多次编辑。
  2. 如果sha256不同,说明测试脚本已变更——拐点可能是测试端的产物,而非服务回归。在边界处将分析拆分为不同阶段,并分别计算每个阶段的趋势。跨脚本变更比较指标会产生误导性趋势——如果脚本只是停止访问慢端点,P95从3000ms降至150ms并非“改进”。在报告中明确说明阶段,并将建议重点放在最近的阶段。
  3. 如果sha256匹配,说明脚本字节完全相同,回归是测试外部的(服务端、基础设施或负载区域)。这是高可信度信息——将其带入步骤9(服务端关联),而非留下“测试是否变更?”的疑问。
这个sha256差异对比是一个5秒的确定性检查,可以排除大量原因。在每个检测到的拐点都运行此检查,而不仅仅是用户询问时。
对于每个指标,构建按时间排序的(运行时间戳,值)对序列。然后计算:
基本统计数据:
  • 所有运行的平均值、标准差、最小值、最大值
  • 当前值(最近一次运行)
  • 基线值(窗口内最早的运行,或前3次运行的平均值以保证稳定性)
趋势方向: 将运行分为两半(时间窗口的前半部分和后半部分)。比较每部分的平均值:
  • 退化:后半部分平均值变差超过10%(延迟/错误更高,成功率更低)
  • 改进:后半部分平均值变好超过10%
  • 稳定:变化在10%以内
  • 波动:标准差超过平均值的25%,无论方向如何
10%和25%的阈值是起点。如果用户的测试有非常严格的容差或非常嘈杂的指标,请调整并解释理由。
变化率: 以每周变化百分比表示:
((近期平均值 - 基线平均值) / 基线平均值) * 100 / 窗口内的周数
。这可以对不同窗口大小进行归一化。
异常检测: 标记任何指标值超出总体平均值2个标准差的运行。这些是值得单独调查的潜在拐点。

Step 8: Headroom analysis

步骤8:余量分析

For metrics that have thresholds defined (from Step 6), compute headroom:
headroom_pct = ((threshold_value - current_value) / threshold_value) * 100
Classify headroom:
  • Comfortable (>50%): well within limits
  • Adequate (20-50%): healthy but worth monitoring
  • Thin (<20%): at risk of breaching if trends continue
  • Breached (<0%): already failing
For degrading metrics with thin headroom, estimate when the threshold will be breached if the current rate of change continues:
weeks_until_breach = headroom_absolute / rate_of_change_per_week
This is a rough projection, not a prediction -- present it as "at the current rate of degradation, this metric could breach in approximately N weeks."
对于有阈值定义的指标(来自步骤6),计算余量:
余量百分比 = ((阈值值 - 当前值) / 阈值值) * 100
对余量进行分类:
  • 充足 (>50%):远在限制范围内
  • 健康 (20-50%):健康但值得监控
  • 紧张 (<20%):如果趋势持续,有突破风险
  • 已突破 (<0%):已失败
对于余量紧张的退化指标,估计如果当前变化率持续,阈值何时会被突破:
突破剩余周数 = 绝对余量 / 每周变化率
这是粗略的预测,而非精确预言——表述为“按照当前退化速度,此指标可能在约N周后突破阈值。”

Step 9: Service-side correlation (when warranted)

步骤9:服务端关联(必要时)

If Step 7 reveals degradation, offer (don't auto-run) to correlate with service-side data to separate service degradation (fix the service), test-environment changes (load-zone latency, LG exhaustion), and script changes (a slower edit). Present findings and ask first.
Hand off to
debug-with-grafana
(via
gcx
) to query the service's Prometheus metrics and Loki logs for the same window; look for service error-rate changes, upstream latency, resource pressure (CPU/memory/pools), and deploys coinciding with inflection points.
When the service has no observability (third-party or another team's service -- no Prometheus job, Loki stream, or probe), fall back to client-side signal, which still localises the regression:
  • Browser tests: compare Tempo iteration traces (k6-manage §7,
    /api/v1/tempo/api/search
    +
    /traces/{id}
    ) for a before/after run. The span-name rollup, slowest spans, and per-URL navigation durations pin the regression to a URL, locator action, or asset; web vitals are
    web_vital.*
    span attributes -- read them directly.
  • Protocol tests: compare
    http_req_duration by (name, status)
    and
    http_req_failed by (name, status)
    ; a regression on one named request points at that endpoint. Split
    http_req_waiting
    (server time) vs
    http_req_receiving
    (transfer) to separate slow processing from slow download.
  • Payload corroboration (both): query
    data_received
    /
    browser_data_received
    . Latency up + payload up -> server content changes; latency up + payload stable -> server processing changes; intermittent payload -> flaky cache or A/B test.
Prefer server-side correlation when available -- it answers "why" directly; the fallbacks answer "where" and "what kind of change", and make a good ticket for the owning team.
如果步骤7发现退化,主动提出(不要自动运行)与服务端数据关联,以区分服务退化(修复服务)、测试环境变更(负载区域延迟、LG耗尽)和脚本变更(较慢的编辑)。先呈现结果并询问用户。
转交至
debug-with-grafana
(通过
gcx
)查询服务的Prometheus指标和Loki日志,查找与拐点同时发生的服务错误率变化、上游延迟、资源压力(CPU/内存/池)和部署。
当服务无可观测性时(第三方或其他团队的服务——无Prometheus任务、Loki流或探针),回退到客户端信号,这仍能定位回归:
  • 浏览器测试:比较变更前后运行的Tempo迭代跟踪(k6-manage第7节,
    /api/v1/tempo/api/search
    +
    /traces/{id}
    )。跨度名称汇总、最慢跨度和按URL的导航持续时间可将回归定位到URL、定位器操作或资源;Web Vital是
    web_vital.*
    跨度属性——直接读取即可。
  • 协议测试:比较
    http_req_duration by (name, status)
    http_req_failed by (name, status)
    ;某个命名请求的回归指向该端点。拆分
    http_req_waiting
    (服务器时间)与
    http_req_receiving
    (传输时间),以区分慢处理和慢下载。
  • 负载验证(两者):查询
    data_received
    /
    browser_data_received
    。延迟上升+负载上升 -> 服务器内容变更;延迟上升+负载稳定 -> 服务器处理变更;负载间歇性变化 -> 缓存不稳定或A/B测试。
如果可用,优先使用服务端关联——它直接回答“为什么”;回退方法回答“在哪里”和“哪种类型的变更”,并可为负责团队生成良好的工单。

Step 10: Present the report

步骤10:呈现报告

Always present findings as a structured report. Never apply changes directly.

始终以结构化报告呈现结果。永远不要直接应用变更。

Report template

报告模板

Use this structure. Omit sections that don't apply (e.g., skip "Anomalies" if none were detected).
markdown
undefined
使用此结构。省略不适用的部分(例如,如果未检测到异常,跳过“异常”部分)。
markdown
undefined

Trend Analysis: {test_name}

趋势分析:{test_name}

Test ID: {test_id} Analysis window: {start_date} to {end_date} ({N} runs analyzed) Overall health: {Healthy | Watch | Degrading | Critical}
测试ID:{test_id} 分析窗口:{start_date}至{end_date}(分析了{N}次运行) 整体健康状况:{健康 | 关注 | 退化 | 严重}

Run Summary

运行摘要

PeriodRunsPassedFailedPass Rate
First halfNNNN%
Second halfNNNN%
时段运行次数通过次数失败次数通过率
前半段NNNN%
后半段NNNN%

Metric Trends

指标趋势

MetricTypeCurrentBaselineChangeTrendThresholdHeadroom
http_req_duration (P95)trend380ms250ms+52%Degrading500ms24%
http_req_failedrate0.8%0.3%+167%Degrading1%20%
http_reqscounter15,23015,100+0.9%Stable--
指标类型当前值基线值变化趋势阈值余量
http_req_duration (P95)趋势380ms250ms+52%退化500ms24%
http_req_failed比率0.8%0.3%+167%退化1%20%
http_reqs计数器15,23015,100+0.9%稳定--

Flagged Issues

标记的问题

1. {metric_name}: {classification}

1. {metric_name}:{分类}

  • Current: {value} | Baseline: {value} | Change: {pct}%
  • Threshold: {threshold} | Headroom: {pct}%
  • Rate of change: {pct}% per week
  • Projected breach: ~{N} weeks at current rate
  • Anomalous runs: {run_ids with dates, if any}
  • 当前值:{value} | 基线值:{value} | 变化:{pct}%
  • 阈值:{threshold} | 余量:{pct}%
  • 变化率:每周{pct}%
  • 预计突破时间:按当前速度约{N}周
  • 异常运行:{带日期的运行ID,如果有}

Threshold Recommendations

阈值建议

MetricCurrent ThresholdRecommendedRationale
http_req_durationp(95)<500p(95)<420Current P95 is 380ms; tightening to 420ms gives 10% headroom from current performance while surfacing further degradation early
指标当前阈值建议阈值理由
http_req_durationp(95)<500p(95)<420当前P95为380ms;收紧至420ms可从当前性能获得10%的余量,同时尽早发现进一步退化

Suggested Next Steps

建议的下一步行动

  • Investigate service side: P95 latency has increased 52% -- consider loading
    debug-with-grafana
    to check service health
  • Deep-dive run {run_id}: anomalous P95 spike on {date} -- consider loading
    k6-cloud-investigate-test
    for this run
  • Tighten thresholds: 2 metrics have >30% headroom that could be tightened -- consider loading
    k6-test-maintenance
    to apply changes
undefined
  • 调查服务端:P95延迟已增加52%——考虑加载
    debug-with-grafana
    检查服务健康状况
  • 深入分析运行{run_id}:{date}出现异常P95峰值——考虑加载
    k6-cloud-investigate-test
    分析此运行
  • 收紧阈值:有2个指标的余量超过30%可收紧——考虑加载
    k6-test-maintenance
    应用变更
undefined

Overall health classification

整体健康状况分类

Derive the overall health from the worst-case metric:
  • Healthy: all metrics stable or improving, headroom comfortable or adequate
  • Watch: at least one metric degrading but headroom still adequate
  • Degrading: at least one metric degrading with thin headroom
  • Critical: at least one metric has breached its threshold, or multiple metrics are degrading with thin headroom

从最差情况的指标推导整体健康状况:
  • 健康:所有指标稳定或改进,余量充足或健康
  • 关注:至少一个指标退化但余量仍健康
  • 退化:至少一个指标退化且余量紧张
  • 严重:至少一个指标已突破阈值,或多个指标退化且余量紧张

Threshold recommendations

阈值建议

When recommending threshold changes, follow these principles:
  • Only tighten, never loosen unless the user asks. Loosening thresholds masks problems.
  • Target 10-20% headroom above the recent P95 of the metric. Enough room for normal variance but tight enough to catch real degradation.
  • Use the second-half mean as the baseline, not the single most recent run (which could be an outlier).
  • Respect the user's intent: if thresholds are currently very loose (>100% headroom), they may be intentionally permissive. Mention the opportunity to tighten but don't push hard -- the user knows their context.
  • Consider grouped thresholds: if the test uses tag-based thresholds (e.g.,
    http_req_duration{name:homepage}
    ), recommend per-endpoint thresholds where the trends differ between endpoints.

建议阈值变更时遵循以下原则:
  • 仅收紧,不放宽,除非用户要求。放宽阈值会掩盖问题。
  • 目标为指标近期P95以上10-20%的余量。足够应对正常波动,但又足够严格以捕捉真实退化。
  • 使用后半段平均值作为基线,而非单次最近运行(可能是异常值)。
  • 尊重用户意图:如果当前阈值非常宽松(>100%余量),可能是用户故意设置的。提及收紧的机会但不要强行推荐——用户了解其上下文。
  • 考虑分组阈值:如果测试使用基于标签的阈值(例如
    http_req_duration{name:homepage}
    ),建议对端点间趋势不同的情况使用端点级阈值。

Gotchas

注意事项

IssueDetail
Zero-observation thresholdsA threshold with zero observations passes by default in k6. If a metric appears to pass but has no data, flag it -- the threshold is not actually being evaluated.
Metric type changes across runsIf a metric's type changed between runs (e.g., script refactor), the multi-run aggregate endpoint uses the latest type. Earlier runs queried with the wrong method return empty. Flag this if detected.
Incomplete runs skew trendsAborted or timed-out runs typically have shorter durations and fewer iterations, producing unrepresentative metric values. Exclude them by default.
LG resource metrics
load_generator_cpu_percent
and
load_generator_file_handles
trending up may indicate the test is outgrowing its load generator allocation, not that the service is degrading. Call this out separately.
Rate metric directionFor
ratio
-type rate metrics (like check pass rates), "degrading" means the value is decreasing (fewer passes), which is the opposite direction from latency metrics.
问题详情
零观测阈值k6中,零观测的阈值默认通过。如果某个指标看似通过但无数据,标记它——阈值实际上未被评估。
跨运行的指标类型变更如果指标类型在运行间变更(例如脚本重构),多运行聚合端点会使用最新类型。使用错误方法查询早期运行会返回空结果。如果检测到这种情况,标记它。
不完整运行影响趋势已中止或超时的运行通常持续时间更短、迭代次数更少,产生不具代表性的指标值。默认排除它们。
LG资源指标
load_generator_cpu_percent
load_generator_file_handles
趋势上升可能表明测试已超出负载生成器的分配,而非服务退化。单独指出这一点。
比率指标方向对于
ratio
类型的比率指标(如检查通过率),“退化”指值下降(通过次数减少),与延迟指标的方向相反。