cargo-observability

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Cargo CLI — Observability

Cargo CLI — 可观测性

Alerts. An alert is a scheduled threshold check. On every cron tick it measures a scope (what to watch), compares the measured value against a threshold (the breach condition), and on breach fires actions — each as its own run — and records an event. This is the proactive counterpart to

cargo-diagnostics

: diagnostics explains a failure after you notice it; an alert tells you the moment a metric crosses a line.

Everything lives under one CLI domain:

bash

cargo-ai observability alert   …   # the alert CRUD + preview surface
cargo-ai observability event   …   # an alert's firing history

告警。告警是一种定时阈值检查机制。每次cron触发时，它会测量一个监控范围（要监控的对象），将测量值与阈值（触发条件）进行比较，当超出阈值时会触发动作——每个动作都会独立运行，并记录一个事件。这是

cargo-diagnostics

的主动式补充：诊断功能是在你发现故障后解释原因；而告警会在指标超出阈值的瞬间主动通知你。

所有功能都位于一个CLI域下：

bash

cargo-ai observability alert   …   # 告警的增删改查 + 预览界面
cargo-ai observability event   …   # 告警的触发历史

The three moving parts of every alert

每个告警的三个核心组件

Part	Flag	What it is
Scope	`--scope <json>`	What to measure — one of six sources: `spans` , `runs` , `records` , `orchestrationQuery` , `storageQuery` , `model` .
Threshold	`--threshold <json>`	When it breaches — a `metric` + `operator` ( `gte` / `lte` ) + `value` . The metric menu depends on the scope.
Actions	`--actions <json>`	What happens on breach — an `Action[]` (connector / tool / agent / native nodes), each fired as its own run. Optional; omit for a silent alert whose breaches you read from its events.

The scope and threshold are a matched pair — a metric can only be computed over the scopes that produce it (e.g.

errorRate

needs telemetry,

freshness

needs a model). The full compatibility matrix, every metric's meaning and units, and every scope filter field are in references/scopes-and-thresholds.md
— read it before writing a

--scope

--threshold

pair you haven't used before.

组件	标识	说明
监控范围	`--scope <json>`	要测量的对象——六种来源之一： `spans` 、 `runs` 、 `records` 、 `orchestrationQuery` 、 `storageQuery` 、 `model` 。
阈值	`--threshold <json>`	触发条件——包含 `metric` （指标） + `operator` （操作符， `gte` / `lte` ） + `value` （数值）。可选的指标取决于监控范围。
动作	`--actions <json>`	触发后的操作——一个 `Action[]` 数组（连接器/工具/Agent/原生节点），每个动作都会独立运行。可选；若省略则为静默告警，需从事件中查看触发记录。

监控范围和阈值是匹配对——一个指标只能在能生成它的监控范围内计算（例如

errorRate

需要遥测数据，

freshness

需要模型）。完整的兼容性矩阵、每个指标的含义和单位、每个监控范围的过滤字段都记录在**

references/scopes-and-thresholds.md

**中——在编写未使用过的

--scope

--threshold

组合前，请先阅读该文档。

The golden rule:

preview

before you

create

黄金准则：创建前先预览

alert preview

evaluates a scope + threshold right now, without firing actions or writing an event. It returns the value the alert would measure and whether that value breaches — so you calibrate the threshold against reality instead of guessing, and you confirm the scope/threshold pairing is even valid before committing it to a schedule.

bash

cargo-ai observability alert preview \
  --scope '{"kind":"runs","workflowUuid":"<uuid>","statuses":["error"]}' \
  --threshold '{"metric":"errorRate","operator":"gte","value":10}' \
  --window-minutes 1440          # last 24h; default 60

outcome: "computed"

→

{ value, total, failed, isBreached }

. Set your threshold from

value

```
outcome: "empty"
```
→ the window had nothing to measure (see the empty-vs-zero rule in
```
references/alert-lifecycle.md
```
).
```
outcome: "notComputed"
```
→
```
{ errorMessage }
```
. A bad SQL query, a deleted model, or an invalid scope/threshold pairing all land here — fix it before creating.

--window-minutes

only shapes the window for telemetry scopes (

spans

runs

records

). A

model

is measured as it stands right now; a query scope windows itself in its SQL.

Always preview first. It is free, it is the only way to size a threshold correctly, and it catches an invalid pairing before it becomes a schedule that writes an

error

event every tick.

alert preview

会立即评估监控范围+阈值，不会触发动作或写入事件。它会返回告警将测量的值以及该值是否超出阈值——因此你可以根据实际情况校准阈值，而不是猜测，并在提交到定时任务前确认监控范围/阈值组合是否有效。

bash

cargo-ai observability alert preview \
  --scope '{"kind":"runs","workflowUuid":"<uuid>","statuses":["error"]}' \
  --threshold '{"metric":"errorRate","operator":"gte","value":10}' \
  --window-minutes 1440          # 最近24小时；默认60分钟

outcome: "computed"

→ 返回

{ value, total, failed, isBreached }

。根据

value

设置你的阈值。

```
outcome: "empty"
```
→ 监控窗口内没有可测量的数据（请查看
```
references/alert-lifecycle.md
```
中的空窗口与零值规则）。
```
outcome: "notComputed"
```
→ 返回
```
{ errorMessage }
```
。可能是SQL查询错误、模型已删除，或监控范围/阈值组合无效——创建前请修复该问题。

--window-minutes

仅适用于遥测类监控范围（

spans

runs

records

）。

model

会根据当前状态进行测量；查询类范围会在其SQL中定义窗口。

务必先预览。预览是免费的，是正确设置阈值的唯一方法，还能在定时任务每次触发都写入

error

事件之前发现无效的组合。

Commands

命令

All commands output JSON. Reads need a token with

observability:read

; create/update/remove need

observability:write

(an admin token has both; a plain member token may not — see Prerequisites).

所有命令均输出JSON。读取操作需要拥有

observability:read

权限的令牌；创建/更新/删除操作需要

observability:write

权限（管理员令牌拥有这两种权限；普通成员令牌可能没有——请查看前提条件）。

Create an alert

创建告警

bash

cargo-ai observability alert create \
  --name "CRM sync error rate" \
  --description "Page when the HubSpot sync starts failing" \
  --cron "*/30 * * * *" \
  --scope '{"kind":"runs","workflowUuid":"<workflow-uuid>","statuses":["error"]}' \
  --threshold '{"metric":"errorRate","operator":"gte","value":10}' \
  --actions '[{"kind":"agent","agentUuid":"<agent-uuid>","config":{"message":"{{alert.name}} breached: {{event.value}}% errors. {{alert.url}}"}}]'

```
--cron
```
— 5-field cron or
```
@every <interval>
```
(e.g.
```
@every 30m
```
), always UTC, at most once a minute. The UI presets bottom out at 30 minutes; go tighter only with reason (every tick scans ClickHouse and can fire paid runs).
```
--disabled
```
— create it paused (evaluate nothing until you
```
update --enabled true
```
).

--folder <uuid>

— file it under a folder (from

cargo-workspace-management

```
--actions
```
— optional. Omit for a silent alert. The config is templated against the firing context (
```
{{alert.*}}
```
,
```
{{event.*}}
```
) — see
```
references/alert-lifecycle.md
```
for the full variable list. Each action's target (
```
agentUuid
```
/
```
toolUuid
```
/
```
connectorUuid
```
) is validated to exist in the workspace at create time.

bash

cargo-ai observability alert create \
  --name "CRM同步错误率" \
  --description "当HubSpot同步开始失败时发送页面告警" \
  --cron "*/30 * * * *" \
  --scope '{"kind":"runs","workflowUuid":"<workflow-uuid>","statuses":["error"]}' \
  --threshold '{"metric":"errorRate","operator":"gte","value":10}' \
  --actions '[{"kind":"agent","agentUuid":"<agent-uuid>","config":{"message":"{{alert.name}}触发告警：{{event.value}}% 错误率。{{alert.url}}"}}]'

```
--cron
```
— 5字段cron表达式或
```
@every <interval>
```
（例如
```
@every 30m
```
），始终使用UTC时区，触发频率最高为每分钟一次。UI预设的最小间隔为30分钟；只有在必要时才设置更短的间隔（每次触发都会扫描ClickHouse，可能会触发付费运行）。
```
--disabled
```
— 创建时处于暂停状态（直到执行
```
update --enabled true
```
才会开始评估）。
```
--folder <uuid>
```
— 将告警归类到指定文件夹（来自
```
cargo-workspace-management
```
）。
```
--actions
```
— 可选。省略则为静默告警。配置内容可使用触发上下文进行模板化（
```
{{alert.*}}
```
、
```
{{event.*}}
```
）——完整的变量列表请查看
```
references/alert-lifecycle.md
```
。创建时会验证每个动作的目标（
```
agentUuid
```
/
```
toolUuid
```
/
```
connectorUuid
```
）是否存在于工作区中。

List, get, update, remove

列出、查询、更新、删除

bash

cargo-ai observability alert list                      # all alerts, each with its lastEvent
cargo-ai observability alert get <uuid>                # one alert + its lastEvent

cargo-ai observability alert update --uuid <uuid> \
  --enabled false                                      # pause it (true/false — must be literal)
cargo-ai observability alert update --uuid <uuid> \
  --threshold '{"metric":"errorRate","operator":"gte","value":20}'   # raise the bar
cargo-ai observability alert update --uuid <uuid> \
  --description none                                    # "none" clears; --folder none unfiles

cargo-ai observability alert remove <uuid>

--enabled

is strict: only the literal

true

false

are accepted —

--enabled yes

is rejected rather than silently disabling the alert. On

update

, any flag you omit is left unchanged;

--description none

--folder none

are the explicit "clear it" spellings.

bash

cargo-ai observability alert list                      # 所有告警，包含各自的最新事件
cargo-ai observability alert get <uuid>                # 查询单个告警 + 其最新事件

cargo-ai observability alert update --uuid <uuid> \
  --enabled false                                      # 暂停告警（值必须是字面量true/false）
cargo-ai observability alert update --uuid <uuid> \
  --threshold '{"metric":"errorRate","operator":"gte","value":20}'   # 提高阈值
cargo-ai observability alert update --uuid <uuid> \
  --description none                                    # "none"表示清除描述；--folder none表示移出文件夹

cargo-ai observability alert remove <uuid>

--enabled

有严格要求：仅接受字面量

true

或

false

——

--enabled yes

会被拒绝，而不会静默禁用告警。执行

update

时，未指定的参数会保持不变；

--description none

--folder none

是明确的“清除”写法。

Inspect firing history

查看触发历史

bash

cargo-ai observability event list <alertUuid>          # latest evaluation events, newest first

Each event carries

status

(

healthy

unhealthy

error

), the measured

value

, a snapshot of the

scope

threshold

actions

as they were when it fired (the alert can change afterwards),

runUuids

(the runs the actions spawned — feed these to

cargo-diagnostics

orchestration run get

), the evaluation window, and

errorMessage

for

error

events.

unhealthy

= breached and fired;

error

= the metric could not be computed.

bash

cargo-ai observability event list <alertUuid>          # 最新的评估事件，按时间倒序排列

每个事件包含

status

（

healthy

unhealthy

error

）、测量的

value

、触发时

scope

threshold

actions

的快照（告警后续可能会更改）、

runUuids

（动作触发的运行实例——可将其传入

cargo-diagnostics

或

orchestration run get

）、评估窗口，以及

error

事件对应的

errorMessage

。

unhealthy

表示超出阈值并触发了动作；

error

表示无法计算指标。

How evaluation actually works

评估的实际工作原理

The lifecycle — cron windows and the ClickHouse indexing lag, the at-most-once firing guarantee (an alert never re-fires on the same rows; a sustained breach is re-detected on the next tick), the empty-window-vs-real-zero rule that makes

lte

a dead-man's switch, and the full

{{alert.*}}

{{event.*}}

templating context — is documented in references/alert-lifecycle.md
. Read it before you rely on an alert for anything time-sensitive.

生命周期——cron窗口和ClickHouse索引延迟、最多触发一次的保证（告警不会针对同一数据重复触发；持续超出阈值会在下次触发时被重新检测）、区分空窗口与实际零值的规则（该规则使

lte

可作为心跳检测开关），以及完整的

{{alert.*}}

{{event.*}}

模板上下文——均记录在**

references/alert-lifecycle.md

**中。在依赖告警处理时间敏感的任务前，请先阅读该文档。

Worked recipes

实战示例

references/examples/recipes.md
— copy-paste starting points: error-rate pager, credit-budget guard, p95-latency watch, a dead-man's switch (

count lte 0

— alert when a workflow stops running), model freshness / empty-model alerts, and a custom SQL-query alert.

references/examples/recipes.md
— 可直接复制使用的示例：错误率页面告警、信用预算防护、p95延迟监控、心跳检测开关（

count lte 0

——当工作流停止运行时触发告警）、模型新鲜度/空模型告警，以及自定义SQL查询告警。

Declarative alternative:

defineAlert

(CDK)

声明式替代方案：

defineAlert

（CDK）

This skill is the imperative surface — one-off

cargo-ai observability alert …

calls. To manage an alert as code (in git, reproducible, deployed alongside the workflow it watches), use CDK's

defineAlert

builder instead — see

../cargo-cdk/SKILL.md

and "Declarative vs imperative" in the router. Same scope/threshold/action model; different authoring mode.

本技能是命令式操作界面——通过单次

cargo-ai observability alert …

调用完成操作。若要以代码形式管理告警（存储在git中、可重现、与所监控的工作流一同部署），请使用CDK的

defineAlert

构建器——详情请查看

../cargo-cdk/SKILL.md

以及路由中的“声明式与命令式对比”。两者使用相同的监控范围/阈值/动作模型；只是创作方式不同。

Cost discipline

成本管控

An alert's actions fire as real runs — if an action calls a paid connector action or an agent, every breach re-bills. A poorly-sized threshold on a tight cron can breach (and bill) every tick. Two safeguards:

Preview to size the threshold so it fires on genuine anomalies, not normal variance.
If an action node calls a credits-based provider action, treat it like any scheduled paid workflow: read that provider's playbook (esp. its Recurring use section) in
```
../cargo-gtm/provider-playbooks/
```
, and apply the spend rules in
```
../cargo-gtm/references/cost-discipline.md
```
. Prefer cheap notification actions (an agent that posts to Slack, a connector notification) over anything that fans out.

告警的动作会作为真实运行实例触发——如果动作调用了付费连接器或Agent，每次触发都会重新计费。阈值设置不当且cron间隔过短的告警可能会每次触发都超出阈值（并产生费用）。有两项防护措施：

通过预览校准阈值，使其仅在出现真正异常时触发，而非正常波动。
如果动作节点调用了基于信用额度的提供商动作，请将其视为定时付费工作流：阅读
```
../cargo-gtm/provider-playbooks/
```
中该提供商的操作手册（尤其是重复使用部分），并遵循
```
../cargo-gtm/references/cost-discipline.md
```
中的支出规则。优先选择低成本的通知动作（例如发送到Slack的Agent、连接器通知），而非会产生分支的操作。

Prerequisites

前提条件

See

../cargo/references/prerequisites.md

for install, login (

--oauth

--token

), JSON conventions, and error shapes. Verify with

cargo-ai whoami

first.

Alerts are guarded by

observability:read

observability:write

permissions. If a create/update/remove returns a permission error, the token lacks

observability:write

— use an admin token or have one granted (

cargo-workspace-management

安装、登录（

--oauth

--token

）、JSON约定和错误格式请查看

../cargo/references/prerequisites.md

。请先使用

cargo-ai whoami

验证。

告警受

observability:read

observability:write

权限保护。如果创建/更新/删除操作返回权限错误，说明令牌缺少

observability:write

权限——请使用管理员令牌或申请该权限（通过

cargo-workspace-management

）。

When the CLI surprises you

当CLI出现异常时

If a documented flag, scope field, or response shape doesn't match what you observe (a fix may have shipped, or the docs may have drifted), re-refresh the CLI and skills; if it still doesn't add up, file a report — it's read by the team:

bash

cargo-ai workspaceManagement report create \
  --title "<one-line summary>" \
  --description "<exact command(s), errorMessage verbatim, expected vs actual, UUIDs>"

如果文档中记录的标识、监控范围字段或响应格式与实际不符（可能已发布修复，或文档已过时），请重新刷新CLI和技能；若问题仍存在，请提交报告——团队会查看报告：

bash

cargo-ai workspaceManagement report create \
  --title "<一行摘要>" \
  --description "<准确的命令、错误信息原文、预期与实际情况、UUID>"

Presenting results

结果展示

../cargo/references/interaction.md

: lead with the outcome ("alert created, will page the on-call agent when the CRM sync's error rate hits 10% over 30 min"), summarize an alert or its events as a compact table, never dump raw

alert get

event list

JSON into the conversation.

请遵循

../cargo/references/interaction.md

：先说明结果（“告警已创建，当CRM同步的错误率在30分钟内达到10%时，会向值班Agent发送页面告警”），将告警或其事件总结为简洁的表格，切勿将原始的

alert get

event list

JSON直接粘贴到对话中。

cargo-observability

Original

Translation

Cargo CLI — Observability

Cargo CLI — 可观测性

The three moving parts of every alert

每个告警的三个核心组件

The golden rule:
`preview`
before you
`create`

黄金准则：创建前先预览

Commands

命令

Create an alert

创建告警

List, get, update, remove

列出、查询、更新、删除

Inspect firing history

查看触发历史

How evaluation actually works

评估的实际工作原理

Worked recipes

实战示例

Declarative alternative:
`defineAlert`
(CDK)

声明式替代方案：
`defineAlert`
（CDK）

Cost discipline

成本管控

Prerequisites

前提条件

When the CLI surprises you

当CLI出现异常时

Presenting results

结果展示

cargo-observability

Original

Translation

Cargo CLI — Observability

Cargo CLI — 可观测性

The three moving parts of every alert

每个告警的三个核心组件

The golden rule: preview before you create

黄金准则：创建前先预览

Commands

命令

Create an alert

创建告警

List, get, update, remove

列出、查询、更新、删除

Inspect firing history

查看触发历史

How evaluation actually works

评估的实际工作原理

Worked recipes

实战示例

Declarative alternative: defineAlert (CDK)

声明式替代方案：defineAlert（CDK）

Cost discipline

成本管控

Prerequisites

前提条件

When the CLI surprises you

当CLI出现异常时

Presenting results

结果展示

The golden rule:
`preview`
before you
`create`

Declarative alternative:
`defineAlert`
(CDK)

声明式替代方案：
`defineAlert`
（CDK）