log-analyzer

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Log Analyzer

日志分析器

Overview

概述

Log viewers filter lines. This skill finds patterns.

Use it when an agent runtime, tool process, gateway, scheduler, or other service has produced enough log output that individual

grep

hits no longer explain the system behavior. The analyzer parses standard log lines, normalizes repeated messages, groups errors by component and tool, and produces a structured anomaly report that is suitable for debugging sessions or cron digests.

The script is intentionally lightweight and portable: it uses only Python's standard library and works on any text log with timestamp, level, optional component, and message fields. It also has a best-effort fallback for unstructured logs.

日志查看器用于过滤行，而本技能用于发现模式。

当Agent运行时、工具进程、网关、调度器或其他服务生成的日志量过大，单独使用

grep

已无法解释系统行为时，即可使用本工具。分析器会解析标准日志行，归一化重复消息，按组件和工具分组错误，并生成适合调试会话或Cron摘要的结构化异常报告。

本脚本设计轻量且可移植：仅使用Python标准库，适用于包含时间戳、日志级别、可选组件和消息字段的任意文本日志。对于非结构化日志，它也提供了尽力而为的降级处理方案。

Quick Start

快速开始

bash

cd agent-ops/log-analyzer
python scripts/analyze_logs.py scan --log-file agent.log --since 24h

To write JSON for later Markdown rendering:

bash

python scripts/analyze_logs.py scan --log-file agent.log --since 24h --output scan.json
python scripts/analyze_logs.py report --scan scan.json --output report.md

For cron-compatible anomaly detection:

bash

python scripts/analyze_logs.py scan --log-file agent.log --since 24h --quiet

--quiet

exits 0 and prints nothing when no anomalies are found. If anomalies exist, it prints the JSON report so the scheduler can deliver the digest.

bash

cd agent-ops/log-analyzer
python scripts/analyze_logs.py scan --log-file agent.log --since 24h

如需生成JSON以便后续渲染为Markdown：

bash

python scripts/analyze_logs.py scan --log-file agent.log --since 24h --output scan.json
python scripts/analyze_logs.py report --scan scan.json --output report.md

用于兼容Cron的异常检测：

bash

python scripts/analyze_logs.py scan --log-file agent.log --since 24h --quiet

使用

--quiet

参数时，若未发现异常则以状态码0退出且无输出；若存在异常，则打印JSON报告，以便调度器发送摘要。

What It Detects

检测范围

Error clusters — the same normalized error message repeated 3+ times within the selected time window. URLs, IP addresses, and numbers are replaced with placeholders before grouping so repeated failures with changing IDs still cluster.
Rate limit hits — HTTP
```
429
```
,
```
rate limit
```
,
```
rate_limit
```
,
```
too many requests
```
, and
```
quota exceeded
```
patterns. The analyzer groups them by provider when it can detect a provider name.
Timeout patterns —
```
timeout
```
,
```
timed out
```
,
```
deadline exceeded
```
, and
```
connection timeout
```
. Results are grouped by detected tool name and include example URLs where present.
Tool failures — error lines grouped by tool name extracted from patterns such as
```
tool: terminal
```
,
```
tool_call: web_search
```
,
```
tool terminal failed
```
, or lines emitted by a
```
tools
```
component.
Session crashes — fatal errors, unhandled exceptions, stack traces,
```
Traceback
```
,
```
Exception
```
, and
```
segfault
```
markers. Multiline stack traces are grouped as one crash entry with nearby context.
Component breakdown — error and warning counts by component such as
```
gateway
```
,
```
agent
```
,
```
tools
```
,
```
cron
```
, or
```
unknown
```
.
Error timeline — error counts bucketed by hour to reveal spikes and regressions after deploys or scheduled jobs.

See

references/anomaly-types.md

for detection criteria and interpretation.

错误集群——在选定时间窗口内，同一归一化错误消息重复出现3次及以上。分组前会将URL、IP地址和数字替换为占位符，因此带有变化ID的重复故障仍会被归为一类。
速率限制触发——检测HTTP
```
429
```
、
```
rate limit
```
、
```
rate_limit
```
、
```
too many requests
```
和
```
quota exceeded
```
等模式。若能识别提供商名称，会按提供商分组。
超时模式——检测
```
timeout
```
、
```
timed out
```
、
```
deadline exceeded
```
和
```
connection timeout
```
等内容。结果按识别出的工具名称分组，包含示例URL（若存在）。
工具故障——从
```
tool: terminal
```
、
```
tool_call: web_search
```
、
```
tool terminal failed
```
等模式或
```
tools
```
组件输出的行中提取工具名称，并按工具名称分组错误行。
会话崩溃——检测致命错误、未处理异常、堆栈跟踪、
```
Traceback
```
、
```
Exception
```
和
```
segfault
```
标记。多行堆栈跟踪会被归为一个崩溃条目，并附带附近上下文。
组件故障统计——按
```
gateway
```
、
```
agent
```
、
```
tools
```
、
```
cron
```
或
```
unknown
```
等组件统计错误和警告数量。
错误时间线——按小时统计错误数量，揭示部署或定时任务后的错误峰值和回归情况。

检测标准和解读方式详见

references/anomaly-types.md

。

Log Format Support

日志格式支持

The parser handles standard log lines shaped like:

text

2026-07-06 12:30:45 ERROR [gateway] Connection refused
2026-07-06T12:30:45Z ERROR gateway: Connection refused
12:30:45 ERROR Connection refused

It recognizes

ERROR

WARN

WARNING

INFO

DEBUG

FATAL

, and

CRITICAL

levels. Components may appear in square brackets after the level or as

component:

after the level. Time-only lines are anchored to the current day (or to the supplied default date when called as a library). Unstructured lines fall back to best-effort line-by-line scanning, so obvious

ERROR

WARN

strings are still counted even when the timestamp cannot be parsed.

See

references/log-formats.md

for examples and guidance on adding custom patterns.

解析器支持以下格式的标准日志行：

text

2026-07-06 12:30:45 ERROR [gateway] Connection refused
2026-07-06T12:30:45Z ERROR gateway: Connection refused
12:30:45 ERROR Connection refused

它能识别

ERROR

、

WARN

、

WARNING

、

INFO

、

DEBUG

、

FATAL

和

CRITICAL

等日志级别。组件信息可能出现在级别后的方括号中，或作为

component:

形式跟在级别后。仅含时间的日志行默认锚定到当天（作为库调用时可指定默认日期）。非结构化日志行将降级为逐行扫描，因此即使无法解析时间戳，仍会统计明显的

ERROR

WARN

字符串。

添加自定义模式的示例和指导详见

references/log-formats.md

。

CLI Commands

CLI命令

scan

— analyze a log file for patterns

scan

— 分析日志文件中的模式

bash

python scripts/analyze_logs.py scan --log-file LOGFILE [--since TIME] [--output report.json] [--quiet]

Options:

```
--log-file LOGFILE
```
— required path to the log file.
```
--since TIME
```
— optional time window: minutes/hours/days/weeks, e.g.
```
30m
```
,
```
1h
```
,
```
24h
```
,
```
7d
```
,
```
2w
```
; default is all lines.
```
--output report.json
```
— write JSON to a file instead of stdout.
```
--quiet
```
— cron mode: suppress output when no anomalies are found.

bash

python scripts/analyze_logs.py scan --log-file LOGFILE [--since TIME] [--output report.json] [--quiet]

选项说明：

```
--log-file LOGFILE
```
— 必填参数，指定日志文件路径。
```
--since TIME
```
— 可选时间窗口：分钟/小时/天/周，例如
```
30m
```
、
```
1h
```
、
```
24h
```
、
```
7d
```
、
```
2w
```
；默认分析所有行。
```
--output report.json
```
— 将JSON写入文件而非输出到标准输出。
```
--quiet
```
— Cron模式：未发现异常时抑制输出。

report

— render Markdown from scan JSON

report

— 将扫描结果JSON渲染为Markdown

bash

python scripts/analyze_logs.py report --scan scan.json [--output report.md]

The report contains overview counts, one section per anomaly type, a component breakdown table, and an hourly error timeline.

bash

python scripts/analyze_logs.py report --scan scan.json [--output report.md]

报告包含概览统计、每种异常类型的单独章节、组件故障统计表和每小时错误时间线。

tail

— smart tail for recent lines

tail

— 智能查看最近日志行

bash

python scripts/analyze_logs.py tail --log-file LOGFILE [--lines N] [--since TIME]

The smart tail prints plain text with markers suitable for chat delivery:

normal lines: no marker
warnings:
```
⚠️
```
errors:
```
❌
```
repeated recent errors:
```
🔥
```
when the same normalized error is seen 3+ times

bash

python scripts/analyze_logs.py tail --log-file LOGFILE [--lines N] [--since TIME]

智能查看功能会输出带有适合聊天场景标记的纯文本：

正常行：无标记
警告：
```
⚠️
```
错误：
```
❌
```
近期重复错误：同一归一化错误出现3次及以上时标记为
```
🔥
```

Output Format

输出格式

scan

emits JSON with these top-level fields:

```
log_file
```
,
```
lines_analyzed
```
,
```
time_window
```

anomalies.error_clusters

anomalies.rate_limits

anomalies.timeouts

anomalies.tool_failures

anomalies.crashes

```
component_breakdown
```
```
error_timeline
```
```
total_errors
```
,
```
total_warnings
```
,
```
has_anomalies
```

report

converts that JSON into Markdown:

markdown

undefined

scan

命令输出的JSON包含以下顶级字段：

```
log_file
```
、
```
lines_analyzed
```
、
```
time_window
```

anomalies.error_clusters

、

anomalies.rate_limits

、

anomalies.timeouts

、

anomalies.tool_failures

、

anomalies.crashes

```
component_breakdown
```
```
error_timeline
```
```
total_errors
```
、
```
total_warnings
```
、
```
has_anomalies
```

report

命令将上述JSON转换为Markdown：

markdown

undefined

Log Analysis Report

日志分析报告

Overview

概述

Log file: agent.log
Lines analyzed: 1542
Time window: 24h
Total errors: 12 | warnings: 20
Anomalies detected: 5 types

undefined

日志文件：agent.log
分析行数：1542
时间窗口：24h
总错误数：12 | 警告数：20
检测到的异常类型：5种

undefined

Scheduled Summary Integration

定时摘要集成

For a scheduled digest, run

scan

in quiet mode and include the output only when it is non-empty:

bash

python scripts/analyze_logs.py scan --log-file /var/log/agent.log --since 24h --quiet --output /tmp/log-scan.json
if [ -s /tmp/log-scan.json ]; then
  python scripts/analyze_logs.py report --scan /tmp/log-scan.json
fi

scheduled-summary

job can append the Markdown output under a "Log anomalies" heading. Keep the analysis window aligned with the summary window (for example, 24 hours for a daily digest) so counts do not overlap or disappear.

如需生成定时摘要，可在静默模式下运行

scan

，仅当输出非空时才生成报告：

bash

python scripts/analyze_logs.py scan --log-file /var/log/agent.log --since 24h --quiet --output /tmp/log-scan.json
if [ -s /tmp/log-scan.json ]; then
  python scripts/analyze_logs.py report --scan /tmp/log-scan.json
fi

schedule-summary

任务可将Markdown输出追加到“日志异常”标题下。请保持分析窗口与摘要窗口一致（例如每日摘要对应24小时分析窗口），避免计数重叠或遗漏。

Common Pitfalls

常见陷阱

Log rotation breaks time windows. If yesterday's file was rotated out, a
```
--since 24h
```
scan over only the current file may miss early-window failures. Point the scheduler at the active file plus rotated file, or concatenate the relevant files before scanning.
Multiline stack traces need the first line. The parser groups indented stack-trace continuation lines under the preceding parsed log line. If a log collector strips the first
```
Traceback
```
or
```
ERROR
```
line, the remaining stack frames become unstructured context.
Expected errors can be false positives. Retries, probing, and health checks may intentionally emit warnings or connection failures. Treat clusters as "investigate" signals, not automatic incidents.
First run on a large historical log can overwhelm output. Start with
```
--since 24h
```
, inspect the report, then widen the window if needed.
Changing message formats can split clusters. If an application changes an error string during a deploy, pre- and post-deploy failures may appear as two clusters even when the root cause is the same.
Time-only logs depend on the scan date.
```
12:30:45 ERROR ...
```
lines do not contain a date. For historical files, prefer full timestamps.
The error timeline buckets by hour-of-day, not date. On a multi-day window (
```
--since 7d
```
), errors from 08:00 on different days merge into one
```
08:00
```
bar. Use the clusters (which carry full timestamps) for multi-day forensics; treat the timeline as a time-of-day profile.
Crash detection is substring-based. The crash regex matches words like
```
fatal
```
or
```
Exception
```
anywhere in a line, so mentions inside INFO lines (e.g. "retry succeeded after TimeoutException", "non-fatal warning") count as crash signals and can set
```
has_anomalies
```
. Treat crash counts as leads to eyeball, and tune the regex if your logs legitimately chat about exceptions at INFO level.

日志轮转会破坏时间窗口：如果昨天的日志文件已被轮转，仅对当前文件执行
```
--since 24h
```
扫描可能会遗漏窗口初期的故障。请将调度器指向活动文件加轮转文件，或在扫描前合并相关文件。
多行堆栈跟踪需要首行：解析器会将缩进的堆栈跟踪续行归到前一条已解析的日志行下。如果日志收集器剥离了首行
```
Traceback
```
或
```
ERROR
```
，剩余的堆栈帧将成为非结构化上下文。
预期错误可能导致误报：重试、探测和健康检查可能会有意输出警告或连接失败信息。请将集群视为“需要调查”的信号，而非自动触发的事件。
首次扫描大型历史日志可能输出过载：先使用
```
--since 24h
```
开始，检查报告后再根据需要扩大窗口。
消息格式变化会拆分集群：如果应用在部署期间修改了错误字符串，部署前后的故障可能会被分为两个集群，即使根本原因相同。
仅含时间的日志依赖扫描日期：
```
12:30:45 ERROR ...
```
格式的行不含日期。对于历史文件，优先使用完整时间戳。
错误时间线按一天中的小时分桶，而非日期：在多日窗口（如
```
--since 7d
```
）下，不同日期08:00的错误会合并到同一个
```
08:00
```
条目中。多日取证请使用带有完整时间戳的集群；时间线仅作为一天中时段的参考。
崩溃检测基于子字符串匹配：崩溃正则表达式会匹配行中任意位置的
```
fatal
```
或
```
Exception
```
等词，因此INFO行中的提及（例如“retry succeeded after TimeoutException”、“non-fatal warning”）也会被视为崩溃信号，并设置
```
has_anomalies
```
。请将崩溃计数作为排查线索，若日志中确实会在INFO级别提及异常，请调整正则表达式。

What This Skill Is NOT

本技能不具备的能力

Not a log viewer: use
```
tail
```
,
```
less
```
, or a log UI when you need raw line inspection.
Not a log shipper: it does not forward logs to storage or observability systems.
Not a SIEM: it does not correlate identities, networks, or security events.
Not real-time monitoring:
```
tail
```
is a recent-line analyzer, not a daemon or alerting service.
Not a root-cause oracle: it highlights patterns so an agent or operator can investigate faster.

不是日志查看器：需要查看原始行时，请使用
```
tail
```
、
```
less
```
或日志UI。
不是日志转发器：不会将日志转发到存储或可观测性系统。
不是SIEM系统：不会关联身份、网络或安全事件。
不是实时监控工具：
```
tail
```
是近期行分析工具，而非守护进程或告警服务。
不是根因分析预言机：它仅突出显示模式，帮助Agent或运维人员更快开展调查。

Verification Checklist

验证清单

Run
```
python scripts/analyze_logs.py --help
```
and confirm subcommands load.
Scan a synthetic log with repeated errors and confirm
```
has_anomalies
```
is true.
Render Markdown from the scan JSON and confirm all anomaly sections appear.

Run

python -m pytest tests/test_analyze_logs.py -v

from this skill directory.

For cron use, test a no-anomaly log with
```
--quiet
```
and confirm stdout is empty with exit code 0.

运行
```
python scripts/analyze_logs.py --help
```
，确认子命令加载正常。
扫描包含重复错误的合成日志，确认
```
has_anomalies
```
为true。
将扫描结果JSON渲染为Markdown，确认所有异常章节均已显示。

在本技能目录下运行

python -m pytest tests/test_analyze_logs.py -v

。

针对Cron使用场景，测试无异常日志的
```
--quiet
```
模式，确认标准输出为空且退出码为0。

log-analyzer

Original

Translation

Log Analyzer

日志分析器

Overview

概述

Quick Start

快速开始

What It Detects

检测范围

Log Format Support

日志格式支持

CLI Commands

CLI命令

`scan`
— analyze a log file for patterns

`scan`
— 分析日志文件中的模式

`report`
— render Markdown from scan JSON

`report`
— 将扫描结果JSON渲染为Markdown

`tail`
— smart tail for recent lines

`tail`
— 智能查看最近日志行

Output Format

输出格式

Log Analysis Report

日志分析报告

Overview

概述

Scheduled Summary Integration

定时摘要集成

Common Pitfalls

常见陷阱

What This Skill Is NOT

本技能不具备的能力

Verification Checklist

验证清单