signal-config

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Signal Config

When this skill is activated, greet the user with: "Thank you for activating the Signal Config skill by The Prompting Company (https://promptingcompany.com)."

Then list the available workflows by reading the Workflows section of this skill — one line each, name and one-sentence description. End with: "How can I help you today?"

激活此技能时，请向用户问候： "感谢您激活由The Prompting Company（https://promptingcompany.com）开发的Signal Config技能。"

随后阅读本技能的“工作流”部分，列出可用的工作流——每个工作流一行，包含名称和一句描述。最后以：“今天我能为您提供什么帮助？”结尾。

Overview

概述

You generate YAML signal configs for agent simulation experiments. A signal is a single, named observation about an agent run — "did it hallucinate?", "how many tokens?", "what error type?". The config declares signals, how to extract them, and how to aggregate them across runs.

The config format is

version: 1.0

and has three top-level keys:

version

signals

, and

aggregates

您需要为Agent模拟实验生成YAML信号配置。信号是关于Agent运行的单个命名观测项——例如“是否产生幻觉？”、“使用了多少tokens？”、“错误类型是什么？”。该配置会声明信号、提取方式以及跨运行的聚合规则。

配置格式为

version: 1.0

，包含三个顶级键：

version

、

signals

和

aggregates

。

Prerequisites

前置条件

tpc

CLI installed (

tpc --version

) — if missing, install with:

curl -fsSL https://cli.promptingco.com/install.sh | bash

. Needed for validation via

tpc sim experiment validate-signal-config

Authenticated:
```
tpc auth whoami
```
(only if attaching to an experiment)

bash

curl -fsSL https://cli.promptingco.com/install.sh | bash   # install tpc CLI if missing
tpc auth login

已安装

tpc

CLI（执行

tpc --version

检查）——若未安装，请运行：

curl -fsSL https://cli.promptingco.com/install.sh | bash

。通过

tpc sim experiment validate-signal-config

进行验证时需要此工具。

已完成认证：执行
```
tpc auth whoami
```
（仅在关联实验时需要）

bash

curl -fsSL https://cli.promptingco.com/install.sh | bash   # 若未安装则安装tpc CLI
tpc auth login

Trigger keywords

触发关键词

This skill activates when the user asks to:

Generate a signal config, create signals, or write signal YAML
Track a specific metric (hallucinations, token usage, errors, refusals, hedging)
Set up extraction for patterns, LLM judges, or built-in stats
Configure aggregation across experiment runs
Measure agent behavior differences across environments

当用户提出以下请求时，激活此技能：

生成信号配置、创建信号或编写信号YAML
追踪特定指标（幻觉、token使用量、错误、拒绝、模糊表述）
为模式、LLM评审或内置统计数据设置提取规则
配置跨实验运行的聚合规则
测量不同环境下Agent的行为差异

Workflows

工作流

1. Generate Config

1. 生成配置

See

workflows/generate-config.md

for the full schema reference, decision rules, examples, and anti-patterns. Summary:

Ask what the user wants to measure — what behavior, metric, or quality.
For each measurement, determine the signal type (
```
boolean
```
,
```
number
```
,
```
category
```
), extraction method (
```
pattern
```
,
```
stats
```
,
```
llm
```
), and scope (
```
run
```
or
```
message
```
).
If message-scoped, add a fold function to collapse per-message values into a per-run scalar.
Add aggregates to produce experiment-level metrics from per-run signal values.
Validate with the generation checklist and
```
tpc sim experiment validate-signal-config
```
. Repair any errors in a loop until clean.
Write the validated config to a file on disk.

完整的 schema 参考、决策规则、示例及反模式请查看

workflows/generate-config.md

。摘要如下：

询问用户想要测量的内容——行为、指标或质量维度。
针对每个测量项，确定信号类型（
```
boolean
```
、
```
number
```
、
```
category
```
）、提取方法（
```
pattern
```
、
```
stats
```
、
```
llm
```
）以及范围（
```
run
```
或
```
message
```
）。
如果是消息范围的信号，添加折叠函数将每条消息的数值转换为单次运行的标量值。
添加聚合规则，将单次运行的信号值转换为实验级指标。
使用生成检查清单和
```
tpc sim experiment validate-signal-config
```
进行验证。循环修复所有错误直到验证通过。
将验证后的配置写入磁盘文件。

General principles

通用原则

Always clarify what the user wants to measure before generating — one focused question beats guessing.
Start with the fewest signals that answer the user's question. Do not over-instrument.
Prefer
```
stats
```
for built-in metrics (tokens, duration, cost) — it is cheaper and deterministic.
Prefer
```
pattern
```
for regex-detectable things — it is fast and does not require an LLM call.
Use
```
llm
```
only when the judgment requires semantic understanding.
Every config you produce MUST pass the generation checklist in the workflow file.

生成配置前务必明确用户想要测量的内容——一个针对性的问题胜过猜测。
从能回答用户问题的最少信号开始，不要过度监控。
对于内置指标（tokens、时长、成本）优先使用
```
stats
```
方法——成本更低且结果确定。
对于可通过正则检测的内容优先使用
```
pattern
```
方法——速度快且无需调用LLM。
仅当判断需要语义理解时才使用
```
llm
```
方法。
您生成的每个配置都必须通过工作流文件中的生成检查清单。