Back to Details

evaluations

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Route an Evaluation Request

路由评估请求

This is a compatibility skill. Do not build an experiment, monitor, or guardrail from this skill.
Classify the user's intent:
IntentCorrect skill
Batch test a dataset, compare prompts or models, benchmark, create a CI quality gate
experiments
Score live traces or threads, monitor production quality, create a guardrail
online-evaluations
If the request remains ambiguous after inspecting context — a bare "make me an eval" that names neither a dataset nor live traffic — do not create anything yet. This choice picks what gets tested, so it is the user's to make, not a default's. Ask it as a question card and stop; the answer arrives as the next message.
Where 
langy-card
blocks render, ask it as a 
choices
block — the only sanctioned question format — last in the reply:
markdown
```langy-card
{
  "kind": "choices",
  "blockId": "eval-kind",
  "question": "What should this evaluate?",
  "options": [
    { "id": "experiment", "label": "A dataset, before deployment" },
    { "id": "online", "label": "Live production traffic" }
  ]
}
```
Neither option names an existing entity, so neither carries a 
ref
. Without that channel, ask the same question as one short line of prose.
A rejected field value is not this kind of choice. If a create later fails with a 
validation_error
whose reason names the field and an 
expected
list, correct that exact field from the list and retry once — never turn a fixable slug into a question for the user.
Then hand off:
  1. If the correct companion skill is available, load it and follow it instead of continuing here.
  2. If 
    experiments
    is missing, tell the user to install it with:
    bash
    npx skills@1.5.19 add langwatch/skills/experiments
  3. If 
    online-evaluations
    is missing, tell the user to install it with:
    bash
    npx skills@1.5.19 add langwatch/skills/online-evaluations
Do not recreate the companion skill's instructions from memory. Load the focused skill so its current workflow, safety checks, and verification steps are used.
这是一个兼容skill。请勿基于此skill构建实验、监控或防护机制。
对用户的意图进行分类：
意图对应skill
批量测试数据集、比较提示词或模型、基准测试、创建CI质量门
experiments
为实时追踪或对话线程评分、监控生产环境质量、创建防护机制
online-evaluations
如果检查上下文后请求仍不明确——比如仅提出“帮我做一个评估”，既未指定数据集也未提及实时流量——则暂不创建任何内容。评估对象的选择应由用户决定，而非默认选项。以问题卡片的形式询问用户并停止操作；用户的回复将作为下一条消息。
在
langy-card
块渲染的位置，以
choices
块的形式提出问题——这是唯一认可的提问格式——放在回复的最后：
markdown
```langy-card
{
  "kind": "choices",
  "blockId": "eval-kind",
  "question": "What should this evaluate?",
  "options": [
    { "id": "experiment", "label": "A dataset, before deployment" },
    { "id": "online", "label": "Live production traffic" }
  ]
}
```
注意，两个选项均未提及现有实体，因此都不包含
ref
字段。如果没有该通道，则用一行简短的文字提出相同的问题。
被拒绝的字段值不属于此类选择。如果后续创建操作因
validation_error
失败，且错误原因指明了字段和
expected
列表，则从列表中修正该特定字段并重试一次——绝不要将可修复的错误转化为向用户提出的问题。
然后进行移交：
  1. 如果对应的配套skill可用，则加载该skill并遵循其流程，而非继续使用当前工具。
  2. 如果
    experiments
    缺失，请告知用户使用以下命令安装：
    bash
    npx skills@1.5.19 add langwatch/skills/experiments
  3. 如果
    online-evaluations
    缺失，请告知用户使用以下命令安装：
    bash
    npx skills@1.5.19 add langwatch/skills/online-evaluations
请勿凭记忆重现配套skill的说明。加载指定的skill以使用其当前的工作流、安全检查和验证步骤。