level-up

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Take Your Agent to the Next Level

将你的Agent提升到新高度

This skill sets up your agent with the full LangWatch stack: tracing, prompt versioning, evaluation experiments, and agent simulation tests. Each step builds on the previous one.

本技能为你的Agent搭建完整的LangWatch技术栈：追踪、提示版本控制、评估实验以及Agent模拟测试。每个步骤都基于前一步构建。

Plan Limits

计划限制

LangWatch's free plan has limits on prompts, scenarios, evaluators, experiments, and datasets. When you hit a limit, the API returns

"Free plan limit of N reached..."

with an upgrade link.

How to handle:

Work within the limits. If 3 resources of the relevant type are allowed, create 3 meaningful ones, not 10.
Make every creation count: each one should demonstrate clear value.
Show what works FIRST. If you hit a limit, summarize what was accomplished and note that upgrading the plan raises it — point to the subscription settings on the platform (license settings instead, if
```
LANGWATCH_ENDPOINT
```
is set — self-hosted).
Do NOT delete existing resources to make room or repurpose an existing resource to evade the limit.

LangWatch的免费计划在提示词、场景、评估器、实验和数据集方面有使用限制。当达到限制时，API会返回

"Free plan limit of N reached..."

并附带升级链接。

处理方式：

在限制范围内操作。如果允许创建3个相关类型的资源，请创建3个有实际意义的资源，而非10个。
让每个资源都发挥价值：每个资源都应展示明确的作用。
先展示可行的内容。如果达到限制，总结已完成的工作，并说明升级计划可提升限制——指向平台的订阅设置（如果设置了
```
LANGWATCH_ENDPOINT
```
即自托管版本，则指向许可证设置）。
请勿删除现有资源腾出空间，或重新利用现有资源规避限制。

Prerequisites

前提条件

Use

langwatch docs <path>

to read documentation as Markdown. Some useful entry points:

bash

langwatch docs                                    # Docs index
langwatch docs integration/python/guide           # Python integration
langwatch docs integration/typescript/guide       # TypeScript integration
langwatch docs prompt-management/cli              # Prompts CLI
langwatch scenario-docs                           # Scenario docs index

Discover commands with

langwatch --help

and

langwatch <subcommand> --help

. List and get commands accept

--format json

for machine-readable output. Read the docs first instead of guessing SDK APIs or CLI flags.

If no shell is available, fetch the same Markdown over plain HTTP. Append

.md

to any docs path (e.g. https://langwatch.ai/docs/integration/python/guide.md). Index: https://langwatch.ai/docs/llms.txt. Scenario index: https://langwatch.ai/scenario/llms.txt

If anything fails or confuses you while following this skill (broken commands, docs that do not match reality, errors you had to work around), ask the user for permission and run

npx langwatch report --user-approved

with a

--title

and

--summary

(or

--session <transcript.jsonl>

) to send it to the LangWatch team. No login needed, secrets and personal data are redacted locally, and it directly shapes what gets fixed.

npx langwatch report --help

explains the options.

Projects and API keys: target a real project, not a personal one.

LangWatch has two kinds of project:

Team / shared projects: real projects inside an organization. Evaluations, experiments, prompts, datasets, simulations and instrumentation must always target one of these.
Personal projects: a private "My Workspace" scratch space tied to a single user. Never send a user's evaluations, experiments or production traces here: it is for personal exploration only and is easily confused with a real project.

And two ways to authenticate:

A project API key in
.env
(
```
LANGWATCH_API_KEY
```
): the credential everything in these skills uses. It is scoped to one real project. This is the default; prefer it unless the user explicitly asks for something else.
langwatch login --device
(AI-tools / SSO): a personal device session for wrapping coding assistants (
```
langwatch claude
```
,
```
langwatch codex
```
, …). It is NOT for evaluations, prompts, datasets, scenarios or SDK instrumentation, and it points at a personal workspace. Do not run it to set up the work in these skills.

So for anything in these skills: make sure

LANGWATCH_API_KEY

for a real, shared project is in the project's

.env

— most environments already have this provisioned. Do NOT run

langwatch login

to pick a project, and never default to a personal project. If

LANGWATCH_ENDPOINT

is set, they are self-hosted, use that endpoint instead of app.langwatch.ai.

使用

langwatch docs <path>

命令以Markdown格式阅读文档。以下是一些实用的入口：

bash

langwatch docs                                    # 文档索引
langwatch docs integration/python/guide           # Python集成指南
langwatch docs integration/typescript/guide       # TypeScript集成指南
langwatch docs prompt-management/cli              # 提示词CLI管理文档
langwatch scenario-docs                           # 场景文档索引

通过

langwatch --help

和

langwatch <subcommand> --help

查看可用命令。列表和获取类命令支持

--format json

参数以输出机器可读格式。编写代码前请先阅读文档，不要猜测SDK API或CLI参数。

如果没有Shell环境，可以通过普通HTTP获取相同的Markdown文档。在任意文档路径后添加

.md

即可（例如：https://langwatch.ai/docs/integration/python/guide.md）。文档索引：https://langwatch.ai/docs/llms.txt。场景文档索引：https://langwatch.ai/scenario/llms.txt

如果在使用本技能过程中遇到任何问题或困惑（命令失效、文档与实际不符、需要解决的错误），请先征得用户许可，然后运行

npx langwatch report --user-approved

命令，附带

--title

和

--summary

参数（或

--session <transcript.jsonl>

）将问题反馈给LangWatch团队。无需登录，机密信息和个人数据会在本地脱敏，反馈内容将直接用于问题修复。

npx langwatch report --help

可查看详细选项。

项目与API密钥：请针对真实项目，而非个人项目。

LangWatch有两种类型的项目：

团队/共享项目：组织内的真实项目。评估、实验、提示词、数据集、模拟和工具集成必须始终指向此类项目。
个人项目：绑定单个用户的私有“我的工作区”临时空间。请勿将用户的评估、实验或生产追踪数据发送到此处：它仅用于个人探索，容易与真实项目混淆。

有两种认证方式：

.env
文件中的项目API密钥（
```
LANGWATCH_API_KEY
```
）：本技能所有操作使用的凭证。它仅作用于一个真实项目。这是默认方式，除非用户明确要求其他方式，否则优先使用此方式。
langwatch login --device
（AI工具/单点登录）：用于包裹编码助手（
```
langwatch claude
```
、
```
langwatch codex
```
等）的个人设备会话。它不适用于评估、提示词、数据集、场景或SDK工具集成，且指向个人工作区。请勿使用它来完成本技能中的设置工作。

因此，对于本技能中的所有操作：确保真实共享项目的

LANGWATCH_API_KEY

已添加到项目的

.env

文件中——大多数环境已预先配置此密钥。请勿运行

langwatch login

命令选择项目，且永远不要默认使用个人项目。如果设置了

LANGWATCH_ENDPOINT

，说明是自托管版本，请使用该端点而非app.langwatch.ai。

Consultant Mode

顾问模式

After completing all steps, don't just stop — summarize everything you set up and suggest 2-3 ways to go deeper based on what you learned about the codebase. Detailed guidance:

After delivering initial results, transition to consultant mode to help the user get maximum value.

Phase 1: read first. Before generating ANY content: read the codebase end-to-end (every system prompt, function, tool definition), study git history for agent-related changes (

git log --oneline -30

, then drill into prompt/agent/eval-related commits because the WHY in commit messages matters more than the WHAT), and read READMEs and comments for domain context.

Phase 2: quick wins. Generate best-effort content based on what you learned. Run everything, iterate until green. Show the user what works and create the a-ha moment.

Phase 3: go deeper. Once Phase 2 lands, summarize what you delivered, then suggest 2-3 specific improvements grounded in the codebase: domain edge cases, areas that need expert terminology or real data, integration points (APIs, databases, file uploads), or regression patterns from git history that deserve test coverage. Ask light questions with options, not open-ended ("Want scenarios for X or Y?", "I noticed Z was a recurring issue. Add a regression test?", "Do you have real customer queries I could use?"). Respect "that's enough" and wrap up cleanly.

Do NOT ask permission before Phase 1 and 2. Deliver value first. Do NOT ask generic questions or overwhelm with too many suggestions. Do NOT generate generic datasets. Everything must reflect the actual domain.

完成所有步骤后，不要直接结束——总结所有已设置的内容，并根据对代码库的了解，提出2-3种深入优化的方向。详细指导：

交付初步成果后，切换到顾问模式，帮助用户获取最大价值。

阶段1：先阅读。在生成任何内容之前：完整阅读代码库（所有系统提示词、函数、工具定义），研究与Agent相关的Git历史记录（

git log --oneline -30

，然后深入查看与提示词/Agent/评估相关的提交，因为提交信息中的“原因”比“内容”更重要），并阅读README和注释以了解领域上下文。

阶段2：快速见效。根据所学内容生成最佳内容。运行所有步骤，迭代直到成功。向用户展示可行的结果，创造“恍然大悟”的时刻。

阶段3：深入优化。当阶段2的成果落地后，总结已交付的内容，然后基于代码库提出2-3个具体的改进建议：领域边缘案例、需要专业术语或真实数据的领域、集成点（API、数据库、文件上传），或Git历史记录中值得添加测试覆盖的回归模式。提出带有选项的轻量问题，而非开放式问题（比如“需要为X还是Y创建场景？”、“我注意到Z是一个反复出现的问题，是否添加回归测试？”、“你有真实的客户查询可以使用吗？”）。尊重用户“足够了”的反馈，并干净利落地结束工作。

在阶段1和阶段2之前无需征得许可。先交付价值。不要提出通用问题或给出过多建议。不要生成通用数据集。所有内容必须反映实际领域。

Step 1: Add Tracing

步骤1：添加追踪功能

Add LangWatch tracing to capture all LLM calls, costs, and latency.

Read the integration guide for this project's framework:

bash

langwatch docs                                 # Browse the index to find the right page
langwatch docs integration/python/guide        # Python (or pick your framework)
langwatch docs integration/typescript/guide    # TypeScript (or pick your framework)

Install the LangWatch SDK (

pip install langwatch

npm install langwatch

)

Add instrumentation following the framework-specific guide
Add
```
LANGWATCH_API_KEY
```
to
```
.env
```

Verify: Run the application briefly and confirm traces appear:

bash

langwatch trace search --limit 5 --format json

添加LangWatch追踪以捕获所有LLM调用、成本和延迟数据。

阅读对应项目框架的集成指南：

bash

langwatch docs                                 # 浏览索引找到对应页面
langwatch docs integration/python/guide        # Python（或选择你的框架）
langwatch docs integration/typescript/guide    # TypeScript（或选择你的框架）

安装LangWatch SDK（

pip install langwatch

或

npm install langwatch

）

按照框架专属指南添加工具集成代码
将
```
LANGWATCH_API_KEY
```
添加到
```
.env
```
文件中

验证：短暂运行应用程序，确认追踪数据已生成：

bash

langwatch trace search --limit 5 --format json

Step 2: Version Your Prompts

步骤2：版本化你的提示词

Move hardcoded prompts to LangWatch Prompts CLI for version control and collaboration.

Read the Prompts CLI docs:
bash
```
langwatch docs prompt-management/cli
```
Initialize:
```
langwatch prompt init
```
Create prompts:
```
langwatch prompt create <name>
```
for each prompt in the code
Update application code to use
```
langwatch.prompts.get("name")
```
instead of hardcoded strings
Sync:
```
langwatch prompt sync
```

Verify:

langwatch prompt list

(or check the Prompts section at https://app.langwatch.ai).

Do NOT hardcode prompts in code. Do NOT add try/catch fallbacks around

prompts.get()

将硬编码的提示词迁移到LangWatch Prompts CLI进行版本控制和协作。

阅读Prompts CLI文档：
bash
```
langwatch docs prompt-management/cli
```
初始化：
```
langwatch prompt init
```
创建提示词：对代码中的每个提示词运行
```
langwatch prompt create <name>
```
更新应用代码，使用
```
langwatch.prompts.get("name")
```
替代硬编码字符串
同步：
```
langwatch prompt sync
```

验证：运行

langwatch prompt list

（或查看https://app.langwatch.ai的提示词板块）。

请勿在代码中硬编码提示词。请勿在

prompts.get()

周围添加try/catch回退逻辑。

Step 3: Create an Evaluation Experiment

步骤3：创建评估实验

Build a batch evaluation to measure your agent's quality across many examples.

Read the experiments SDK docs:

bash

langwatch docs evaluations/experiments/sdk

Analyze the agent's code to understand what it does
Generate a dataset of 10-20 examples tailored to the agent's domain (NOT generic examples)
Create an experiment file:
- Python: Jupyter notebook with
```
langwatch.experiment.init()
```
  , evaluation loop, and evaluators
- TypeScript: Script with
```
langwatch.experiments.init()
```
  and
```
evaluation.run()
```
Include at least one evaluator (LLM-as-judge for quality is a good default)

Verify: Run the experiment (

jupyter nbconvert --to notebook --execute experiment.ipynb

npx tsx experiment.ts

) and check results appear in the LangWatch Experiments view.

构建批量评估，在多个示例中衡量Agent的质量。

阅读实验SDK文档：

bash

langwatch docs evaluations/experiments/sdk

分析Agent代码以了解其功能
生成10-20个针对Agent领域的数据集示例（请勿使用通用示例）
创建实验文件：
- Python：包含
```
langwatch.experiment.init()
```
  、评估循环和评估器的Jupyter笔记本
- TypeScript：包含
```
langwatch.experiments.init()
```
  和
```
evaluation.run()
```
  的脚本
至少包含一个评估器（默认使用LLM作为质量评判器是不错的选择）

验证：运行实验（

jupyter nbconvert --to notebook --execute experiment.ipynb

或

npx tsx experiment.ts

），并确认结果已显示在LangWatch实验视图中。

Step 4: Add Agent Simulation Tests

步骤4：添加Agent模拟测试

Create scenario tests to validate agent behavior in realistic multi-turn conversations.

Read the Scenario docs:

bash

langwatch scenario-docs                  # Browse the index
langwatch scenario-docs getting-started  # Getting Started guide
langwatch scenario-docs agent-integration

Install the Scenario SDK (

pip install langwatch-scenario

npm install @langwatch/scenario

)

Write scenario tests with

AgentAdapter

UserSimulatorAgent

, and

JudgeAgent

Use semantic criteria in JudgeAgent (NOT regex matching)

Verify: Run the tests (

pytest -s

npx vitest run

) and confirm they pass.

NEVER invent your own testing framework. Use

@langwatch/scenario

langwatch-scenario

创建场景测试，验证Agent在真实多轮对话中的行为。

阅读场景文档：

bash

langwatch scenario-docs                  # 浏览索引
langwatch scenario-docs getting-started  # 入门指南
langwatch scenario-docs agent-integration

安装Scenario SDK（

pip install langwatch-scenario

或

npm install @langwatch/scenario

）

使用

AgentAdapter

、

UserSimulatorAgent

和

JudgeAgent

编写场景测试

在JudgeAgent中使用语义判定标准（请勿使用正则匹配）

验证：运行测试（

pytest -s

或

npx vitest run

）并确认测试通过。

切勿自行开发测试框架。请使用

@langwatch/scenario

langwatch-scenario

。

Common Mistakes

常见错误

Do NOT skip any step -- each builds on the previous
Do NOT use generic datasets in the experiment -- tailor them to the agent's domain
Do NOT hardcode prompts -- use the Prompts CLI
Do NOT invent testing frameworks -- use Scenario
Do NOT skip verification steps -- run the application/experiment/tests after each step
Always read docs via
```
langwatch docs ...
```
/
```
langwatch scenario-docs ...
```
before writing code; do not work from memory of past framework versions

请勿跳过任何步骤——每个步骤都基于前一步构建
请勿在实验中使用通用数据集——请针对Agent领域定制
请勿硬编码提示词——使用Prompts CLI
请勿自行开发测试框架——使用Scenario
请勿跳过验证步骤——每个步骤完成后运行应用/实验/测试
编写代码前请始终通过
```
langwatch docs ...
```
/
```
langwatch scenario-docs ...
```
阅读文档；请勿依赖过往框架版本的记忆