operating-livekit-agents
Compare original and translation side by side
🇺🇸
Original
English🇨🇳
Translation
ChineseOperating LiveKit agents
运维LiveKit agents
Everything after the agent works: getting a version onto LiveKit Cloud, keeping it fast and alive
under real load, and changing it without breaking what's already running. The commands live under
; read and each subcommand's help rather than trusting this skill for
flags — it deliberately doesn't restate them. has the deployment and
observability docs.
lk agentlk agent --helpreading-livekit-docs当agent正常工作之后的所有内容:将版本发布到LiveKit Cloud,在真实负载下保持其快速和存活,以及在不破坏正在运行内容的情况下进行修改。命令位于下;请阅读和每个子命令的帮助,而不是依赖本skill中的标志——本skill有意不重述这些标志。包含部署和可观测性文档。
lk agentlk agent --helpreading-livekit-docsDeploying to LiveKit Cloud
部署到LiveKit Cloud
The shape is stable even as the flags move:
- A project directory is bound to an agent by a config file the CLI writes (). Commands run from that directory find the agent without an id.
livekit.toml - The agent ships as a container. The CLI can generate a Dockerfile for the project, or you
bring a prebuilt image. Run the container's start command locally () before the first deploy — it's production mode, with production logging and a shutdown drain, and it is not what
lk agent startmode runs.dev - Each deploy creates a version and rolls it out. ,
status, andversions(build logs and deploy logs are separate) tell you what's live and why a rollout failed.logs - Secrets are injected as environment variables and managed apart from the code — never baked into the image or committed. Changing secrets restarts the agent.
- Rollback returns to a previous version. How instant that is depends on the plan; the docs say. Know the rollback command before you need it.
After a deploy, verify with the same tools you'd use on a stranger's agent: for the
rollout, for the first minutes, and a real conversation — can
run a scenario file against the deployed agent by name, which is the cheapest end-to-end check that
the thing serving traffic is the thing you meant to ship.
statuslogsrunning-livekit-simulations即使标志发生变化,结构也是稳定的:
- **项目目录通过CLI写入的配置文件()绑定到一个agent。**从该目录运行的命令无需id即可找到agent。
livekit.toml - **agent以容器形式发布。**CLI可以为项目生成Dockerfile,也可以使用预构建镜像。在首次部署前,先在本地运行容器的启动命令()——这是生产模式,具有生产日志和关闭排空,并且不是
lk agent start模式所运行的内容。dev - 每次部署都会创建一个版本并进行滚动发布。、
status和versions(构建日志和部署日志是分开的)告诉你当前正在运行什么以及为什么滚动发布失败。logs - 密钥作为环境变量注入,并与代码分开管理——永远不要将其烘焙到镜像中或提交到代码库。更改密钥会重启agent。
- **回滚会返回到以前的版本。**即时性取决于套餐;文档中有说明。在需要之前就知道回滚命令。
部署后,使用与检查陌生agent相同的工具进行验证:查看滚动状态,查看前几分钟,以及一次真实的对话——可以按名称对已部署的agent运行场景文件,这是确认服务流量的东西正是你打算发布的内容的最经济的端到端检查。
statuslogsrunning-livekit-simulationsThe worker process model
worker进程模型
Both SDKs run sessions in worker processes spawned from a parent. Misunderstanding this is the
single most common source of production-only bugs.
The parent prewarms; children inherit. Load expensive, read-only resources — VAD and turn
detection models, persistent clients — once in the parent through the SDK's prewarm hook, and
each session inherits them without re-loading. Everything shared this way must be read-only or
concurrency-safe; mutating parent state from a child is undefined behavior. Don't prewarm
session-specific state, and don't prewarm what costs more memory than it saves — every byte in the
parent is in every child's footprint. Then verify the child actually uses the prewarmed
instance: the classic mistake is prewarming a model and having session code load a fresh one
anyway, so the prewarm did nothing and startup is still slow.
The framework owns the event loop. Never create a new async runtime inside a worker, and never
block on an async call from a synchronous constructor to force a result. If initialization needs
async work, load lazily on first use from an already-async method, or split construction from an
awaited step. When you see errors about events bound to a different loop, a loop
already running, tasks destroyed while pending, or a closed loop that can't be reused, the cause is
almost always one of those two things — trace back to where a runtime was created or a sync path
awaited something.
initialize两个SDK都在从父进程派生的worker进程中运行会话。误解这一点是生产环境独有bug最常见的来源。
父进程预热;子进程继承。通过SDK的预热钩子在父进程中加载一次昂贵的只读资源——VAD和语音活动检测模型、持久客户端——每个会话都会继承它们,无需重新加载。以这种方式共享的所有内容必须是只读的或并发安全的;从子进程修改父进程状态是未定义行为。不要预热特定于会话的状态,也不要预热消耗的内存比节省的更多的内容——父进程中的每个字节都在每个子进程的内存占用中。然后验证子进程确实使用了预热的实例:经典的错误是预热了模型,但会话代码仍然加载了一个新模型,因此预热没有起作用,启动仍然很慢。
**框架拥有事件循环。**永远不要在worker内部创建新的异步运行时,也不要从同步构造函数阻塞异步调用来强制获得结果。如果初始化需要异步工作,在第一次使用时从已经是异步的方法中懒加载,或者将构造与一个await的步骤分开。当你看到有关事件绑定到不同循环、循环已在运行、任务在挂起时被销毁或循环已关闭无法复用的错误时,原因几乎总是这两种情况之一——回溯到创建运行时或同步路径等待某个东西的位置。
initializeProviders
Providers
STT, TTS, LLM, VAD and any backend will fail in production: rate limits, timeouts, overload,
outages. Set timeouts — a call that hangs is worse than one that fails fast. Distinguish transient
failures worth retrying from persistent ones that need a fallback or escalation. Degrade to a
meaningful spoken response, never to silence. Log provider response times, because rising latency
is usually the first sign of an outage.
Adding a provider to an existing agent: check whether the SDK already has a plugin before writing
one; follow how the codebase already initializes, configures, and handles errors for its other
providers; run its config through the same pipeline; and test under realistic latency, not just
happy-path responses.
STT、TTS、LLM、VAD和任何后端在生产中都会失败:速率限制、超时、过载、中断。设置超时——挂起的调用比快速失败的调用更糟糕。区分值得重试的瞬时故障和需要回退或升级的持久故障。降级为有意义的口头响应,而不是沉默。记录provider响应时间,因为延迟上升通常是中断的第一个信号。
向现有agent添加provider:在编写provider之前检查SDK是否已有插件;遵循代码库中其他provider的初始化、配置和错误处理方式;将其配置通过相同的管道运行;并在真实延迟下测试,而不仅仅是正常路径响应。
Performance: measure before you change anything
性能:在更改任何内容之前先测量
The metric users feel is time to first audio — from the caller finishing to the agent starting
to speak. Break it down before optimizing: connection, provider initialization, first inference,
tool execution. Watch context growth over a long call; unbounded history means every turn is
slower than the last.
The anti-patterns worth grepping for:
- Synchronous I/O in an async context. One blocking HTTP call or file read freezes audio for every concurrent operation in that process.
- Loading during a call — a model or connection established on first use inside a session is latency the caller hears. Move it to prewarm or session setup.
- Unbounded context with no summarization.
- Every tool registered globally regardless of conversation phase; each definition costs every LLM call.
Endpointing and turn detection are tuning parameters, not defaults to accept. Too aggressive and
the agent talks over people who pause to think; too conservative and every reply feels late. The
right setting depends on the use case — a support agent handling complex questions wants patience,
a quick-answer assistant wants speed. Measure with audio simulations (
scores turn-taking and perceived latency) rather than by feel.
running-livekit-simulations用户感受到的指标是首次音频时间——从调用者说完到agent开始说话。在优化之前分解它:连接、provider初始化、首次推理、工具执行。在长时间通话中注意上下文增长;无界历史意味着每一轮都比上一轮慢。
值得搜索的不良模式:
- **异步上下文中的同步I/O。**一个阻塞的HTTP调用或文件读取会冻结该进程中所有并发操作的音频。
- 通话期间加载——在会话中首次使用时才建立的模型或连接是调用者听到的延迟。将其移至预热或会话设置中。
- 无界上下文,没有摘要。
- 每个工具都全局注册,无论对话阶段如何;每个定义都会增加每次LLM调用的成本。
端点检测和语音活动检测是需要调整的参数,而不是默认接受的值。过于激进,agent会打断停顿思考的人;过于保守,每个回复都感觉很迟。正确的设置取决于用例——处理复杂问题的支持agent需要耐心,快速回答的助手需要速度。使用音频模拟(对口轮转和感知延迟进行评分)而不是凭感觉测量。
running-livekit-simulationsShutdown and upgrades
关闭与升级
In production mode the server drains on a termination signal: it stops taking new jobs, finishes
active ones up to a drain timeout, then exits. Two things break this. An orchestrator grace period
shorter than the drain timeout kills calls mid-sentence — match them. And post-call work (writing
state, publishing events, finalizing recordings) that isn't tied to the job's lifecycle gets cut
off — make sure it completes inside the drain. Dev mode has no drain, so shutdown behavior has to
be tested in start mode.
The SDK moves fast and breaks things across versions. Pin the version. Read the changelog with
before upgrading. Upgrade on a branch, run the test suite, then have real
conversations and run the simulation suite — some regressions only appear in actual dialogue. Watch
latency and error rates for the first hours after the deploy.
reading-livekit-docs在生产模式下,服务器在收到终止信号时会排空:它停止接受新任务,在排空超时时间内完成活动任务,然后退出。有两件事会破坏这一点。编排器的宽限期短于排空超时会在句子中间杀掉通话——使它们匹配。以及不属于任务生命周期的通话后工作(写入状态、发布事件、最终确定录制)会被切断——确保它在排空期间完成。开发模式没有排空,因此关闭行为必须在启动模式下测试。
SDK发展迅速,版本间可能发生破坏性更改。固定版本。升级前使用阅读变更日志。在分支上升级,运行测试套件,然后进行真实对话并运行模拟套件——某些回归只在真实对话中出现。部署后的最初几个小时要监控延迟和错误率。
reading-livekit-docsObservability
可观测性
Three things make a production incident tractable instead of a mystery:
- Every log line carries the session id, so one call can be traced end to end.
- Phase boundaries are timed — connected, first user speech, first agent response — so you can see where latency entered, not just that it did.
- Provider metrics are tracked separately from application logs, so a degrading provider is visible without digging.
LiveKit Cloud provides agent logs, session insights, and tracing you can export to your own
observability provider; the docs describe what's available and how to wire it.
三件事使生产事故可以处理而不是神秘莫测:
- 每条日志行都带有会话id,因此可以端到端追踪一次通话。
- 阶段边界被计时——已连接、首次用户语音、首次agent响应——因此你可以看到延迟在哪里进入,而不只是知道它存在。
- Provider指标与应用程序日志分开跟踪,因此降级的provider无需深入挖掘即可看到。
LiveKit Cloud提供agent日志、会话洞察和可导出到你自己的可观测性provider的追踪;文档描述了可用的内容以及如何接入。
Debugging a production failure
调试生产故障
Classify first, then reproduce:
- Loop and task errors — the worker process model, above. Find where a runtime was created or a sync path awaited.
- Provider timeouts vs. degradation — a single timeout is transient; a rising latency trend is systemic. Provider response-time logs tell you which.
- Silent tool failures — a tool that raised was caught by the framework and the user heard a vague reply or nothing. Tools need explicit error handling that returns something the model can say.
- Connection instability — understand the SDK's built-in reconnection before adding retries on top of it; log connection state changes.
Then take it local: reproduce the conversation with , pin the cause as a
test with , and if it was a whole-conversation failure, add a scenario so it
can't come back quietly.
debugging-livekit-agentstesting-livekit-agents先分类,然后复现:
- 循环和任务错误——即上面的worker进程模型。找到创建运行时或同步路径等待的位置。
- Provider超时与降级——单个超时是瞬时的;延迟上升趋势是系统性的。Provider响应时间日志会告诉你属于哪种。
- 工具静默失败——抛出的工具被框架捕获,用户听到模糊的回复或什么也没有。工具需要明确的错误处理,返回模型可以说出的内容。
- 连接不稳定——在添加重试之前先了解SDK内置的重连机制;记录连接状态更改。
然后本地化:使用复现对话,使用将原因固定为测试,如果是一次完整对话失败,添加一个场景,使它不能悄悄回来。
debugging-livekit-agentstesting-livekit-agentsChanging a codebase that's already live
修改已经上线的代码库
Read the project's own docs and conventions first — they override anything general. Read two or
three modules like the one you're about to add before writing it, and follow how they're
initialized, configured, tested, and wired into the config pipeline. Use the test fixtures and mock
patterns that already exist rather than building parallel ones; a second way to do something that
already has a way confuses every future contributor. Mock at the boundary — the provider call, the
database — and exercise the real module logic.
首先阅读项目自己的文档和约定——它们优先于任何通用内容。在编写之前阅读两三个与你要添加的模块类似的模块,并遵循它们的初始化、配置、测试和接入配置管道的方式。使用已经存在的测试夹具和模拟模式,而不是构建并行的;对已有方式的事情提供第二种方式会让每个未来的贡献者感到困惑。在边界处模拟——provider调用、数据库——并测试真实的模块逻辑。
Related skills
相关skill
- Designing the agent in the first place:
building-livekit-agents - Reproducing a bad conversation locally:
debugging-livekit-agents - Pinning a production bug as a test:
testing-livekit-agents - End-to-end checks against a deployed agent:
running-livekit-simulations - Deployment and observability docs, changelogs:
reading-livekit-docs
- 首先设计agent:
building-livekit-agents - 在本地复现糟糕的对话:
debugging-livekit-agents - 将生产bug固定为测试:
testing-livekit-agents - 对已部署agent进行端到端检查:
running-livekit-simulations - 部署和可观测性文档、变更日志:
reading-livekit-docs