voice-agent

Compare original and translation side by side

🇺🇸

Original

English
🇨🇳

Translation

Chinese

Voice Agents

语音Agent

A voice agent answers a phone call, holds a conversation, calls your code when it needs to do something, and hands off to a person when it should. It is the same agent object as a messaging agent, with a
voice
block on it.
Zavu runs the whole pipeline: speech recognition, the agent's model, speech synthesis, and interruption handling. You supply the prompt and the skills.
One thing to know before anything else: voice is the channel where tools actually run. On plain text the model is not offered tools at all (see
ai-agent
). If your agent has to do something, voice and flows are where that works today.

语音Agent可接听电话、开展对话、在需要执行操作时调用你的代码,并在合适时机转接给人工。它与消息Agent属于同一对象,只是额外配置了
voice
块。
Zavu负责运行整个流程:speech recognition、Agent模型、speech synthesis以及中断处理。你只需提供提示词和技能。
首要须知:语音是工具实际运行的渠道。在纯文本场景下,模型根本不会调用工具(详见
ai-agent
)。如果你的Agent需要执行某些操作,目前只有语音和流程能实现这一点。

Senders vs accounts (one paragraph)

发送方与账号(简述)

A Sender is the API handle you pass as
Zavu-Sender
; accounts (a WhatsApp Business Account, a Facebook Page, a Telegram bot, a phone number) are the connections it routes — and what bills. Senders are free. Connecting an account in the dashboard auto-creates its sender; find it with
GET /v1/senders
and trust its
channels
array for what it can send. See the
channel-setup
skill for the full model.
Sender是你在请求头中传递的
Zavu-Sender
标识;账号(WhatsApp商业账号、Facebook主页、Telegram机器人、电话号码)是其路由的连接对象——也是计费依据。Sender是免费的。在控制台中连接账号会自动创建对应的Sender;可通过
GET /v1/senders
接口查询,并通过其
channels
数组确认支持的发送渠道。完整模型详见
channel-setup
技能。

Fastest path: take a factory agent

最快上手:使用预制Agent

Zavu ships working voice agents. Talking to one on zavu.dev and then owning its code is the same agent.
bash
npx zavudev agents catalog                     # what's available
npx zavudev agents pull kepler --dir kepler    # scaffold it locally
Zavu提供现成可用的语音Agent。在zavu.dev上体验的Agent与你获取代码后部署的是同一个。
bash
npx zavudev agents catalog                     # 查看可用预制Agent
npx zavudev agents pull kepler --dir kepler    # 本地生成脚手架

booking agents: add --calendar calcom for a working Cal.com client

预约类Agent:添加--calendar calcom参数可集成可用的Cal.com客户端

cd kepler npm install # types + local runs npx zavudev senders list # find your sender id npx zavudev fn secrets set SENDER_ID <senderId> npx zavudev deploy npx zavudev agents list # confirm what actually landed

An agent can answer on more than one number. Connect them by id:

```bash
npx zavudev agents senders connect --agent <agentId> --sender <senderId>
agents pull
writes a real TypeScript project:
index.ts
with the agent and its skills, a
tsconfig.json
, and a local stand-in for
@zavudev/functions
so the project typechecks and runs on your machine. Edit it freely — the file is yours, and
npx zavudev deploy
reconciles whatever it declares.

cd kepler npm install # 安装类型定义及本地运行依赖 npx zavudev senders list # 查询你的sender id npx zavudev fn secrets set SENDER_ID <senderId> npx zavudev deploy npx zavudev agents list # 确认部署结果

一个Agent可绑定多个号码,通过ID关联:

```bash
npx zavudev agents senders connect --agent <agentId> --sender <senderId>
agents pull
会生成一个完整的TypeScript项目:包含Agent及其技能的
index.ts
tsconfig.json
,以及
@zavudev/functions
的本地替代包,确保项目可进行类型检查并在本地运行。你可以自由编辑文件,
npx zavudev deploy
会根据文件声明同步部署内容。

Declaring a voice agent in code

以代码形式定义语音Agent

ts
import { defineAgent, defineTool } from "@zavudev/functions"

defineAgent({
  senderId: process.env.SENDER_ID!,
  name: "Kepler",
  provider: "zavu",
  model: "openai/gpt-4o-mini",
  channels: ["voice", "whatsapp"],
  voice: {
    enabled: true,
    model: "openai/gpt-4o",
    greeting: "Hi, I'm Kepler. What day would you like to book?",
    // Per-language greeting, used when the caller's language differs.
    greetings: { es: "Hola, soy Kepler. ¿Qué día querés reservar?" },
    // language: "es",           // omit to follow the caller's language
    interruptible: true,         // caller can talk over the agent
    maxCallDurationMinutes: 12,
    voiceSpeed: 1.15,
  },
  prompt: `# Personality
You book meetings. One idea per turn.
ts
import { defineAgent, defineTool } from "@zavudev/functions"

defineAgent({
  senderId: process.env.SENDER_ID!,
  name: "Kepler",
  provider: "zavu",
  model: "openai/gpt-4o-mini",
  channels: ["voice", "whatsapp"],
  voice: {
    enabled: true,
    model: "openai/gpt-4o",
    greeting: "Hi, I'm Kepler. What day would you like to book?",
    // 多语言问候语,当来电者语言不同时使用
    greetings: { es: "Hola, soy Kepler. ¿Qué día querés reservar?" },
    // language: "es",           // 留空则自动匹配来电者语言
    interruptible: true,         // 来电者可打断Agent发言
    maxCallDurationMinutes: 12,
    voiceSpeed: 1.15,
  },
  prompt: `# Personality
You book meetings. One idea per turn.

Environment

Environment

You are on a phone call. They can hear you, they cannot see a screen. Never read a URL or a code block aloud.`, })
undefined
You are on a phone call. They can hear you, they cannot see a screen. Never read a URL or a code block aloud.`, })
undefined

The
voice
block

voice
配置块

FieldWhat it does
enabled
Whether the agent answers calls. Required.
false
means the number is not answered and outbound calls are rejected.
greeting
First line spoken when the call connects. Omit to let the caller speak first.
greetings
Per-language greeting, keyed by language tag:
{ es: "Hola…" }
.
language
BCP-47 tag (
en
,
es
,
pt-BR
). Omit to follow the caller's language.
model
Model driving the conversation. Can differ from the text model.
ttsProvider
/
ttsVoiceId
Which synthesized voice speaks. Omit for a neutral default.
sttProvider
/
sttModel
Speech recognition. Omit for the default.
voiceSpeed
0.5–1.5. Only honoured by voices that support rate control.
interruptible
Barge-in.
true
(default) lets the caller cut the agent off.
maxCallDurationMinutes
Hard cap, 1–120. The call ends when reached.
maxIdleSeconds
Silence before the agent hangs up. 5–300.
voicemailAction
hangup
(default) or
leave_message
when an answering machine picks up.
voicemailMessage
Spoken when
voicemailAction
is
leave_message
. Falls back to
greeting
.
transferPhoneNumber
E.164 number for human handoff. Setting it gives the agent a transfer tool.
channels
and
voice
are separate:
channels: ["voice"]
routes calls to this agent,
voice.enabled
configures how it behaves on them. You need both. An agent routed for voice with no
voice
block will not answer properly.
字段功能说明
enabled
是否启用Agent接听电话。必填项。设为
false
时,号码不会接听来电,且会拒绝外呼请求。
greeting
电话接通时的第一句发言。留空则让来电者先说话。
greetings
多语言问候语,以语言标签为键:
{ es: "Hola…" }
language
BCP-47语言标签(如
en
es
pt-BR
)。留空则自动匹配来电者语言
model
驱动对话的模型,可与文本模型不同。
ttsProvider
/
ttsVoiceId
合成语音的提供商及语音ID。留空则使用默认中性语音。
sttProvider
/
sttModel
语音识别的提供商及模型。留空则使用默认配置。
voiceSpeed
语速范围0.5–1.5。仅支持语速控制的语音生效。
interruptible
插话功能。默认
true
,允许来电者打断Agent发言。
maxCallDurationMinutes
通话时长硬限制,范围1–120分钟。达到时长后自动结束通话。
maxIdleSeconds
静音超时挂断时间,范围5–300秒。
voicemailAction
遇到答录机时的操作:
hangup
(默认)或
leave_message
voicemailMessage
voicemailAction
设为
leave_message
时的留言内容。默认使用
greeting
transferPhoneNumber
人工转接的E.164格式号码。设置后Agent会获得转接工具。
channels
voice
是独立配置:
channels: ["voice"]
表示将电话路由至该Agent,
voice.enabled
配置其通话行为。两者缺一不可。若Agent被路由至语音渠道但未配置
voice
块,将无法正常接听来电。

Human handoff

人工转接

The headline claim is "answer, resolve, and hand off to a person." The handoff is
transferPhoneNumber
: set it, and the agent is given a transfer tool it can decide to use. Say when to use it in the prompt.
ts
voice: {
  enabled: true,
  transferPhoneNumber: "+14155551234",
},
prompt: `Transfer to a human when the caller asks for one, is upset,
or asks something about their account that you cannot answer.`,

核心功能是“接听、解决问题、转接人工”。通过
transferPhoneNumber
配置转接号码后,Agent会获得转接工具并自主决定是否使用。可在提示词中说明转接时机。
ts
voice: {
  enabled: true,
  transferPhoneNumber: "+14155551234",
},
prompt: `Transfer to a human when the caller asks for one, is upset,
or asks something about their account that you cannot answer.`,

Booking without writing a backend

无需编写后端即可实现预约功能

A voice agent that books meetings needs no code at all: add
check_availability
and
book_meeting
from the dashboard (agent, Tools, Library) and connect a Cal.com or Google calendar in the same place. Zavu hosts both skills.
Use this instead of scaffolding a booking function when the only thing the agent has to do is read a calendar and put something on it. Reach for
npx zavudev agents pull kepler --calendar calcom
when the booking logic is yours: qualifying first, routing to different hosts, writing to your own system too.
实现预约功能的语音Agent无需编写任何代码:在控制台(Agent → ToolsLibrary)中添加
check_availability
book_meeting
工具,并在同一位置连接Cal.com或Google日历即可。Zavu会托管这些技能。
如果Agent仅需读取日历并创建预约,可直接使用此方式。若需要自定义预约逻辑(如资格预审、路由至不同负责人、同步至自有系统等),则使用
npx zavudev agents pull kepler --calendar calcom
生成脚手架。

Skills the agent can call

Agent可调用的技能

Tools run on voice. Declare them next to the agent; they execute in your function, in Zavu Cloud.
ts
defineTool({
  name: "check_availability",
  description: "Find open meeting slots. Call before offering a time.",
  parameters: {
    type: "object",
    properties: {
      preferred_time: { type: "string", description: "What the caller asked for" },
    },
    required: ["preferred_time"],
  },
  handler: async (args, ctx) => {
    const res = await fetch(`https://api.example.com/slots?q=${args.preferred_time}`)
    if (!res.ok) return { ok: false, reason: "lookup_failed", slots: [] }
    return { ok: true, slots: (await res.json()).slots }
  },
})
Run one locally without deploying:
bash
npx zavudev fn invoke --tool check_availability --args '{"preferred_time":"tomorrow 3pm"}'
Never return a fake success. A handler that answers
{ booked: true, confirmationCode: "ABC-123" }
when it did nothing makes the agent tell a real caller their meeting exists. Return
{ ok: false, reason: "not_configured" }
and let the prompt handle it — the agent can say it cannot book right now, which is recoverable. A false confirmation is not.

工具在语音渠道运行。可在Agent旁定义工具,它们会在你的函数中执行,部署于Zavu Cloud。
ts
defineTool({
  name: "check_availability",
  description: "Find open meeting slots. Call before offering a time.",
  parameters: {
    type: "object",
    properties: {
      preferred_time: { type: "string", description: "What the caller asked for" },
    },
    required: ["preferred_time"],
  },
  handler: async (args, ctx) => {
    const res = await fetch(`https://api.example.com/slots?q=${args.preferred_time}`)
    if (!res.ok) return { ok: false, reason: "lookup_failed", slots: [] }
    return { ok: true, slots: (await res.json()).slots }
  },
})
无需部署即可本地测试工具:
bash
npx zavudev fn invoke --tool check_availability --args '{"preferred_time":"tomorrow 3pm"}'
切勿返回虚假成功结果。若处理函数未执行任何操作却返回
{ booked: true, confirmationCode: "ABC-123" }
,Agent会告知真实来电者预约已成功。应返回
{ ok: false, reason: "not_configured" }
,并让提示词处理该情况——Agent可告知用户当前无法预约,这是可恢复的;而虚假确认则无法挽回。

Placing and inspecting calls

发起与查看通话

bash
npx zavudev calls create --to +14155551234        # outbound. COSTS MONEY.
npx zavudev calls list --status completed
npx zavudev calls get <callId>                    # includes the transcript
npx zavudev calls hangup <callId>
calls get
prints the conversation turn by turn, including tool calls. It is the only record of what the agent actually said, and the first place to look when a call went wrong.
From the SDK (
@zavudev/sdk
0.56.0+; earlier versions have no
calls
resource — call the REST endpoints directly):
ts
const { call } = await zavu.calls.create({
  to: "+14155551234",
  greeting: "Hi, this is Acme calling about your appointment.",
  language: "es-ES",            // or "auto" to follow the caller
  metadata: { campaign: "reminders" },
})

bash
npx zavudev calls create --to +14155551234        # 外呼电话,会产生费用
npx zavudev calls list --status completed
npx zavudev calls get <callId>                    # 包含通话记录
npx zavudev calls hangup <callId>
calls get
会逐轮打印对话内容,包括工具调用记录。这是Agent实际发言内容的唯一记录,也是排查通话问题的首要依据。
通过SDK(
@zavudev/sdk
0.56.0及以上版本;早期版本无
calls
资源——需直接调用REST接口):
ts
const { call } = await zavu.calls.create({
  to: "+14155551234",
  greeting: "Hi, this is Acme calling about your appointment.",
  language: "es-ES",            // 设为"auto"则自动匹配来电者语言
  metadata: { campaign: "reminders" },
})

Testing before you spend money

付费前测试

There is no way to hear the agent without a phone or a browser. But you can test everything upstream of the audio, for free:
bash
undefined
必须通过电话或浏览器才能听到Agent的语音,但你可以免费测试音频之前的所有环节:
bash
undefined

The agent's brain: prompt, model, knowledge base. Nothing is delivered.

测试Agent的核心逻辑:提示词、模型、知识库。不会发起真实通话。

npx zavudev agents test --agent <agentId> --message "do you have anything Tuesday?"
npx zavudev agents test --agent <agentId> --message "do you have anything Tuesday?"

A skill's handler, locally.

本地测试技能处理函数

npx zavudev fn invoke --tool check_availability --args '{"preferred_time":"Tuesday"}'

`agents test` runs the **text** path, so it will not exercise tool calling even
though voice would — it warns you when that applies. Use it for the prompt, and
`fn invoke` for the handlers.

To actually hear it, you need a phone number and a real call. There is no way
to listen to the agent for free: `agents test` is text-only, and it says so on
every run. Everything before the call is verifiable (config, prompt, handlers);
the audio itself is not. Budget for that before promising a delivery date.

Inbound requires owning a number:

```bash
npx zavudev phone-numbers search --country US
Do not decide from the
capabilities
column alone. It is carrier-reported and has been wrong in both directions: numbers that place and answer calls have listed
["sms"]
, and numbers listing
voice
have failed to complete a call. Every number sold through Zavu is provisioned for calls when you buy it. The check that actually answers the question is the sender's own channel list:
bash
npx zavudev senders get <senderId>    # channels includes "voice" once it can call

npx zavudev fn invoke --tool check_availability --args '{"preferred_time":"Tuesday"}'

`agents test`运行的是**文本**流程,因此即使语音渠道会调用工具,该测试也不会触发工具调用——测试时会对此发出警告。可使用该命令测试提示词,使用`fn invoke`测试处理函数。

若要实际听到Agent语音,你需要一个电话号码和真实通话。目前无法免费收听Agent语音:`agents test`仅支持文本测试,每次运行都会提示这一点。通话前的所有环节(配置、提示词、处理函数)都可验证;但音频本身无法免费测试。在承诺交付日期前需考虑这一成本。

接听来电需要拥有电话号码:

```bash
npx zavudev phone-numbers search --country US
切勿仅根据
capabilities
列判断。该信息由运营商提供,可能存在误差:可正常接听和拨打电话的号码可能仅标注
["sms"]
,标注
voice
的号码可能无法完成通话。通过Zavu购买的所有号码在购买时都会配置通话功能。真正能确认的方式是查看Sender自身的渠道列表:
bash
npx zavudev senders get <senderId>    # 当支持通话时,channels会包含"voice"

Requirements and cost

要求与费用

  • The sender's agent needs
    voice.enabled: true
    , or
    POST /v1/calls
    returns
    400 "The sender's agent does not have voice enabled"
    .
  • Not available with test-mode API keys.
  • Inbound needs a phone number with the
    voice
    capability.
  • Billed per connected minute plus telephony, from your prepaid balance. A short estimate is reserved when the call starts; you are charged the real duration when it ends.
    402
    means insufficient balance.
  • Sender对应的Agent需设置
    voice.enabled: true
    ,否则
    POST /v1/calls
    接口会返回
    400 "The sender's agent does not have voice enabled"
  • 测试模式API密钥无法使用此功能
  • 接听来电需要具备
    voice
    能力的电话号码。
  • 费用按通话连接时长加电信费计算,从你的预付费余额中扣除。通话开始时会预留一笔预估费用;通话结束时按实际时长扣费。返回
    402
    表示余额不足。

Webhook events

Webhook事件

Every voice event carries
callId
,
direction
,
from
,
to
,
status
,
durationSeconds
,
endReason
and
transcriptAvailable
.
EventWhen
call.initiated
Outbound dialing, or inbound received.
status: ringing
call.answered
Connected, the agent is live.
status: in_progress
call.completed
Ended after a conversation
call.failed
Busy, no answer, canceled, or an error
bash
npx zavudev senders update <senderId> \
  --webhook-url https://api.example.com/hooks/zavu \
  --webhook-events call.completed,call.failed

每个语音事件都会携带
callId
direction
from
to
status
durationSeconds
endReason
transcriptAvailable
字段。
事件触发时机
call.initiated
发起外呼或收到来电时,
status: ringing
call.answered
通话接通,Agent开始工作时,
status: in_progress
call.completed
对话结束后
call.failed
占线、无人接听、取消或发生错误时
bash
npx zavudev senders update <senderId> \
  --webhook-url https://api.example.com/hooks/zavu \
  --webhook-events call.completed,call.failed

Getting it wrong

常见问题排查

  • Agent answers but says nothing useful. Check the knowledge base. A prompt that says "only state what retrieval returned" with no documents attached will invent answers instead of refusing. Verify with
    agents test
    — it reports how many knowledge chunks were used.
  • The number rings and the agent does not pick up. Check the sender first:
    bash
    npx zavudev senders get <senderId>    # channels must be non-empty
    An empty
    channels
    array means the sender is not wired to anything and cannot send or receive on any channel, voice included. That is the cause people miss, because the agent,
    voice.enabled
    , and the number all look correct while it is true. Attach a number your project owns and turn voice on:
    bash
    npx zavudev phone-numbers update <phoneNumberId> --sender <senderId>
    npx zavudev senders update <senderId> --enable-voice
    Only once
    channels
    includes
    voice
    is it worth checking
    voice.enabled
    and that the agent itself is enabled.
  • A field you set has no effect.
    voiceSpeed
    is only honoured by voices that support rate control.
    greetings
    needs a language tag that matches what the caller actually speaks.
  • Deploy said it synced but nothing changed. Read the lines above the ✓, and do not trust the exit code.
    npx zavudev deploy
    prints its warnings before the success line, but it still exits 0 for cases that leave your agent unreachable — two agents on one sender, for instance, where only one answers. A green checkmark means the deploy ran, not that your agent will be reached. If you gate CI on this, gate it on the warning lines, not on the exit status.
  • Agent接听但内容无意义:检查知识库。若提示词要求“仅返回检索结果”但未关联任何文档,Agent会编造答案而非拒绝回答。可使用
    agents test
    验证——该命令会报告使用了多少知识库片段。
  • 号码响铃但Agent未接听:首先检查Sender:
    bash
    npx zavudev senders get <senderId>    # channels必须非空
    channels
    数组为空,表示Sender未绑定任何渠道,无法发送或接收任何类型的消息,包括语音。这是容易被忽略的原因,因为此时Agent的
    voice.enabled
    配置和号码看起来都正常。需绑定项目所属的号码并启用语音功能:
    bash
    npx zavudev phone-numbers update <phoneNumberId> --sender <senderId>
    npx zavudev senders update <senderId> --enable-voice
    只有当
    channels
    包含
    voice
    后,才需要检查
    voice.enabled
    和Agent是否启用。
  • 设置的字段未生效
    voiceSpeed
    仅对支持语速控制的语音生效。
    greetings
    需要与来电者实际使用的语言标签匹配。
  • 部署提示同步成功但无变化:查看成功标记上方的内容,不要仅信任退出码。
    npx zavudev deploy
    会在成功行之前打印警告信息,但即使存在导致Agent无法访问的情况(例如一个Sender绑定两个Agent,仅其中一个能接听),它仍会返回退出码0。绿色对勾仅表示部署完成,不代表Agent可正常访问。若在CI中校验,需基于警告信息而非退出状态。

Prompting for voice

语音场景提示词撰写要点

Voice punishes prose. What matters, in order:
  1. One idea per turn. Two sentences maximum. They cannot skim.
  2. Lead with the answer, then offer detail.
  3. Never read URLs, code, or JSON aloud. Say what it does and where it is.
  4. Read numbers and codes one character at a time.
  5. Expect the transcription to mangle names, including your own product's. Tell the agent what it might sound like and to assume it means you.
  6. Say what is out of scope and what to do in one sentence when it comes up.
语音场景对冗长文本零容忍。优先级从高到低为:
  1. 每次仅传递一个信息点:最多两句话。来电者无法快速浏览内容。
  2. 先给出答案,再补充细节
  3. 切勿朗读URL、代码或JSON:说明其用途和位置即可。
  4. 数字和代码需逐字符朗读
  5. 语音识别可能会混淆名称,包括你的产品名称。需告知Agent可能的发音变体,并默认指代你的产品。
  6. 明确说明超出范围的内容及处理方式,用一句话表述。