rw-integrate-audio

Compare original and translation side by side

🇺🇸

Original

English

🇨🇳

Translation

Chinese

Integrate Audio Generation

集成音频生成

PREREQUISITE: Run
+rw-check-compatibility
first. Run
+rw-fetch-api-reference
to load the latest API reference before integrating. Requires
+rw-setup-api-key
for API credentials. Requires
+rw-integrate-uploads
for local audio/video files.

Help users add Runway audio generation to their server-side code.

前提条件： 首先运行
+rw-check-compatibility
。在集成前运行
+rw-fetch-api-reference
加载最新的API参考文档。需要运行
+rw-setup-api-key
获取API凭证。处理本地音频/视频文件需要
+rw-integrate-uploads
。

帮助用户在服务端代码中添加Runway音频生成功能。

Available Models

可用模型

Model	Endpoint	Use Case	Cost
`eleven_multilingual_v2`	`POST /v1/text_to_speech`	Text to speech	1 credit/50 chars
`eleven_text_to_sound_v2`	`POST /v1/sound_effect`	Sound effect generation	1-2 credits
`eleven_voice_isolation`	`POST /v1/voice_isolation`	Isolate voice from audio	1 credit/6 sec
`eleven_voice_dubbing`	`POST /v1/voice_dubbing`	Dub audio to other languages	1 credit/2 sec
`eleven_multilingual_sts_v2`	`POST /v1/speech_to_speech`	Voice conversion	1 credit/3 sec

模型	接口	使用场景	成本
`eleven_multilingual_v2`	`POST /v1/text_to_speech`	文本转语音	1积分/50字符
`eleven_text_to_sound_v2`	`POST /v1/sound_effect`	音效生成	1-2积分
`eleven_voice_isolation`	`POST /v1/voice_isolation`	从音频中分离语音	1积分/6秒
`eleven_voice_dubbing`	`POST /v1/voice_dubbing`	音频多语言配音	1积分/2秒
`eleven_multilingual_sts_v2`	`POST /v1/speech_to_speech`	语音转换	1积分/3秒

Text-to-Speech

文本转语音（TTS）

Generate speech from text using the ElevenLabs multilingual model.

使用ElevenLabs多语言模型将文本转换为语音。

Node.js SDK

javascript

import RunwayML from '@runwayml/sdk';

const client = new RunwayML();

const task = await client.textToSpeech.create({
  model: 'eleven_multilingual_v2',
  promptText: 'Hello, welcome to our application!',
  voice: { type: 'runway-preset', presetId: 'Maya' }
}).waitForTaskOutput();

const audioUrl = task.output[0];

javascript

import RunwayML from '@runwayml/sdk';

const client = new RunwayML();

const task = await client.textToSpeech.create({
  model: 'eleven_multilingual_v2',
  promptText: 'Hello, welcome to our application!',
  voice: { type: 'runway-preset', presetId: 'Maya' }
}).waitForTaskOutput();

const audioUrl = task.output[0];

Python SDK

python

from runwayml import RunwayML

client = RunwayML()

task = client.text_to_speech.create(
    model='eleven_multilingual_v2',
    prompt_text='Hello, welcome to our application!',
    voice={ 'type': 'runway-preset', 'presetId': 'Maya' }
).wait_for_task_output()

audio_url = task.output[0]

python

from runwayml import RunwayML

client = RunwayML()

task = client.text_to_speech.create(
    model='eleven_multilingual_v2',
    prompt_text='Hello, welcome to our application!',
    voice={ 'type': 'runway-preset', 'presetId': 'Maya' }
).wait_for_task_output()

audio_url = task.output[0]

Sound Effects

音效生成

Generate sound effects from text descriptions.

javascript

const task = await client.soundEffect.create({
  model: 'eleven_text_to_sound_v2',
  promptText: 'Thunder rolling across a stormy sky'
}).waitForTaskOutput();

python

task = client.sound_effect.create(
    model='eleven_text_to_sound_v2',
    prompt_text='Thunder rolling across a stormy sky'
).wait_for_task_output()

根据文本描述生成音效。

javascript

const task = await client.soundEffect.create({
  model: 'eleven_text_to_sound_v2',
  promptText: 'Thunder rolling across a stormy sky'
}).waitForTaskOutput();

python

task = client.sound_effect.create(
    model='eleven_text_to_sound_v2',
    prompt_text='Thunder rolling across a stormy sky'
).wait_for_task_output()

Voice Isolation

语音分离

Extract voice from audio with background noise.

javascript

// If using a local file, upload first
const upload = await client.uploads.createEphemeral(
  fs.createReadStream('/path/to/noisy-audio.mp3')
);

const task = await client.voiceIsolation.create({
  model: 'eleven_voice_isolation',
  audioUri: upload.runwayUri
}).waitForTaskOutput();

从带有背景噪音的音频中提取语音。

javascript

// 如果使用本地文件，请先上传
const upload = await client.uploads.createEphemeral(
  fs.createReadStream('/path/to/noisy-audio.mp3')
);

const task = await client.voiceIsolation.create({
  model: 'eleven_voice_isolation',
  audioUri: upload.runwayUri
}).waitForTaskOutput();

Voice Dubbing

语音配音

Dub audio/video into other languages.

javascript

const task = await client.voiceDubbing.create({
  model: 'eleven_voice_dubbing',
  audioUri: 'https://example.com/speech.mp3',
  targetLang: 'es'  // Spanish
}).waitForTaskOutput();

将音频/视频配制成其他语言。

javascript

const task = await client.voiceDubbing.create({
  model: 'eleven_voice_dubbing',
  audioUri: 'https://example.com/speech.mp3',
  targetLang: 'es'  // 西班牙语
}).waitForTaskOutput();

Speech-to-Speech

语音转语音

Convert one voice to another.

javascript

const task = await client.speechToSpeech.create({
  model: 'eleven_multilingual_sts_v2',
  media: { type: 'audio', uri: 'https://example.com/original-speech.mp3' },
  voice: { type: 'runway-preset', presetId: 'Noah' }
}).waitForTaskOutput();

将一种语音转换为另一种语音。

javascript

const task = await client.speechToSpeech.create({
  model: 'eleven_multilingual_sts_v2',
  media: { type: 'audio', uri: 'https://example.com/original-speech.mp3' },
  voice: { type: 'runway-preset', presetId: 'Noah' }
}).waitForTaskOutput();

Integration Pattern

集成模式

Express.js — Text-to-Speech Endpoint

Express.js — 文本转语音接口

javascript

import RunwayML from '@runwayml/sdk';
import express from 'express';

const client = new RunwayML();
const app = express();
app.use(express.json());

app.post('/api/text-to-speech', async (req, res) => {
  try {
    const { text, voiceId } = req.body;

    const task = await client.textToSpeech.create({
      model: 'eleven_multilingual_v2',
      promptText: text,
      voice: { type: 'runway-preset', presetId: voiceId || 'Maya' }
    }).waitForTaskOutput();

    res.json({ audioUrl: task.output[0] });
  } catch (error) {
    console.error('TTS failed:', error);
    res.status(500).json({ error: error.message });
  }
});

javascript

import RunwayML from '@runwayml/sdk';
import express from 'express';

const client = new RunwayML();
const app = express();
app.use(express.json());

app.post('/api/text-to-speech', async (req, res) => {
  try {
    const { text, voiceId } = req.body;

    const task = await client.textToSpeech.create({
      model: 'eleven_multilingual_v2',
      promptText: text,
      voice: { type: 'runway-preset', presetId: voiceId || 'Maya' }
    }).waitForTaskOutput();

    res.json({ audioUrl: task.output[0] });
  } catch (error) {
    console.error('TTS failed:', error);
    res.status(500).json({ error: error.message });
  }
});

FastAPI — Sound Effects

FastAPI — 音效生成

python

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from runwayml import RunwayML

app = FastAPI()
client = RunwayML()

class SoundRequest(BaseModel):
    prompt: str

@app.post("/api/sound-effect")
async def generate_sound(req: SoundRequest):
    try:
        task = client.sound_effect.create(
            model='eleven_text_to_sound_v2',
            prompt_text=req.prompt
        ).wait_for_task_output()
        return {"audio_url": task.output[0]}
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

python

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from runwayml import RunwayML

app = FastAPI()
client = RunwayML()

class SoundRequest(BaseModel):
    prompt: str

@app.post("/api/sound-effect")
async def generate_sound(req: SoundRequest):
    try:
        task = client.sound_effect.create(
            model='eleven_text_to_sound_v2',
            prompt_text=req.prompt
        ).wait_for_task_output()
        return {"audio_url": task.output[0]}
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

Tips

提示

Output URLs expire in 24-48 hours. Download audio files to your own storage.
For local audio files (voice isolation, dubbing, speech-to-speech), upload via
```
+rw-integrate-uploads
```
first.
Voice IDs can be listed via the voices endpoint — see
```
+rw-api-reference
```
for details.
Text-to-speech cost scales with text length: 1 credit per 50 characters.

输出URL的有效期为24-48小时。请将音频文件下载到您自己的存储系统中。
对于本地音频文件（语音分离、配音、语音转语音），请先通过
```
+rw-integrate-uploads
```
上传。
语音ID可通过语音接口查看详情——请参考
```
+rw-api-reference
```
。
文本转语音的成本随文本长度递增：每50字符1积分。