xAI Voice API 提供一系列由 Grok 驱动的强大 voice 能力，并具备企业级可靠性和亚秒级延迟。

Speech to Speech

由 Grok 驱动、支持工具使用的实时 Speech to Speech 对话。

延迟: 亚秒级
Realtime: 起价 $0.05 / min
Endpoint: /v1/realtime

阅读文档在 Playground 中试用

Text to Speech

使用富有表现力的多语言 voice、speech tag 和电话 codec 生成语音。

Voices: 富有表现力、多语言
价格: $15.00 / 1M chars
Endpoint: /v1/tts

阅读文档在 Playground 中试用

Speech to Text

使用 batch 和 streaming 模式，将 25 种语言的音频转录为文本。

Batch: $0.10 / hour
Streaming: $0.20 / hour
Endpoint: /v1/stt

阅读文档在 Playground 中试用

Speech to Speech

通过 WebSocket 构建实时 Speech to Speech voice agent，并支持低延迟轮次交互和工具使用。对于客户端应用，请使用 Ephemeral Tokens进行安全连接，避免暴露 API key。

import asyncio
import json
import os
import websockets

async def voice_agent():
    async with websockets.connect(
        "wss://api.x.ai/v1/realtime?model=grok-voice-latest",
        additional_headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"}
    ) as ws:
        # Configure voice and enable tools
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {
                "voice": "eve",
                "instructions": "You are a helpful customer support agent.",
                "turn_detection": {"type": "server_vad"},
                "tools": [{"type": "web_search"}]
            }
        }))
        
        # Stream audio and receive responses
        async for message in ws:
            event = json.loads(message)
            if event["type"] == "response.output_audio.delta":
                # Play audio: base64.b64decode(event["delta"])
                pass

asyncio.run(voice_agent())

import WebSocket from "ws";

const ws = new WebSocket("wss://api.x.ai/v1/realtime?model=grok-voice-latest", {
  headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` },
});

ws.on("open", () => {
  // Configure voice and enable tools
  ws.send(JSON.stringify({
    type: "session.update",
    session: {
      voice: "eve",
      instructions: "You are a helpful customer support agent.",
      turn_detection: { type: "server_vad" },
      tools: [{ type: "web_search" }]
    }
  }));
});

ws.on("message", (data) => {
  const event = JSON.parse(data);
  if (event.type === "response.output_audio.delta") {
    // Play audio: Buffer.from(event.delta, "base64")
  }
});

Demo 应用： Web Agent · Twilio Phone Agent · WebRTC Agent · iOS Tester App

Text to Speech

使用丰富且富有表现力的 voice 将文本转换为语音。支持 inline speech tag（笑声、耳语、停顿），以及从高保真 MP3 到电话 μ-law 的多种输出格式。可使用 unary 请求或 WebSocket streaming。

curl -X POST https://api.x.ai/v1/tts \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Welcome to xAI. How can I help you today?",
    "voice_id": "eve",
    "language": "en"
  }' \
  --output welcome.mp3

import os
import requests

response = requests.post(
    "https://api.x.ai/v1/tts",
    headers={
        "Authorization": f"Bearer {os.environ['XAI_API_KEY']}",
        "Content-Type": "application/json",
    },
    json={
        "text": "Welcome to xAI. How can I help you today?",
        "voice_id": "eve",
        "language": "en",
    },
)

with open("welcome.mp3", "wb") as f:
    f.write(response.content)

import fs from "fs";

const response = await fetch("https://api.x.ai/v1/tts", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.XAI_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    text: "Welcome to xAI. How can I help you today?",
    voice_id: "eve",
    language: "en",
  }),
});

const buffer = Buffer.from(await response.arrayBuffer());
fs.writeFileSync("welcome.mp3", buffer);

实际应用示例： LiveKit · Pipecat

Speech to Text

通过一次调用转录音频文件，或通过 WebSocket streaming。支持 12 种音频格式、word-level timestamp、multichannel、speaker diarization、Smart Turn 轮次结束检测和 25 种语言。

curl -X POST https://api.x.ai/v1/stt \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -F file=@recording.mp3

import os
import requests

response = requests.post(
    "https://api.x.ai/v1/stt",
    headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},
    files={"file": ("recording.mp3", open("recording.mp3", "rb"), "audio/mpeg")},
)

print(response.json()["text"])

import fs from "fs";

const formData = new FormData();
formData.append("file", new Blob([fs.readFileSync("recording.mp3")]), "recording.mp3");

const response = await fetch("https://api.x.ai/v1/stt", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` },
  body: formData,
});

const result = await response.json();
console.log(result.text);

实际应用示例： Voximplant

快速开始：Custom Voices

从一段简短的参考音频 clip 克隆 voice，然后可以在任何支持内置 voice 的位置使用生成的 voice_id：

# 1. Create a custom voice from a reference audio clip (max 120s).
curl -X POST https://api.x.ai/v1/custom-voices \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -F "name=Friendly Narrator" \
  -F "language=en" \
  -F "file=@reference.wav;type=audio/wav"

# Response: { "voice_id": "nlbqfwie", ... }

# 2. Use the custom voice for TTS.
curl -X POST https://api.x.ai/v1/tts \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "text": "Hello! This is my custom voice.",
    "voice_id": "nlbqfwie",
    "language": "en"
  }' \
  --output custom.mp3

import os
import requests

# 1. Create a custom voice from a reference audio clip (max 120s).
with open("reference.wav", "rb") as f:
    create = requests.post(
        "https://api.x.ai/v1/custom-voices",
        headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},
        files={"file": ("reference.wav", f, "audio/wav")},
        data={"name": "Friendly Narrator", "language": "en"},
    )
voice_id = create.json()["voice_id"]

# 2. Use the custom voice for TTS.
speech = requests.post(
    "https://api.x.ai/v1/tts",
    headers={
        "Authorization": f"Bearer {os.environ['XAI_API_KEY']}",
        "Content-Type": "application/json",
    },
    json={
        "text": "Hello! This is my custom voice.",
        "voice_id": voice_id,
        "language": "en",
    },
)
with open("custom.mp3", "wb") as f:
    f.write(speech.content)

import fs from "fs";

// 1. Create a custom voice from a reference audio clip (max 120s).
const form = new FormData();
form.append("file", new Blob([fs.readFileSync("reference.wav")]), "reference.wav");
form.append("name", "Friendly Narrator");
form.append("language", "en");

const create = await fetch("https://api.x.ai/v1/custom-voices", {
  method: "POST",
  headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` },
  body: form,
});
const { voice_id } = await create.json();

// 2. Use the custom voice for TTS.
const speech = await fetch("https://api.x.ai/v1/tts", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.XAI_API_KEY}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    text: "Hello! This is my custom voice.",
    voice_id,
    language: "en",
  }),
});
fs.writeFileSync("custom.mp3", Buffer.from(await speech.arrayBuffer()));

自定义 voice_id 同样适用于 streaming TTS WebSocket 和 Speech to Speech realtime API。完整 API 请参阅 Custom Voices 指南。

Voices

使用 Speech to Speech API 或 Text to Speech 时，可以从完整的内置 voice 集合中进行选择。每种 voice 都有自己的个性与音色，请选择最适合应用的一种（eve 为默认 voice）：

Voice	音色与用例	Sample
`carina`	柔和、富有同理心且令人安心WellnessSupport
`zagan`	有力、戏剧化且极具辨识度CharactersNarration
`helix`	大胆、动感且充满冲劲CommentaryPodcast
`orion`	浑厚、电影感且富有共鸣NarrationAudiobooks
`luna`	温和、耐心且极具关怀感EducationAssistant
`iris`	友好、积极且自然迷人SalesSupport
`altair`	优雅、精致且具有高级感AdvertisingNarration
`zenith`	敏锐、专注且充满驱动力SalesAdvertising
`perseus`	坚定、自信且值得信赖AdvertisingNarration
`helios`	活泼、充满能量且用途广泛AssistantWellness
`lux`	沉稳、平静且含蓄睿智WellnessNarration
`kepler`	创新、前瞻且富有魅力AdvertisingPodcast
`rigel`	精准、专业且从容自信AssistantSupport
`cosmo`	明快、好奇且易于理解EducationPodcast
`celeste`	富有同理心、自信且令人安心SupportAssistant
`ursa`	友好、温暖且坚定可靠AssistantPodcast
`sirius`	机敏、聪慧且俏皮CommentaryCharacters
`lumen`	温暖、表达清晰且富有感染力EducationAdvertising
`castor`	富有魅力、亲切且随和SalesSupport
`naksh`	温暖、周到且睿智AssistantSupport
`atlas`	自信、有掌控力且令人安心SalesAssistant
`ara`	温暖友好
`eve`	充满活力且积极
`leo`	权威有力
`rex`	自信清晰
`sal`	流畅均衡