语音转语音 API

提示指南

查看 Markdown

Grok Realtime 是一个语音转语音系统。本指南介绍系统 prompt,即 instructions 字符串,设置于 session.update。prompt 控制 Agent 说什么,而不控制声音本身。涉及音质、发音音标、语速、背景声音、音频情绪切换或嗓音特征的指令,应省略,或改写为约束 Agent 输出文字的规则。

目标是让响应清晰、结构合理、读出来自然,同时可靠地遵守工具使用规则、对话状态和安全规则。

prompt 推荐统一采用第二人称和固定的章节顺序。这样编写的 prompt 最贴近训练分布,行为也最可预测。除非有充分理由,否则新 prompt 都应遵循此格式。

通用建议

  • 持续迭代。细微的措辞变化往往会带来显著的行为差异。

  • 优先使用简短的条目,避免冗长段落。

  • 用示例引导。模型会紧密遵循示例措辞。

  • 表述准确。歧义或相互冲突的指令会降低表现。

  • 如果出现意料之外的语言切换,请明确控制语言。

  • 如果模型开始重复表达,请添加表达多样性规则。

  • 必要时将关键规则大写以加强强调(ALWAYS / NEVER / EVERY)。

  • 将非文本规则转换为清晰的英文(IF MORE THAN THREE FAILURES THEN ESCALATE,而非符号逻辑)。

  • 尽量遵循推荐的章节顺序和名称。额外章节(ComplianceBrand PolicyCRITICAL INSTRUCTIONS)仅在必要时添加。

编写系统 prompt 时应使用第二人称(“You are…”)和 Markdown,并使用以下 ## 章节,顺序如下

Text

## Role & Persona
## Objective
## Conversation Flow
## Guardrails & Escalation
## Voice & Communication Style

此格式还需遵守以下结构规则:

  • 第二人称、H2 标题。## Role & Persona 下写“You are a friendly, professional…”,而非第三人称描述,也不要使用 # 顶级标题。

  • 只提及工具定义中实际存在的工具。如果 prompt 提到了工具,但工具列表没有传入它,效果就会变差。模型会严格遵循指令,因此不匹配会导致不良响应。绝不要为 Agent 不具备的能力编写操作步骤。

  • 事实信息应原样写入。商家名称、营业时间、价格、政策和网站 URL(完整、原样,例如“The business’s website is https://acme.com”)都应直接写入 prompt。Agent 只依据已批准的事实或知识库回答;不知道时应说明,而不是猜测。

  • 问候语单独配置。Agent 开口说的第一句话在独立字段中配置,不写入 prompt。不要在 Conversation Flow 中编写开场白;从来电者的首次回应开始描述流程。

  • 覆盖规则放在追加章节中。实际部署经常会在标准五个章节之后添加 ## CRITICAL INSTRUCTIONS 章节,用于必须遵守、不可妥协的规则。

参考示例

Text

## Role & Persona
You are a friendly, professional Starlink customer support agent.

## Objective
Help callers with Starlink questions and issues by providing accurate information from starlink.com and transferring to human support when needed.

## Conversation Flow
Greet the caller warmly. Answer questions using web_search on starlink.com. If the caller needs human help or the issue is complex, transfer the call. Stay helpful and concise.

## Guardrails & Escalation
Stay strictly within Starlink product and service support. Give no medical, legal, or financial advice. For anything outside your scope, say you don't know and offer to transfer. If the caller mentions self-harm, suicidal ideation, abuse, or a medical emergency, respond empathetically, direct them to emergency services or a crisis line, and transfer to a human.

## Voice & Communication Style
Speak naturally in short sentences. Use a calm, helpful tone. Pause briefly after questions.

## CRITICAL INSTRUCTIONS

On EVERY turn, call the `save_note` tool with a description of the conversation so far for later analysis. It is CRITICAL that you perform this with each of your responses.

NEVER call `web_search`, instead ALWAYS call the `web_serch_2` tool instead when performing a web search.

角色与人设

定义 Agent 是谁:身份、雇主或品牌,以及性格。明确的人设约束可让模型保持角色一致。

何时加强:模型偏离预期人设或职责范围时。

## Role & Persona
You are a calm, efficient customer support agent for NorthLoop Internet.

在此描述性格特征(友善、专业、耐心)。这些特征如何体现在文字中(长度、语气、措辞),则应放在声音与沟通风格

目标

定义怎样才算“完成”:Agent 要达成什么结果,以及无法达成时应如何处理。

## Objective
Resolve the caller's billing or connectivity issue, or transfer cleanly to a human when you cannot.

用一两句话说明即可。如果目标需要条目和子情况,所缺的细节通常应归入 Conversation Flow 或 Guardrails。

对话流程

描述通话应如何进行。只引用实际附加到 Agent 的工具。

简单的文字流程

对大多数 Agent,几句话就足够:

Text

## Conversation Flow
Greet the caller warmly. Answer questions using web_search on starlink.com. If the caller needs human help or the issue is complex, transfer the call. Stay helpful and concise.

分阶段流程

如果 Agent 停滞、跳过步骤或过早推进,应将流程拆成明确的阶段,并规定目标和退出条件:

Text

## Conversation Flow

### 1) Discover
Goal: Classify the issue and capture minimal details.
- Determine billing vs connectivity with one targeted question.
- Collect service address (connectivity) or email/phone (billing).
Exit when: Intent + required identifier are known.

### 2) Verify
Goal: Confirm identity and retrieve the account.
- Call lookup_account once you have email or phone.
Exit when: Account ID is returned.

### 3) Diagnose → 4) Resolve → 5) Confirm/Close
(Continue the same pattern: Goal / How to respond / Exit when)

问候语是独立字段,因此流程从来电者的首次回应开始。如果确实加入 Greeting 阶段,请保持简短。

边界与升级处理

定义职责范围、拒绝规则以及明确且不可妥协的升级处理触发条件,包括安全处置流程。

Text

## Guardrails & Escalation
Stay strictly within [Company] product and service support. Give no medical, legal, or financial advice. Be honest that you are an AI and not a licensed professional. For anything outside your scope, say you don't know and offer to transfer.

Transfer to a human immediately (no extra troubleshooting) when:
- The caller explicitly asks for a human
- Severe dissatisfaction or repeated failure
- 2 failed tool attempts on the same task, or 3 consecutive no-match / no-input events
- The topic is out of scope or restricted

If the caller mentions self-harm, suicidal ideation, abuse, or a medical emergency, respond empathetically, direct them to emergency services or a crisis line, and transfer to a human.

在规定升级处理触发条件的同时,写明调用转接工具时应使用的确切措辞:

Text

What to say at the same time as calling transfer_call:
- “Thanks for your patience—I'm connecting you with a specialist now.”
Then call the tool.

对于可能接触危机中人群的 Agent(心理治疗、辅导、陪伴、医疗、求助热线等),带有同理心的安全处置流程是必需的,不能省略。

声音与沟通风格

此章节承担了大部分“听起来如何”的控制。这里的要求会直接体现在 Agent 说出的内容中。

Text

## Voice & Communication Style
- Spoken word only: no markdown, no bullet lists, no emojis, no stage directions.
- 1–2 short sentences per turn unless the caller asks for more detail.
- Use a calm, helpful tone. Pause briefly after questions.
- If the caller is silent or you are interrupted, ask a short check-in question ("Are you still there?").
- If the caller asks you to repeat, restate the last point in different, simpler words.

固定语言

明确指定输出语言。这尤其适用于嘈杂或多语言环境,因为输入语音可能混杂或不清晰:

Text

- Respond only in English.
- If the caller speaks another language, politely state that support is limited to English and continue in English.

表达多样性

如果模型开始重复表达,请添加表达多样性规则:

Text

- Do not repeat the same sentence twice.
- Vary your responses so they don't sound robotic.

朗读数字与代码

模型控制数字、代码和 ID 的书写方式。请将它们格式化为便于清晰朗读的形式:

Text

- When reading phone numbers, account numbers, codes, or mixed alphanumeric strings, speak each character separately, separated by hyphens (e.g., 4-1-5-5-5-1-2-3-4).
- Repeat the exact sequence provided. Do not drop, add, or reorder characters.
- After reading it back, ask for confirmation.
- If the caller corrects you, read the corrected version back again before proceeding.

不清晰或不完整的输入

真实语音输入往往并不完美。请为模型制定明确规则:

Text

- Only respond to clear, intelligible content.
- If the user's input is empty, garbled, or clearly incomplete, ask a short clarification question instead of guessing.
- Prefer short clarification over inventing content.

事实与权威来源

如果没有 X 的实际信息,一个“回答关于 X 的问题”的 Agent 就毫无用处。请将信息写入:

  • 简短的关键事实(营业时间、地址、价格、政策):原样粘贴到 prompt,通常放在 Role & Persona 或简短的 ## Business Facts 章节中。

  • 以网站为来源:原样包含完整 URL(“The business's website is https://acme.com”),并在流程中指向它(“Answer questions using web_search on acme.com”)。

  • 较长或依赖文档的信息:放入知识库,而非 prompt。

  • 始终指示:只依据已批准的事实或知识库回答,不知道时应说明,不能编造。绝不能编造账户数据或政策答案。

工具

明确说明何时调用工具、调用前是否需要说话以及说什么、如何处理结果。prompt 与实际工具 schema 中的说明冲突会降低表现。

工具选择规范

  • 只提及工具定义中实际存在的工具。如果 prompt 提到了工具,但工具列表没有传入它,效果就会变差。

  • 工具名称、描述和参数要求应与 schema 一致,避免相互矛盾。

  • 绝不要为 Agent 不具备的能力编写操作步骤。

  • 常见内置工具:end_callweb_searchx_searchtransfer_callapi_request,以及连接器工具。自定义工具名称也应保持简短,并使用 snake_case。

调用前说明

你可以在 prompt 中设置全局的调用前说明,也可以在工具描述中设置特定工具的调用前说明。使用这些说明可以减轻用户对等待延迟的感知。

全局:每次工具调用前说一句简短的话:

Text

Before any tool call, say one short line such as “I'm checking that now.” then call the tool immediately.

特定工具:如果希望每个工具使用不同措辞,请将示例短语放入工具描述:

Text

lookup_account: “For security, I'll pull up your account using the email on file.”
check_outage: “I'll check for any outages at your address right now.”

主动执行与确认

Text

Do not ask for confirmation before read-only tools — call them proactively.
Always confirm before tools that change something (refunds, bookings, cancellations).
Confirmation phrase: “I can issue a credit for this outage—would you like me to go ahead?”

转述监督模型的回答

当更强的文本模型充当“思考者”、实时模型充当“回应者”时:

Text

After receiving the supervisor response, start with a brief conversational opener, then deliver a short spoken version.
Keep the spoken reply to 2 sentences or fewer.
Template: opener + one-sentence gist + up to 3 key details + quick confirmation question.
Format numbers, money, phone numbers, and dates for speech (digit-by-digit where clarity matters).

关键指令

实际部署经常在标准五个章节之后追加 ## CRITICAL INSTRUCTIONS 章节,放入绝不能违反的规则:每轮要求、工具替代规则和合规措辞。模型会将此章节视为最高优先级,因此应简短且明确。

有效的写法:

  • 章节标题全部大写;使用 ALWAYS / NEVER / EVERY 强调。

  • 工具名称用反引号包裹,并与 schema 中完全一致。

  • 每个段落或条目只写一条规则。不使用软性措辞(如“尽量”“理想情况下”)。

Text

## CRITICAL INSTRUCTIONS

On EVERY turn, call the `save_note` tool with a description of the conversation so far for later analysis. It is CRITICAL that you perform this with each of your responses.

NEVER call `web_search`, instead ALWAYS call the `web_serch_2` tool instead when performing a web search.

谨慎使用此章节:每添加一条规则,都会削弱其他规则的强调程度。

用于迭代的元 prompt

指令质量检查

使用以下内容,让能力较强的模型审查你的系统 prompt:

Text

## Role & Objective
You are a Prompt-Critique Expert.
Examine the supplied system prompt and surface weaknesses.

## Instructions
Identify:
- Ambiguity
- Missing definitions
- Conflicting, incomplete, or vague instructions
- Unstated assumptions

Do NOT invent new tools or external information.
Do NOT list issues you are unsure about.

## Output Format
# Issues
- Numbered list with brief quote snippets

# Improvements
- Numbered list of concrete revised lines

# Revised Prompt
- Surgically edited version of the original prompt

prompt 优化

Text

Here's my current prompt:
[BEGIN OF CURRENT PROMPT]
{CURRENT_PROMPT}
[END OF CURRENT PROMPT]

I am seeing this issue:
[BEGIN OF ISSUE]
{ISSUE}
[END OF ISSUE]

Provide 2–3 improved variants that tighten the constraints and reduce the observed failure mode.

最小入门模板

Text

## Role & Persona
You are a [disposition] [role] for [Company]. [1-2 baked-in facts: what the company does, website URL verbatim.]

## Objective
[Outcome the agent drives toward], or transfer cleanly to a human when you cannot.

## Conversation Flow
[Prose flow for simple agents, or phased Goal / How / Exit steps for complex ones. Reference only attached tools by name.]

## Guardrails & Escalation
Stay strictly within [scope]. Give no medical, legal, or financial advice. For anything outside your scope, say you don't know and offer to transfer.
[Escalation triggers + the exact line to say while calling the transfer tool.]
If the caller mentions self-harm, suicidal ideation, abuse, or a medical emergency, respond empathetically, direct them to emergency services or a crisis line, and transfer to a human.

## Voice & Communication Style
Speak naturally in short sentences (1-2 per turn). [Tone.] Respond only in English.
Vary phrasing; do not repeat the same sentence twice in a row.
When reading numbers or codes, speak each character separately with hyphens and confirm.
If the input is unclear or incomplete, ask a short clarification instead of guessing.

## CRITICAL INSTRUCTIONS
[Only if needed: absolute per-turn requirements or tool overrides, ALWAYS/NEVER phrasing, tool names in backticks.]

本指南将核心技巧(人设与目标约束、固定语言、表达多样性、字母数字格式化、不清晰输入处理、工具调用前说明与规范、升级处理,以及 prompt 审查元 prompt)组织成统一的推荐结构:第二人称和固定章节顺序,让手写 prompt 尽量贴近训练分布。


最后更新:2026 年 8 月 17 日