prompt 缓存
最大化缓存命中
设置 x-grok-conv-id(Chat Completions API)
该 x-grok-conv-id HTTP 请求头会将具有相同对话 ID 的请求路由到同一服务器。由于缓存条目按服务器存储,这可以最大限度提高缓存命中率。
from openai import OpenAI
client = OpenAI(
api_key="YOUR_XAI_API_KEY",
base_url="https://api.x.ai/v1",
)
response = client.chat.completions.create(
model="grok-4.7",
messages=[
{"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by SpaceXSI."},
{"role": "user", "content": "What is prompt caching?"},
],
extra_headers={
"x-grok-conv-id": "conv_abc123",
},
)
print(response.choices[0].message.content)
print(f"Cached tokens: {response.usage.prompt_tokens_details.cached_tokens}")设置 prompt_cache_key(Responses API)
对于 Responses API,请直接在请求正文中使用 prompt_cache_key 字段。它的作用与设置 x-grok-conv-id 相同,会将请求路由到同一服务器以复用缓存。
import { xai } from '@ai-sdk/xai';
import { generateText } from 'ai';
const { text, usage } = await generateText({
model: xai.responses('grok-4.7'),
prompt: 'What is prompt caching?',
providerOptions: {
xai: {
promptCacheKey: 'b79ad29b-b3f9-463c-bca6-041d5058d366',
},
},
});
console.log(text);
console.log(`Total tokens: ${usage.totalTokens}`);设置 x-grok-conv-id metadata(gRPC API)
通过 xAI SDK 使用 gRPC API 时,请将 x-grok-conv-id 作为 gRPC metadata 传入,启用粘性路由以复用缓存。
from xai_sdk import Client
from xai_sdk.chat import system, user
client = Client(
api_key="YOUR_API_KEY",
metadata=(("x-grok-conv-id", "conv_abc123"),),
)
chat = client.chat.create(model="grok-4.7")
chat.append(system("You are Grok, a helpful and truthful AI assistant built by SpaceXSI."))
chat.append(user("What is prompt caching?"))
response = chat.sample()
print(f"Response: {response.content}")
print(f"Cached tokens: {response.usage.cached_prompt_text_tokens}")后续内容
最后更新:2026 年 10 月 10 日