Realtime

The Realtime page configures Realtime Dialog — a spoken, voice-to-voice conversation mode that runs alongside your agent's existing text chat rather than replacing it. Users can talk to the AI assistant out loud and hear its responses spoken back, and switch between voice and text at any point. Everything the agent already does — knowledge base lookups, web search, custom tools, and so on — keeps working the same way behind the scenes; only how the conversation is carried in and out changes.

Navigate to Persona → Realtime in the left sidebar.

Realtime voice consumes tokens differently from text chat (audio in, audio out, plus transcription), so enabling it affects the cost of running the agent.

Realtime Dialog

Enable Realtime Audio

Turns Realtime Dialog on or off for this agent. When enabled, end users get a way to start a spoken conversation from the chat interface — via the + menu next to the chat input — in addition to typing as usual.

When this is off, none of the other settings on this page apply and no voice entry point is shown to users.

Default Voice Display

Controls how a voice session is presented the moment it starts:

This is only the starting point — the end user can still switch to the other display from their own preferences once a call is underway.

Immersive View Welcome Text

Only shown when Enable Realtime Audio is on. Free text displayed in the immersive view while the canvas is still empty — before any chart, table, or source page has arrived. Leave blank to use the built-in default placeholder ("Speak to the assistant. Charts, tables and sources will appear here as pages you can browse.").

Canvas Display

Only shown when Enable Realtime Audio is on. During a voice call, some content is better shown on screen than spoken aloud — charts, tables, long lists, code, math, scripture, or anything that reads awkwardly out loud. These settings control when the AI assistant pushes content to the canvas instead of speaking it directly.

Automatic Canvas Content Detection

When enabled, the AI decides for itself, response by response, which parts to speak conversationally and which parts to show on the canvas instead — not limited to charts and tables. Leave this off if you'd rather rely only on the explicit list below.

Mandatory Canvas Content Types

A list of free-text content types (for example chart, table, arabic scripture, mathematical formula, cards) that must always be rendered on the canvas instead of spoken as plain text, whenever a response contains that kind of content. This applies regardless of whether Automatic Canvas Content Detection is on.

Click Add Content Type to add a row, and the delete icon on a row to remove it. This only changes where the content is rendered — the agent still speaks whatever explanation the answer calls for alongside it.

Voice & Transcription

Only shown when Enable Realtime Audio is on. Choose the voice the AI assistant speaks with, and the model used to transcribe what the user says into text.

Voice Persona

Selects the voice used for the assistant's spoken responses. Available options: alloy, ash, ballad, coral, echo, sage, shimmer, verse, marin, cedar.

Click the ▶ button next to the dropdown to hear a short English sample in the selected voice before saving.

Transcription Model

Selects the model used to transcribe the user's spoken input into text.

Voice Activity Detection (VAD)

Only shown when Enable Realtime Audio is on. VAD is what tells the system when the user is speaking and when they've finished, so it knows when it's the AI's turn to respond. Without good VAD, the assistant may cut users off mid-thought, or respond to background noise and filler sounds like "hmm" as if they were a real turn.

Use semantic VAD

When enabled (the default), turn detection is model-based rather than purely energy-based — it is better at telling real speech apart from filler sounds and background noise.

VAD eagerness

Only shown when Use semantic VAD is on. How eagerly the model ends the user's turn and jumps in: low, medium, high, or auto (default). Lower is calmer and less likely to interrupt.

Interruption sensitivity (VAD threshold)

Only shown when Use semantic VAD is off. A slider from 0 to 1 (default 0.8) setting the energy-based activation threshold. Higher makes the assistant harder to interrupt — background noise is ignored — while lower makes it easier to barge in, at the cost of more false interruptions.

Push-to-talk

Only shown when Enable Realtime Audio is on. Push-to-talk requires users to hold down a button while they speak — the microphone only streams audio while it's held. This guarantees background noise or other conversations can never be mistaken for a real turn, which is especially useful in noisy environments or when extra privacy is needed.

Enable push-to-talk

Shows a hold-to-talk button in the voice call UI. When on, the user can toggle at runtime between this and automatic (VAD-driven) listening; when off, the call always runs in automatic mode.

Default voice activation

Only shown when Enable push-to-talk is on. Which mode a voice session opens in:

Saving Changes

Click Save after adjusting any setting. Publish your agent to make the changes visible to end users.

Related pages

• Dialog Settings — language, formatting, and response suggestions for text chat
• Behavior — mission and personality, which also shape how the agent speaks during a voice call
• Advanced — AI model selection and conversation context memory