OpenAI Realtime vs Cascaded Voice Agents: What the Numbers Actually Show (and Where Hybrid Wins)
This is a follow-up to Part 6, which showed how to run Hindi voice AI in production with a hybrid stack. The question underneath that decision is bigger than Hindi: should a new voice agent in 2026 be built on OpenAI Realtime (speech-to-speech) or on a cascaded STT + LLM + TTS pipeline? This post pulls the comparison apart with real public figures and names the hybrid lane where most Indian voice-AI products actually land.
What the two architectures actually are
A cascaded voice agent is three separate models chained together: a speech-to-text (STT) model, a text LLM, and a text-to-speech (TTS) model. A thin orchestration layer handles VAD (voice activity detection), barge-in and streaming between stages.
A speech-to-speech (S2S) agent — OpenAI Realtime, Gemini Live, ElevenLabs Conversational — folds all three into one model that reads and writes audio directly. There is no explicit text layer in the middle.
A hybrid takes the best of both. The usual shape is: Realtime for ASR, LLM and semantic VAD with `modalities=["text"]`, and a dedicated TTS swapped in for the voice. Part 6 describes this stack in detail with Veena as the TTS.
The honest headline: nobody publishes a hard split
There is no credible survey that says "X% of production voice agents use Realtime and Y% use cascaded". a16z, Deepgram, LiveKit, Inworld, Gradium, Soniox and RTCLeague all agree on the direction, but none publish a measured percentage. Anyone who quotes a hard split with confidence is estimating.
So this post splits into two parts: the real cited statistics, then a defensible estimate built on top of them.
Real cited statistics
a16z Voice AI 2025 Update
- Voice-agent companies made up 22 percent of the most recent YC class.
- Of those, 69 percent are B2B, 18 percent healthcare, 13 percent consumer.
- Within B2B: fintech 16.9 percent, customer-support operations 12.4 percent.
- a16z does not break down the sample by architecture.
Deepgram State of Voice AI 2025 (400+ enterprises, 83 percent from companies with over $100M revenue)
- 97 percent already use some voice technology (IVR, STT, TTS, agents).
- 80 percent still rely on traditional voice-agent systems (IVR plus legacy).
- Only 21 percent are "very satisfied" with their current voice tech.
- 15 percent are actively developing AI voice agents today.
- 98 percent of those plan to go to production within 12 months.
- The report frames "speech-to-speech is the future" — Deepgram is building its own S2S model, so this is a vendor framing, not a neutral measurement.
Framework and timing signals
- LiveKit Agents (cascaded framework): about 6k+ GitHub stars.
- Pipecat (cascaded framework): about 4k+ stars.
- Vocode: about 2.5k stars, mindshare declining.
- OpenAI Realtime API went GA in August 2025 — only about 14 months in production as of late 2026.
- Public OpenAI Realtime case studies: roughly 15 to 20 named logos (Zendesk, LiveKit, T-Mobile, Lemonade and others).
- Y Combinator S24/W25/S25 voice-AI startups: 40 to 60 companies. Most that disclose their stack use cascaded, with Realtime added later.
What analysts and vendors actually say
| Source | Position |
|---|---|
| LiveKit (framework vendor, supports both) | "Both approaches are in active production use. Neither is universally right. Most pipeline deployments will be slower than a realtime model." |
| Gradium / Kyutai (Moshi S2S lab) | "Cascade remains the dominant production architecture in 2026, largely because tool use, function calling and structured output are solved problems in the text layer." |
| Inworld AI (S2S vendor) | Even Inworld describes cascaded as "the right production architecture" today and frames its own S2S as "a configurable cascaded pipeline over a single stateful connection". |
| Soniox | "The pipeline gives you control, inspection and modularity at some cost in latency and expressiveness. S2S gives you latency and expressiveness at a real cost in control and inspectability." |
| RTCLeague | "For most enterprise and commercial voice-agent deployments in 2025, the cascaded streaming pipeline remains the correct architectural default. Real-time S2S is not a replacement; it is a specialised tool." |
Defensible estimate for production voice agents (late 2025 / 2026)
Treating the sources above as evidence, and marking the estimate honestly, the split for production deployments looks like this. Treat the percentages as roughly ±15 points.
| Stack | Rough share | Who runs it |
|---|---|---|
| Cascaded (STT + LLM + TTS, orchestrated via LiveKit Agents, Pipecat, Vocode or custom) | 70–80% | Most production call-centre, outbound dialer and Indian voice-agent startups (Sarvam, Smallest, Skit, Gnani, CoRover, Bland.ai, Retell AI, Vapi, Synthflow). Also most healthcare, legal and finance apps that need custom TTS, strict compliance or a specific language. |
| Pure Realtime / S2S (OpenAI, Gemini Live, ElevenLabs Conversational) | 10–20% | Newer startups built after October 2024, prototypes, consumer apps, demos, English-only use cases. OpenAI Realtime dominates this slice. |
| Hybrid (Realtime for ASR + LLM, custom TTS override) | 5–15% and growing | Teams that want Realtime's low latency and VAD but need better voice quality in a specific language. Vapi, Retell and several Indian startups are here. This is also where our Hindi stack lives. |
In India specifically, cascaded share is almost certainly higher — probably 80 to 90 percent — because of the Hindi and Indic TTS quality gap in Realtime, PSTN-only deployment, and cost sensitivity.
Why cascaded still dominates
Five structural reasons, consistently cited across the sources above:
- Cost. S2S is around 10× cascaded at scale because audio context accumulates every turn and the model re-charges on the full prior-audio history each turn. For a 10k-calls-a-day dialer this is the biggest line item.
- Telephony. PSTN is 8 kHz narrowband. S2S's wideband acoustic advantage largely disappears over phone networks, which is where production voice AI mostly runs.
- Multilingual coverage. Realtime APIs are English-first. Cascaded lets you pick a Hindi-strong STT (Sarvam Saarika) and a Hindi-strong TTS (Veena) independently.
- Observability. Compliance, moderation, QA sampling and the training-data flywheel all need text at every stage boundary. Cascaded gives this for free. S2S is opaque.
- LLM swap cadence. Frontier LLMs ship every 2 to 4 months. S2S models retrain every 6 to 12 months. Cascaded lets you ride the fast curve by swapping the LLM alone.
Where OpenAI Realtime does win
Realtime is not a worse technology. It is a better fit for a different set of jobs:
- Consumer-facing short-dialog apps in English: companion apps, voice shopping, Duolingo-style education, voice gaming.
- Prototypes and demos where shipping in a weekend beats quality tuning.
- Teams without deep ML or infrastructure headcount — Realtime hides most of the plumbing.
- Expressive turn-taking and emotional tone, where S2S preserves acoustic context that a text-middle pipeline would throw away.
- English call flows where sub-800 ms turn latency and vendor-managed VAD are worth the premium.
Cost comparison
| Cost line | OpenAI Realtime (Opus-quality voice) | Cascaded (Deepgram + GPT-4o-mini + ElevenLabs Flash) |
|---|---|---|
| Approximate per-minute cost | $0.06 – $0.10 | $0.03 – $0.05 |
| Scaling behaviour | Audio-token context accumulates — long calls get expensive fast | Flat per-minute across stages |
| Voice selection | ~10 preset voices | Branded or cloned voices via the TTS vendor |
| LLM swap | Locked to the Realtime model | Any chat-completions LLM |
| Transcript | Needs separate speech-to-text pass | Produced natively at every stage |
Latency comparison for a typical turn
End-to-end turn latency is measured from the user's last audible word to the first byte of agent audio they hear. Thresholds: 800 ms feels natural, 1,200 ms feels sluggish, 1,800 ms feels broken. See Part 6 for the full breakdown.
| Stack | Typical end-to-end turn | Notes |
|---|---|---|
| OpenAI Realtime (US-East endpoint, Indian caller) | 800–1,200 ms | Adds 150–220 ms per round-trip for Indian traffic. |
| Hybrid: Realtime (ASR+LLM) + Veena TTS in Mumbai | 900–1,200 ms | Fixes Hindi voice quality. Still pays US RTT on ASR+LLM. |
| Cascaded in Mumbai (Sarvam STT + GPT-4o-mini + Veena) | 700–950 ms | Everything on the audio path is India-hosted. |
| Naïve cascaded (no streaming, sentence-level TTS buffering) | 2–4 s | Common failure mode — feels broken on calls. |
The hybrid lane in one sentence
The explicit recommendation from RTCLeague, LiveKit and the Indic voice-AI startups is the same: use Realtime where it is strong (ASR, LLM, semantic VAD) and swap in a dedicated TTS where it is weak (Hindi, Hinglish, brand voices). That is the ~5 to 15 percent hybrid slice. In India it is probably closer to 20 to 30 percent among newer builds.
How to pick for a new project
- English-only, consumer, short dialogs, no PSTN: start with Realtime. Rebuild only if the economics force it.
- Hindi or Indic PSTN calls: start hybrid (Realtime for ASR + LLM + VAD, dedicated TTS). This is the fastest path to good voice quality.
- High-volume outbound dialer with tight unit economics: go cascaded from day one.
- Regulated enterprise (healthcare, finance, legal): go cascaded for the native text transcripts and vendor independence.
- Need branded or cloned voice: cascaded (or hybrid with a TTS vendor that supports cloning).
The short version
Realtime is the fastest way to ship an English voice demo. Cascaded is still the dominant production architecture, and the gap gets wider the more cost-sensitive, multilingual, regulated or observable the deployment is. For Indian voice AI specifically, hybrid is the sweet spot: Realtime for the parts it does well, open-weight or Indic-tuned models for the parts it does not. That is the same conclusion Part 6 arrives at from a latency-first angle.
Is OpenAI Realtime cheaper than a cascaded stack?
No. For high-volume production, Realtime is roughly 2–3× the per-minute cost of a well-tuned cascaded stack (Deepgram or Sarvam STT + GPT-4o-mini + ElevenLabs Flash or Veena). S2S cost grows with call length because audio context accumulates every turn.
What percentage of production voice agents use OpenAI Realtime?
No vendor publishes a measured figure. The defensible estimate from a16z, Deepgram State of Voice AI 2025, LiveKit, Gradium, Inworld, Soniox and RTCLeague is roughly 10–20 percent of production deployments on pure Realtime (or Gemini Live or ElevenLabs Conversational), 70–80 percent on cascaded, and 5–15 percent on hybrid stacks.
Why is cascaded still winning in India?
Three reasons: Hindi and Indic TTS in Realtime is weaker than open models like Veena; Indian voice AI runs almost entirely over 8 kHz PSTN, where S2S's wideband advantage disappears; and the Indian market is cost-sensitive, so the 2–3× cost multiplier of S2S matters.
When does OpenAI Realtime make clear sense?
English consumer apps, prototypes, short dialogs, and products where sub-800 ms latency and vendor-managed VAD are more valuable than per-minute cost or multilingual quality.
What is a hybrid voice agent?
A pipeline that uses OpenAI Realtime (or similar) for ASR, LLM and VAD with text-only modalities, then overrides TTS with a dedicated model such as Veena, ElevenLabs or Cartesia. It keeps Realtime's low-latency reasoning and VAD while fixing the voice-quality gap in languages like Hindi. Part 6 covers the production shape.
Comments
Loading comments…