Voice agents should not rebuild WebRTC

OpenAI published the architecture behind its low-latency voice serving — the plumbing under ChatGPT voice and the Realtime API, infrastructure that operates at the scale of products with more than 900 million weekly active users. (The newsletter headline implies all 900 million are voice users; the primary source describes the wider product infrastructure, so keep the claim where it belongs.) The design is genuinely elegant: a stateless relay layer that only forwards packets, a stateful transceiver service that owns the WebRTC session — ICE, DTLS, SRTP, lifecycle — with the transceiver's address encoded into the connection handshake so the first packet routes without a state lookup, and geographically distributed relays keeping the first hop short.

Enjoy the elegance, then notice the decision underneath it, because that decision is the transferable part: even OpenAI did not reinvent media transport. They kept standard WebRTC behavior at the client boundary and innovated in how packets route inside their own infrastructure. The protocol stack that survives hostile hotel Wi-Fi — jitter buffers, congestion control, NAT traversal, reconnection, device switching mid-call — is decades of accumulated scar tissue, and the largest AI company on earth chose to preserve it rather than replace it.

Now look at what voice-agent teams actually spend engineering cycles on. At work I reviewed a voice startup's sprint history this year: roughly a third of their engineering time had gone into connection recovery, network traversal edge cases, and a homegrown relay layer — transport problems, every one of them solved to a higher standard by infrastructure they could rent. Meanwhile the conversation itself — the thing users experience and the thing their customers were evaluating — handled interruptions badly, lost task state on reconnect, and had no evaluation harness at all. They were digging a private well behind a building with municipal plumbing, while the kitchen sent out undercooked food.

The build-versus-buy boundary in voice is unusually clean, and OpenAI's disclosure is evidence for where it sits. Below the line: media transport, relays, traversal, codec plumbing — mature, commoditized, undifferentiating; buy it, rent it, or use the platform's. Above the line: turn-taking and interruption behavior (the thing users call "it feels natural"), task state that survives a dropped call and resumes mid-transaction, domain tools wired to real systems, evaluation of conversation quality against your acceptance bar, and failure recovery when the model mishears or the tool errs mid-utterance. Every hour spent below the line is an hour your competitor spends above it.

One honest limit: OpenAI's architecture is evidence for the boundary, not an argument that every workload belongs on its Realtime API. Regulated deployments, on-prem constraints, or telephony-heavy stacks may justify owning more of the transport. But "we own our transport" should be a compliance conclusion, arrived at reluctantly — never an engineering ambition. The teams that treat transport as a craft project are spending their scarcest resource on the layer where excellence is invisible.

Steal this audit for your next sprint review: label every voice-related ticket from the last quarter either below the line (transport, connection, media) or above the line (conversation, state, tools, evals). If below-the-line exceeds a fifth of the total, you are funding infrastructure that exists, at the expense of product that doesn't. Rebalance before your users — or the next OpenAI blog post — do it for you.

The road is already built and better than yours will ever be — put the engineering into what you deliver on it.