4 posts tagged with "voice-agents"

Pick a voice stack and you are really picking two things: who owns the audio transport, and what you can see when a turn goes wrong. The vendor list barely matters — LiveKit Agents and Pipecat both put the same speech-to-text, language model and text-to-speech providers behind the same three seams, and both let you swap any of them in a line.

LangSmith shipped tracing for both in July, alongside OpenAI Realtime and Gemini Live. That is a useful forcing function for this comparison, because a trace shows you where a framework's abstraction actually sits — and the two answers are genuinely different.

"Add Twilio" sounds like one integration. It is three, and they disagree about the most basic question in a voice agent: who holds the audio. LiveKit takes the call as a SIP trunk and turns the caller into a participant in a room. Pipecat takes it as a WebSocket carrying raw audio frames. And Twilio's own ConversationRelay keeps the audio entirely, and hands your server text.

The previous post compared LiveKit and Pipecat on the seam where a voice agent breaks: a caller talking over the bot. Put a phone line in front of either one and that seam moves, because now there is a third system buffering audio that neither framework controls.

In July, LangSmith launched voice tracing for four frameworks: Pipecat, LiveKit, OpenAI Realtime and Gemini Live. The launch post lists what a trace captures, including full conversation audio overlaid on the trace, speech-to-text and text-to-speech latency, voice activity detection events, and interruptions and overlapping speech.

It is not the only way to trace a voice agent, and the alternatives are more alike than their marketing suggests. Most of them read the same spans from the same place. The real differences are further down: which audio each one records, how much of each is actually open source, and what happens when there are no spans to read.

Here is a fact that should make telephony simple. A Twilio call carries μ-law audio at 8000 Hz, and ElevenLabs' text-to-speech API will produce exactly that if you ask for the ulaw_8000 output format. One vendor's output is the other vendor's wire format. No resampling, no transcoding.

And yet none of the four common ways to put an ElevenLabs voice on a Twilio call just passes those bytes through. Each one converts somewhere, and where it converts tells you who is in charge of the audio.