Skip to main content

LiveKit, Pipecat, and the seam where a voice agent actually breaks

· 8 min read
Vadim Nicolai
Senior Software Engineer

Pick a voice stack and you are really picking two things: who owns the audio transport, and what you can see when a turn goes wrong. The vendor list barely matters — LiveKit Agents and Pipecat both put the same speech-to-text, language model and text-to-speech providers behind the same three seams, and both let you swap any of them in a line.

LangSmith shipped tracing for both in July, alongside OpenAI Realtime and Gemini Live. That is a useful forcing function for this comparison, because a trace shows you where a framework's abstraction actually sits — and the two answers are genuinely different.

Twilio behind LiveKit, behind Pipecat, or on its own: three places to put the phone line

· 9 min read
Vadim Nicolai
Senior Software Engineer

"Add Twilio" sounds like one integration. It is three, and they disagree about the most basic question in a voice agent: who holds the audio. LiveKit takes the call as a SIP trunk and turns the caller into a participant in a room. Pipecat takes it as a WebSocket carrying raw audio frames. And Twilio's own ConversationRelay keeps the audio entirely, and hands your server text.

The previous post compared LiveKit and Pipecat on the seam where a voice agent breaks: a caller talking over the bot. Put a phone line in front of either one and that seam moves, because now there is a third system buffering audio that neither framework controls.

LangSmith's voice tracing against the rest: same spans, different audio

· 15 min read
Vadim Nicolai
Senior Software Engineer

In July, LangSmith launched voice tracing for four frameworks: Pipecat, LiveKit, OpenAI Realtime and Gemini Live. The launch post lists what a trace captures, including full conversation audio overlaid on the trace, speech-to-text and text-to-speech latency, voice activity detection events, and interruptions and overlapping speech.

It is not the only way to trace a voice agent, and the alternatives are more alike than their marketing suggests. Most of them read the same spans from the same place. The real differences are further down: which audio each one records, how much of each is actually open source, and what happens when there are no spans to read.

An ElevenLabs voice on a Twilio call: four routes, one 8 kHz wire

· 10 min read
Vadim Nicolai
Senior Software Engineer

Here is a fact that should make telephony simple. A Twilio call carries μ-law audio at 8000 Hz, and ElevenLabs' text-to-speech API will produce exactly that if you ask for the ulaw_8000 output format. One vendor's output is the other vendor's wire format. No resampling, no transcoding.

And yet none of the four common ways to put an ElevenLabs voice on a Twilio call just passes those bytes through. Each one converts somewhere, and where it converts tells you who is in charge of the audio.

RemoteGraph, and what every other framework does instead

· 13 min read
Vadim Nicolai
Senior Software Engineer

LangGraph ships a class that turns an agent running on another machine into a node in your graph. You construct a RemoteGraph, pass it to builder.add_node(...), and the call site reads exactly like a local subgraph. It subclasses PregelProtocol, so as far as your parent graph is concerned there is no network there at all.

I wanted to know what that costs, and what the other frameworks offer in its place. Two of them can be measured: the same parent-and-child pair runs against a real LangGraph server and against a LlamaIndex workflow server, with the same five contract mismatches pushed through each boundary. CrewAI, the OpenAI Agents SDK and A2A are read from their docs and specs, because two of the three turn out to have no remote-agent primitive to measure.

The headline result is an inversion. RemoteGraph looks typed and validates nothing at the boundary: I renamed one field in the child, redeployed it, left the parent untouched, and the parent returned its own input with no error and no warning — a result indistinguishable from the child never running. LlamaIndex has no RemoteGraph at all; its server-and-client pair looks like a raw HTTP call, publishes a JSON Schema, and rejects a malformed request with a 400 and per-field errors. The same rename throws a KeyError on the line I wrote.

A fifth of my trading universe was delisted stocks, understating every signal by half

· 19 min read
Vadim Nicolai
Senior Software Engineer

The number I did not expect was 20.5%. That is the share of my screen-eligible universe that had already delisted and fallen out of the research panel before I scored a single signal on it. The direction was the bigger surprise. Dropping those names did not flatter my backtest; it suppressed it, understating every lane's spread by a mean 19.3 bps (se 6.5) against a headline lane spread of 37.99 bps. Just over half of a result, traceable to an instrument nobody had audited.

The number a trading objective has to beat before it means anything: my system finally measured the base rate for a 10% move in five days, and it is forty-two times larger than anyone had measured

· 21 min read
Vadim Nicolai
Senior Software Engineer

For a year the board has run a cross-sectional screen over roughly 12,500 US-listed equities a day. It ranks names on momentum, reversal, short-interest flow and intraday range, then scores every lane against how the winners and losers actually moved. The pipeline is unremarkable — the same architecture-capability-adaptation stack that Xia et al. (2026) audit across 77 LLM-trading studies, minus the LLM. It has measured a great deal. It had never measured the one number its own standing objective rests on.

The 8-K category separation did not survive honest standard errors and a holdout — the loop withdrew its own headline

· 22 min read
Vadim Nicolai
Senior Software Engineer

title: "AI-native trading: The 8-K category separation did not survive honest standard errors and" status: published

The 8-K category separation did not survive honest standard errors and a holdout — the loop withdrew its own headline

Nine days after the trading loop published a result showing that 8-K categories separate the winners from the losers in a cross-sectional equity screen, it sat down with the three caveats its own record had attached to that result and ran all three. The first correction made the number bigger. The "financial results" delta went from +4.83 percentage points to +9.38, a 4.55-point move produced by fixing the sampling frame rather than the question — the kind of movement Zhang et al. (2026) make measurable when they toggle one evaluation convention at a time while holding everything else fixed.

That is where the claim died. Not because a correction shrank it. Because a correction moved it.

A statistic that unstable was never about the category.

Cloudflare Tunnel: post-quantum by default, not by guarantee

· 16 min read
Vadim Nicolai
Senior Software Engineer

The Cloudflare Tunnel overview states its promise in a single line: connect your origin servers, APIs, and services to Cloudflare "with post-quantum encrypted tunnels — no public IPs required."

That sentence describes the software's default behaviour. One level down, in the run-parameter reference, the same documentation explains what happens when the default does not hold. cloudflared connects over QUIC using post-quantum cryptography, the docs say, "but will fall back to non-PQ if there are issues connecting."

Both statements are accurate. The distance between them is one flag wide, and Cloudflare's own roadmap explains why that distance matters: "Adding support for PQ cryptography is not enough. Systems must disable support for quantum-vulnerable cryptography to be secure against downgrade attacks." The tunnel ships the support. The flag is how you disable the fallback.

Autonomous trading: sample size sets the bar, not the number

· 20 min read
Vadim Nicolai
Senior Software Engineer

The largest spread-to-standard-error ratio on the board is 4.2996, and it is not significant. It has to clear 4.302653 — the two-sided 5% critical value on 2 degrees of freedom. That is not the 1.959964 the rest of the one-day table is measured against. 4.2996 falls 0.003 short. Nothing about the number is wrong; it was judged against a bar that belonged to a different sample size — the error Bailey and López de Prado (2014) built the deflated Sharpe ratio to catch, where the length of the track record behind a statistic is part of the threshold and not a footnote to it.

That gap is worth a long article not because 0.003 is large, but because the board prints no column that says so, and because the system that produced the number declined to promote, demote or score anything on the strength of it. The failure mode has a name. Bailey and López de Prado (2014) describe it as an undeflated ratio: a performance statistic reported without controlling for the number of trials behind it, the length of the track record, and the non-normality of the sample. The correction applied to this cell is the crudest possible version of their adjustment — the one that comes free with a t-table.