Concurrency and parallelism in LlamaIndex
Most write-ups about concurrency in LlamaIndex tell you half the story. They show you how to decorate a step with num_workers=5 and call it a day. What they don't tell you is that the storage layer under those parallel steps fails in opposite ways. The default 4 workers guarantee you'll hit both failure modes eventually. I’ve timed the workflow side on llama-index-workflows 2.22.2 (PyPI release) and I’ve lived through the silent corpus inflation and lock-contention meltdowns that happen when those workers hammer the vector store. Here’s the full picture.
Part 1: Workflow-Level Concurrency
The typed fan-out/fan-in that finally works
LlamaIndex Workflows got a serious overhaul in June 2026. The list[Event] return type for fan-out landed in PR #683 (typed list fan-out) and multi-parameter fan-in in PR #673 on June 15, 2026, followed by the Collect/Take/All machinery in PR #674 (typed list[E] join) the next day. All three shipped together as llama-index-workflows 2.22.0 on June 17, 2026 (docs). That means the typed pattern has only been the front-line recommendation since June 17, 2026; before that, the dynamic ctx.send_event/ctx.collect_events API was the canonical guidance.
I installed 2.22.2 (current, released June 30) and ran every claim below. The numbers are not theoretical:
| Pattern | Result |
|---|---|
Typed fan-out/fan-in, num_workers=5, 5×1s tasks | 1.01s — fully parallel |
Same with num_workers=1 | 5.02s — sequential |
Bare @step, 5 tasks | 2.01s = ceil(5/4)×1s — confirms default of 4 workers |
Workers returning None | join fired with only surviving branches |
Collect(Take(1)) | returned the fastest result in 0.21s, did not wait for the rest |
The old ctx.send_event/collect_events pattern | runs unchanged on 2.22.2 |
Two implications jump out. First, the @step decorator bakes in num_workers: int = 4 — every step is concurrent by default, bounded at 4 workers, without you asking for it (workflow guide). Second, the dynamic API is not deprecated. It is still the right tool when you have conditional emission, unknown event counts, or want downstream steps to start while the producer is still running. The trade-off: a list return is all-or-nothing — an exception before the return emits nothing. ctx.send_event fires immediately and the event stays emitted even if the step later fails.
What Take(n) does not do
Take(3) on a Collect of 5 tasks does not kill the slower two. They keep running to completion; they just never reach the join. In my timing run, Collect(Take(1)) returned the fastest branch in 0.21s — but the other four tasks ran to the end anyway. If those branches are LLM calls, you still pay for every one of them. Take buys latency, not cost. Don’t use it expecting to save the spend of the slower branches.
The version-pin gotcha
The latest llama-index-core (0.14.23) pins llama-index-workflows>=2.14.0,<3. A fresh install (as of July 2026) resolves to 2.22.2 and everything works. But any environment or lockfile from before mid-June 2026 can be sitting on 2.14–2.21, where the typed fan-out/fan-in does not exist. Code copied from the current docs fails there. A fan-out edge-case fix ("keep implicit waiters distinct across fan-out") shipped June 23 in 2.22.2 — be on ≥2.22.2, not 2.22.0. pip install -U llama-index-workflows is the fix.
How I actually use Workflows in production
One of my daily batch-drafting lanes is a Workflow whose fan_out step returns list[TopicTask] and whose worker step is @step(num_workers=workers) — num_workers is baked at class-definition time, so I generate the class from a factory to make the worker count a runtime parameter. Each worker runs a per-topic generation as an async subprocess with its own timeout/kill.
My audio prewarm lane uses the dynamic pattern (ctx.send_event/collect_events) deliberately: ctx.send_event(ChapterEvent(...)) per chapter, @step(num_workers=N) workers, and fan-in via ctx.collect_events(ev, [HookEvent] * n, buffer_id=...) — the chapter count is only known at runtime.
I have also rejected the fan-in pattern on the merits in one lane: a grounding pipeline where per-item isolation and simple sequential recovery mattered more than wall-clock. And one lane deliberately does not use Workflows at all: my parallel grounding runner is a plain ThreadPoolExecutor fanning chapter queries over one shared LlamaIndex query engine. That is safe because query() builds fresh per-call retriever/synthesizer state — concurrent reads over a shared vector index are fine — and I cap workers at 6 anyway because the binding constraint is the LLM provider’s rate ceiling, not CPU. The lesson: workflow parallelism is often throttled by the slowest external dependency, and that dependency is usually the LLM API or the vector store.
Part 2: Storage-Layer Pitfalls
What parallel steps hit: LanceDB vs Qdrant write paths
num_workers=4 by default means the storage layer underneath your steps sees concurrent and repeated access whether you planned for it or not. The two stores I run behind LlamaIndex — LanceDB and Qdrant — sit at opposite ends of the coordination spectrum.
LanceDB is embedded and file-based (docs). No server process, no daemon lock. Tables are versioned and writes commit with optimistic concurrency. Concurrent readers are cheap and safe. The danger is the write path semantics: what a "write" means depends on how you construct the LlamaIndex objects.
Qdrant embedded (local path mode) is single-writer (storage concepts): the process holds a directory lock, and a second process opening the same path gets a storage-lock error ("already accessed by another instance"). Concurrency across processes is simply not allowed.
Qdrant server moves coordination server-side: many clients, concurrent upserts, point-level idempotency by ID, and an explicit administrative write-lock API (POST /locks with write: true makes the whole instance read-only, with a reason string) for maintenance windows (administration guide).
War story 1: the LanceDB silent-append bug
In one of my daily drafting lanes, each run built a VectorStoreIndex over the topic's LanceDB table via LanceDBVectorStore (API reference). What I missed: constructing a VectorStoreIndex with nodes over an existing LanceDB table appends — it does not replace. There was no error, no warning, and retrieval still returned results. It ran unnoticed for 34 daily runs, by which point the token-economics table held 36,958 rows over 2,173 distinct chunks — every run had re-embedded and re-appended the whole corpus. The casualty was retrieval quality: similarity_top_k=20 returned ~20 copies of one or two passages, so the reranker's top-8 was duplicates of duplicates. That’s a 17× corpus inflation (36,958 / 2,173) that took 34 days to notice.
The fix I shipped has three parts:
- Fingerprint the table name with a sha256 of the sorted source paths, so "same corpus" is a stable identity.
- On open, read the row count. If
rows == len(nodes), the table is a finished index — attach read-only viaVectorStoreIndex.from_vector_store(...)and skip the write path entirely. - Otherwise the table is partial or stale — rebuild with
LanceDBVectorStore(..., mode="overwrite")(appending to a wrong-count table would compound the fault), then drop stale fingerprinted tables for the same topic.
The LlamaIndex LanceDBVectorStore API exposes exactly the knob that matters here: the mode parameter ("create"/"overwrite"/append behavior). The general rule: with an embedded store there is no server to enforce "this index already exists, don't double-write"; idempotency is your job, and the default path is append.
War story 2: why that lane is on LanceDB at all — the Qdrant embedded lock
The same repo’s canonical corpus lives in embedded Qdrant. Embedded Qdrant is single-process: the directory lock means my daily generation run collided with the audio and grounding lanes whenever they overlapped — the second process gets a lock-contention error, not a queue. My memory_common layer grew a whole immune system around this:
- a
_lock_contention()classifier that recognizes the "already accessed" / lock errors; _connect_with_backoff()— bounded exponential backoff (default 3 attempts, 0.5s base) that applies to both server and embedded modes, because embedded’s dominant failure IS another process holding the single-writer dir lock;- a process-wide cached client behind a
threading.Lock, with a sticky "unavailable" latch and a cooldown window so one blip doesn’t silently degrade the whole process onto an in-memory backend; - a
REQUIRE_QDRANT=1fail-closed policy, because the worst outcome is not a crash — it’s a run that "succeeds" against an in-memory store and persists nothing.
The pragmatic resolution: the colliding lane got its own private, file-based LanceDB store (concurrent readers tolerated, no dir lock to fight over), while embeddings are reused from a shared sqlite embed-cache keyed by sha256(model ‖ text) so only the index rebuild is duplicated work. Store choice here was a concurrency decision, not a features decision.
The upsert contrast: why my Qdrant lane never had the append bug
The Qdrant ingest path in the same codebase is idempotent by construction (QdrantVectorStore API): deterministic point IDs plus delete-then-reinsert by ref_doc_id, so a forced re-sync upserts rather than duplicates, and unchanged docs skip split/embed/insert entirely via a per-doc sha manifest. Re-running it 34 times produces the same collection. That is the exact opposite failure surface of the LanceDB lane: Qdrant’s point-ID semantics make repeated writes converge; LanceDB’s append semantics make repeated writes accumulate. Neither is "better" — but you must know which contract you are writing against, because num_workers=4 and daily cron runs will exercise it.
Decision framework for concurrency in LlamaIndex
Based on the evidence above, here’s a grounded decision framework:
| Your workload | Recommended pattern | Storage choice | Worker count heuristic |
|---|---|---|---|
| Known-size fan-out (e.g., fixed document list) | Typed list[E] fan-out/fan-in on ≥2.22.2 | Whichever store; enforce idempotency | Cap to LLM rate limit, not CPU |
| Unknown-size / conditional emission | Dynamic ctx.send_event + collect_events | Whichever store; enforce idempotency | Cap to store write throughput |
| Repeated daily writes | LanceDB with mode="overwrite" + fingerprint | LanceDB (prefer overwrite mode) | 1-2 workers to minimize collision |
| Concurrent read-heavy load | Shared VectorStoreIndex.from_vector_store() (read-only) | LanceDB for reads; Qdrant for coordinated upserts | 6-8 workers; watch rate limits |
| Single-process strict | Embedded Qdrant with backoff + fail-closed | Qdrant embedded (single-writer) | 1 writer, multiple readers if read-only |
| Multi-process coordination | Qdrant server with deterministic point IDs | Qdrant server; use locks API for maintenance | Up to server’s concurrent upsert capacity |
Inside a workflow: prefer the typed list[E] fan-out/fan-in on ≥2.22.2 for known-size fan-outs (it deletes num_to_collect bookkeeping and None-check dances and validates the event graph). Keep the dynamic API for conditional/unknown-count emission. Remember Take(n) does not cancel — the losing tasks still run to completion.
Parallel reads over one shared LlamaIndex index/query engine are safe; per-call state is fresh. Repeated/concurrent writes require explicit idempotency: against LanceDB, fingerprint + row-count check + mode="overwrite", attach read-only when the index is already built; against embedded Qdrant, assume single-writer and add backoff + fail-closed policies; against Qdrant server, lean on deterministic IDs/upserts and use the locks API for maintenance windows.
Fail closed on storage degradation: a lane that silently falls back to an in-memory store reports success and persists nothing.
Practical guidance to close on
The LlamaIndex workflow layer has matured into a genuinely useful concurrency model — but only if you pin to the right version and understand the trade-offs between typed and dynamic APIs. The storage layer underneath is where most production surprises live, not in the Python orchestration.
Finally, cap num_workers to the real bottleneck. In my grounding pipeline, the binding constraint is the LLM provider’s rate ceiling, not CPU count — six workers saturate it. Measure the actual throughput before turning the knob to 10 — otherwise you’re just generating retries faster.
The LlamaIndex ecosystem gives you the tools to build concurrent pipelines. It also gives you enough rope to hang yourself with silent appends and lock contention. The difference is knowing which end is which before you go to production.