Forever-Loops on asyncio.to_thread Starve LangGraph Sync Nodes
LangGraph hands every plain def node under ainvoke to the event loop's default thread pool — the same one asyncio.to_thread borrows from. Fill that pool with loops that never return and sync nodes queue behind them silently, no error, no timeout.
LangGraph in production, a 10-part series. ← Part 7 · Part 9 →
Under ainvoke, a sync node runs on the same pool asyncio.to_thread borrows from
add_node coerces def nodes into runnables, and under async invocation LangChain's helper calls the running loop's run_in_executor(None, ...). The argument that matters is None: "The default executor is used if executor is None." asyncio.to_thread is a thinner wrapper over the identical call (separate thread) and lands on the same pool.
So a restock poller started with to_thread and a pick-list node inside the graph are tenants of one landlord with a fixed number of rooms. loop.getaddrinfo runs on that executor too, which is why an HTTP client fetching shipment status can stall on DNS for exactly the reason the graph stalls at its first sync node.
asyncio sizes that pool from the CPU count, so the smallest box hangs first
The default is "min(32, os.cpu_count() + 4)"; from 3.13 the count follows os.process_cpu_count(). A four-CPU container gets eight workers.
No number is written anywhere in your code, which is why the failure cannot be reproduced where it was written. The test has to pin the small size rather than inherit the developer's: "Set executor as the default executor used by run_in_executor()." Pin it to eight, and the stall becomes a two-second test.
A loop that never returns keeps its worker, and the graph says nothing
A forever-loop is a function that has not returned; the worker that picked it up never comes back. Once every worker is inside one, new submissions sit in the executor's queue with nobody to drain it. The graph hangs at its first sync node — no exception, no log line, no timeout unless the caller set one.
There is a second symptom: shutdown. asyncio.run waits for the default executor to close, so loops left on it also hang the process at exit.
Here is the measured shape. Pin the pool to a small box's size, start one blocking loop per worker under to_thread, and a one-node graph whose node counts bins does not answer inside a two-second timeout. Move the same loops onto named daemon threads, keep the same pinned pool, and it answers.
loop.set_default_executor(ThreadPoolExecutor(max_workers=8)) # a four-CPU box
for feed in supplier_feeds:
threading.Thread(target=watch, args=(feed, stop),
name=f"watch-{feed}", daemon=True).start()
Output flips from graph starved: no free worker for its sync node to graph answered: {'bins': 3}.
"This is an application bug, not a LangGraph bug" — right about the cause, wrong about the fix
The objection deserves stating fairly: LangGraph offloads plain def nodes correctly, an async def node never touches the pool, and an application starting blocking while True pollers on a shared executor has broken the contract. Raising max_workers is legitimate capacity planning.
All true, and none of it changes the remedy. Raising the pool buys room for exactly the loops that exist today; add one supplier feed and the silent stall returns in live mode rather than in review.
A dedicated executor gives explicit sizing and shutdown, and is better when there are dozens of loops; starve it and the hang comes back inside it. Converting nodes to async def is correct only when the client is genuinely non-blocking. Process isolation is the only real escape from a callable that never returns, and it costs an IPC boundary. A named daemon thread per loop costs one thread and a stop event, and is the smallest change that holds.
Put the rule in a test, not in the pool size
Three things belong in the suite. Assert the hang itself against a pinned pool, so a future Python or LangGraph change gets noticed instead of silently invalidating your reasoning. Scan the loop-starting code and fail on to_thread or run_in_executor(None, ...) for anything that never returns — plant a violation and confirm the test catches it. And name the threads, because the only artefact you will have during an outage is a thread dump.
A shared default executor has no concept of a tenant that never leaves. Ownership of infinite work is an architectural decision, and no amount of capacity is a substitute for it.