Durable Execution for LLM Agents: The Complete Guide

Durable execution for LLM agents ensures that an agent workflow can survive infrastructure failures by persisting its state at each step, allowing automatic retry and resumption without data loss or duplicate side effects — using patterns like idempotency keys, event sourcing, and workflow engines such as Temporal.

Picture an agent mid-run when the server dies—someone hits Ctrl-C on the wrong terminal, a redeploy rolls the process, the host reclaims the instance. In a naive setup, that run is simply gone. In a durable one, it isn’t: GET /workflows/runs/ABC123 returns status: waiting_human with the full conversation history intact. A moment earlier that workflow was a fragile in-memory object; kill the process and it would have vanished. The difference is three small modules I built into my own FastAPI workflow service: a SQLite store, a runner that snapshots the context after every event, and a route to resume parked runs. Durable execution for LLM agents, at the simplest scale that still works, looks like this.

The design fork every builder hits is the real decision: checkpoint the state, or journal the events. Underneath it sits a silent assumption—that long-running agents can restart after a crash—that the academic literature has only just begun to examine (Ding et al., 2026).

What Is Durable Execution for LLM Agents?

Durable execution is the property that an agent workflow—a sequence of LLM calls, tool invocations, and human-in-the-loop pauses—survives a process crash, a redeploy, or an indefinite wait. The workflow’s state persists to a durable store after each step. On recovery, the agent resumes from the last saved state, not from scratch. The guarantee is at-least-once execution: every step runs at least once, and the system must handle duplicates via idempotency.

For LLM agents, this means retiring the mental model where a workflow lives only in RAM. If you’ve ever lost a multi-turn agent after a server restart, you know the cost: wasted API calls, lost context, frustrated users. Durable execution eliminates that fragility by design.

Why Standard Retry Logic Fails for AI Agents

Simple retry logic—wrap the LLM call in a try/except, retry on timeout—looks like it works, but fails in three ways that matter.

First, duplicate side effects. An LLM call succeeds on the server, but the network response times out. Your code retries. The LLM generates a different function call. Now you’ve just sent two different emails or deducted money twice. Idempotency keys can mitigate this, but only if the LLM API supports them (OpenAI’s retry headers, or your own request ID that the LLM checks). Without a durable store tracking which calls were actually performed, you can’t know whether the first attempt succeeded.

Second, lost intermediate state. A research agent that has gathered three sources and is waiting for the fourth to arrive loses everything when the process dies. Retry logic restarts from the beginning—discarding completed work, burning API budgets, and introducing inconsistency (the LLM may choose different sources the second time).

Third, non-determinism breaks replay. Goertzel (2023) notes that “the cognitive processes exhibited by current LLMs... is fundamentally more limited than the general intelligence of humans.” Practically, this means two retries of the same prompt can produce different valid outputs. A naive retry cannot guarantee the same result, which matters for auditability and reproducibility.

Durable execution saves the agent’s full context—not just the last response—so recovery picks up exactly where execution stopped, without re-executing completed steps.

Core Patterns: Workflows, State Snapshots, and Idempotency

Three patterns underpin durable execution for LLM agents.

Workflow orchestration treats the agent’s run as a state machine with defined steps, each step having a success and failure path. Tools like Temporal, AWS Step Functions, and custom workflow engines provide this abstraction. The workflow defines the overall structure: call LLM, call tool, wait for human, decide.

State snapshots save the agent’s in-memory context to a durable store after every event. This is the approach I used: after each LLM call or tool invocation, the runner calls ctx.to_dict(serializer) and writes the JSON to a SQLite table. On resume, Context.from_dict(workflow, payload) rebuilds the context to continue from exactly where it stopped. The overhead is small—a few kilobytes per snapshot in my runs.

Idempotency keys ensure that retries don’t cause duplicate side effects. Before making any externally visible call (API, database write, payment), the agent generates a unique key—often a hash of the step number and the agent instance ID. The key is stored in the checkpoint. On retry, the agent checks if the key was already processed. If so, it returns the cached result instead of calling the external system again.

These patterns compose naturally. The snapshot captures the agent’s state including the idempotency keys. The workflow orchestrator retries failed steps, and the idempotency mechanism prevents double execution.

Architecture: Where Durable Execution Lives in an Agent Stack

The durable execution layer sits between the agent’s decision loop and the LLM API calls. In a typical stack:

User Input
   |
   v
Agent Orchestrator (workflow runner)
   |
   +--> Durable Execution Layer
   |      - State snapshotter (saves context after each event)
   |      - Idempotency checker (caches LLM responses with keys)
   |      - Failure handler (retries steps, parks on human wait)
   |
   +--> LLM API (GPT-4o, Claude, etc.)
   +--> Tool Invocations (search, database, email)
   |
   v
Durable Store (SQLite, PostgreSQL, Temporal Cloud)

The orchestrator never directly calls the LLM. Instead, it calls the durability layer, which records the intent, calls the LLM, records the result, and returns the response. If the process crashes between recording the intent and receiving the result, the next recovery discovers the pending call and retries it (with idempotency key to avoid duplicates).

In my implementation, the durable store is a SQLite database (runs.db) with a pending_event_json column. The waiting_human status is equally important: when the workflow emits an InputRequiredEvent, the runner sets status to waiting_human and stores the pending question. The run sits at zero compute cost until a human responds via the respond endpoint.

Code Example: Using a Snapshot-Based Runner for LLM Agent Orchestration

Below is a simplified version of the runner glue that makes durable execution work. It uses LlamaIndex’s Context serialization.

import json
import sqlite3
from llama_index.core.workflow import Context

snapshot_db = sqlite3.connect(".workflow-runs/runs.db")

def persist_run(run_id, workflow_handler):
    """Drain a workflow handler, snapshotting after each event."""
    ctx = Context(workflow=workflow_handler)
    while True:
        event = next(workflow_handler, None)
        if event is None:
            break
        # process event...
        # after processing, snapshot the context
        snapshot = ctx.to_dict(serializer=json)
        snapshot_db.execute(
            "UPDATE runs SET context_json = ?, status = ? WHERE id = ?",
            (json.dumps(snapshot), "running", run_id)
        )
        snapshot_db.commit()
        if event.status == "waiting_human":
            snapshot_db.execute(
                "UPDATE runs SET status = 'waiting_human', pending_event_json = ? WHERE id = ?",
                (json.dumps(event.data), run_id)
            )
            snapshot_db.commit()
            break  # park the run

def resume_run(run_id):
    """Load persisted context and resume the workflow."""
    row = snapshot_db.execute(
        "SELECT context_json, workflow_name FROM runs WHERE id = ?", (run_id,)
    ).fetchone()
    if not row:
        raise ValueError("Run not found")
    context_json = json.loads(row[0])
    # workflow class import omitted for brevity
    workflow = get_workflow(row[1])
    ctx = Context.from_dict(workflow, context_json)
    return run_workflow(workflow, ctx)

The waiting_human pattern is crucial. When the agent needs human input, it parks the run rather than polling. The respond endpoint injects a HumanResponseEvent and calls resume_run. This avoids waste and is the same property Temporal markets as “waiting indefinitely at no cost.”

Durable Execution vs. Traditional Queue-Based Retries

Queue-based retries (e.g., Celery, SQS with retry) handle worker crashes by re-enqueuing the message for another attempt. But they treat each LLM call as an independent job, not as part of a multi-step workflow. If an agent makes five LLM calls before crashing, a queue retry would redeliver only the last message—not the full sequence. The agent loses all prior context.

Durable execution, by contrast, saves the entire state machine. The run resumes from the last snapshot, not from the last message. This distinction matters most for agents that maintain conversation history, tool callchains, or multi-turn plans.

The overhead difference is real but small. These are rough, un-benchmarked numbers from my own single-node setup, not published figures: snapshotting adds on the order of tens of milliseconds per event for serialization and a SQLite write, against a fraction of a millisecond for a bare queue enqueue. For agents that spend 1–10 seconds on an LLM call, that overhead is negligible. For high-throughput, low-latency agents, it may become a factor worth measuring.

Common Failure Modes Durable Execution Handles

Durable execution for LLM agents handles four failure modes that plain retries cannot:

1. Process crash mid-step. If the server dies after recording a tool call’s result but before replying to the user, the durable store has the result cached. On restart, the runner skips the duplicate call and returns the cached result.

2. LLM API timeout after success. Same as above—the durable store must record the response before attempting the next step.

3. Human-in-the-loop timeout. The durable store parks the run. If the human never responds, the run stays parked indefinitely without costing compute. A separate scheduler can alert or abort the run after a deadline.

4. Memory poisoning (state integrity). Srivastava & He (2025) show that “attacker-poisoned experiences can introduce persistent, long-term compromise of agent behavior.” If a durable store blindly reloads poisoned state, the agent remains compromised indefinitely. The fix is to validate checkpoints against a separate integrity log before resuming—for example, checking a hash chain that proves no tampering. This is an open challenge; no mainstream durability framework addresses it today.

Performance Trade-Offs and Cost Considerations

The cost of durability is not just infrastructure—it’s also the complexity of deterministic replay. LLM calls are non-deterministic. If you use journal replay (recording every event and replaying the history to rebuild state), you must intercept every LLM call and memoize the response, because replaying the same prompt will likely produce a different answer. This memoization layer adds storage and code complexity.

Snapshot checkpointing avoids this: it saves the state after each event, so on resume it never re-executes an LLM call. The trade-off is snapshot size. In my own runs, a context snapshot for a research agent lands in the low single-digit kilobytes per step (roughly 2–5 KB, by rough estimate rather than benchmark), growing linearly with conversation length. At that rate, 100 steps is a few hundred kilobytes per run—still trivial for SQLite. For long-running agents (hours, thousands of steps), journals can be more space-efficient because they store only the delta, not the full context each time.

The crossover point between snapshot and journal depends on run length and node count. For runs under 10 minutes on a single node, snapshots are simpler and less error-prone. For runs over an hour or any multi-node setup, journals buy you exactly-once semantics and cluster consistency that snapshots cannot match.

When Durable Execution Is Overkill (and What to Use Instead)

Not every LLM agent needs durable execution. Use the following decision framework:

Agent Type	Recommended Approach	Rationale
Single-turn chatbot, no side effects	Stateless, no durability	Simplicity; retries can be handled by the client
Multi-turn but short-lived (3–5 steps)	Snapshot checkpointing (SQLite)	Low overhead; easy to implement
Long-running research agent (hours)	Journal replay (Temporal, Restate)	Exactly-once semantics across steps
Agent with external payments/emails	Idempotency keys + snapshot checkpointing	Prevent duplicate actions
Agent with human-in-the-loop waits	Park-run pattern (waiting_human status)	Zero cost while waiting; separate infrastructure not needed
High-throughput agent (100+ calls/s)	Queue-based retry per API call, separate workflow engine	Reduce overhead; snapshot overhead may be unacceptable
Agent in multi-node distributed system	Journal replay + consistent state store	Snapshots cannot guarantee consistency across nodes

If your agent crashes and losing the last 30 seconds of work is acceptable, skip durable execution. If each crash costs real money or degrades user trust, invest in the pattern.

FAQ

Q: What is durable execution in the context of LLM agents? A: Durable execution is a pattern that preserves an agent's state across failures, so if an LLM call crashes or a server restarts, the agent resumes from the last checkpoint rather than losing all progress.

Q: How does durable execution differ from simple retry logic? A: Simple retry logic repeats the entire failed operation from scratch, while durable execution saves intermediate state and resumes work from the last successful step, preventing duplicate API calls and preserving context.

Q: Do I need durable execution for every LLM agent? A: No — simple stateless agents (e.g., single-turn chatbots) don't benefit; durable execution is valuable for multi-step agents, long-running workflows, and any system where losing intermediate work costs time or money.

Q: What tools support durable execution for LLM agents? A: Popular open-source options include Temporal, Inngest, and DBOS; managed services include AWS Step Functions and Azure Durable Functions — each with different state store and programming model trade-offs.

Q: Can durable execution eliminate all LLM agent failures? A: No, durable execution handles infrastructure and state failures (crashes, network errors) but cannot fix model-level failures like hallucination or refusal — those require separate guardrails.

The Gaps That Remain

The academic literature on LLM agents consistently ignores execution reliability. Across the papers I examined — including frameworks for multi-step planning (Wang et al., 2023), strategic reasoning in competitive auctions (Chen et al., 2023), scientific agent automation (Ren et al., 2025), and healthcare workflows (Vatsal et al., 2026) — none reports failure rates, retry counts, or state recovery times. The consensus is that current research prioritizes capability over reliability.

Two open challenges stand out for the field:

• Deterministic replay with stochastic LLMs. No current framework cleanly handles the tension between memoizing LLM responses forever (storage cost) and allowing model updates (invalidation). Early work reframes this as a governance problem—Chang et al. (2026), for one, admit each agent output as an untrusted proposal against a declared constraint set rather than trusting a model's replayed output—but a general answer is still open.
• Integrity guarantees for persistent memory. Srivastava & He (2025) demonstrate that poisoned experience retrieval can compromise agents persistently. Durable stores must either validate state on load or treat all stored memory as untrusted until verified — an integration that no mainstream workflow engine supports.

The Substrate That the Rest of the Stack Assumes

Durable execution for LLM agents is where microservices were in 2015: everyone knows they need it, but few have a clean abstraction. The snapshot model works for small-scale services and is trivially implementable today. The journal model scales but requires solving deterministic replay for nondeterministic LLMs.

If you’re building production agents today, the choice isn’t whether to make them durable. It’s which durability model matches your scale and your tolerance for replay complexity. My vote, for now, is a SQLite store, a waiting_human status, and the conviction that the simplest correct thing is the right thing until you’ve got a fleet. The papers are arriving; the engineering has started. The gap between what the literature promises and what production demands is exactly where this work lives.