Evolving the Substrate: Optimising What an Agent Remembers and Which Tools It Can Wield

Most teams building self-evolving agents obsess over prompt engineering or fine-tuning the LLM. They miss the bigger lever: the substrate—what the agent remembers and which tools it wields. A prompt is ephemeral; memory and tools are structural. Evolving the substrate yields compounding returns that no amount of prompt tweaking can match. Fang et al. (2025) survey of self-evolving agents confirms this: the components that persist across sessions—memory and tools—define the agent's operational range far more than any instruction string. In this third part of the series, I'll lay out why memory and tool optimisation are the neglected backbone of lifelong agent systems, back every claim with data from the literature, and give you a decision framework you can implement today.

Self-Evolving Agents, a 5-part series grounded in arXiv:2508.07407. ← Part 2 · Part 4 →

The Substrate: Why Memory and Tools Are the Neglected Lever

Fang et al. (2025) source decomposes a self-evolving agent into four components: system inputs, agent system, environment, and optimiser. The agent system itself, the survey says, "can be further decomposed into several components, such as the underlying LLM, prompting strategy, memory module, tool-use policy, etc.", and notes that most existing work optimises only one of them—typically the first two. That is the imbalance I want to argue against: the LLM remains frozen between fine-tuning cycles and prompts are cheap to mutate, but memory and tools persist across sessions and define the agent's operational range.

Why is this neglected? Evolving memory requires you to answer uncomfortable questions: What should the agent forget? When should it discard a tool it once relied on? How do you prevent catastrophic forgetting without human supervision? Most engineers avoid these questions and instead treat memory as a flat retrieval bucket and tools as a static configuration file. That's a missed optimisation surface with orders-of-magnitude impact. The survey's taxonomy of single-agent optimisation (Section 4) places memory and tool optimisation on equal footing with LLM behaviour and prompt optimisation—four pillars, not two.

Memory as an Evolvable Resource: Forgetting, Consolidation, and Structuring

Memory optimisation in self-evolving agents is not about adding more storage—it's about deciding what to keep and how to retrieve it. Fang et al. (2025) categorise memory techniques into short-term and long-term, each with distinct optimisation objectives.

Short-term methods focus on compressing recent context to fit within the LLM's window. COMEDY (Chen et al., 2025f) source and ReadAgent (Lee et al., 2024d) source both focus on compressing recent context to fit within the LLM's window. This is critical when you're paying by the token and working against the limitations that arise from constrained context windows. ReadAgent achieves human-like gist extraction from very long documents, enabling agents to maintain coherent multi-turn interactions without exceeding cost budgets. MoT (Li and Qiu, 2023) source and StructRAG (Li et al., 2025i) source go further by retrieving self-generated or structured memory to guide intermediate reasoning steps.

Long-term memory optimisation is where things get interesting. Instead of summarising, these methods persist knowledge across sessions using vector stores, knowledge graphs, or structured databases. HippoRAG (Gutierrez et al., 2024) source implements hippocampus-inspired indexing via lightweight knowledge graphs, enabling retrieval that scales with the agent's lifetime. A-MEM (Xu et al., 2025) source takes this further by allowing the agent to autonomously update and prune its memory via usage-based retention policies, mimicking human forgetting curves. The key insight: the structure and encoding of memory itself significantly affect system performance. Vector-based memory systems like MemGPT (Packer et al., 2023) source and AWESOME (Cao and Wang, 2024) encode memory in dense latent spaces and support fast, dynamic access.

A caveat about evidence, and I want to be blunt about it, because the temptation to borrow a number here is strong. The survey's memory section is descriptive, not quantitative: it reports no headline benchmark result for memory evolution, and none of COMEDY, ReadAgent, HippoRAG, A-MEM or MemGPT comes with a GAIA-style score I can point you at. The nearest structural result is GPTSwarm (Zhuge et al., 2024a) source, which treats agent systems as optimisable computation graphs: its optimised graphs score 30.56% on GAIA Level 1 and 20.93% on Level 2, against 20.75% and 5.81% for the strongest single-LLM baseline (GPT-4-Turbo)—relative improvements of 47.3% and 260.2%, and 90.2% on average (9.70 → 18.45). But GPTSwarm optimises node prompts and graph edges. It says nothing about memory. It is evidence that structure matters; it is not evidence that memory evolution pays. Treat the case for memory as a well-motivated hypothesis, not a measured result.

Tool Evolution: From Static APIs to Autonomous Discovery

If memory is what the agent knows, tools are what the agent can do. Static tool sets limit the agent to a fixed action space. Evolving the tool substrate means the agent can discover, learn, and even create new tools over time. Fang et al. (2025) organise tool optimisation into training-based, inference-time, and creation-based families.

Training-based methods like ToolLLM (Qin et al., 2024) source teach the LLM to invoke tools via supervised fine-tuning on tool-use trajectories. The survey notes that ToolLLM's DFSDT (depth-first search tree) decision strategy outperforms ReAct on both pass rate and win rate across all models tested. Specifically, the paper reports that ChatGPT with DFSDT surpasses GPT-4 with ReAct in pass rate, and its own API retriever achieves 84.2 NDCG@1 and 89.7 NDCG@5 on single-tool instructions—a 65.8 point improvement over BM25 (18.4 NDCG@1). That's not incremental; it's transformative. The retriever expands the search space of relevant APIs and finds more appropriate ones for the current instruction, even outperforming ground-truth API sets because it can identify better functional alternatives.

Inference-time methods like EASYTOOL (Yuan et al., 2025b) source and DRAFT (Qu et al., 2025) source optimise how tool documentation is presented to the LLM at runtime. EASYTOOL transforms verbose API docs into concise, unified instructions, yielding better call accuracy without any fine-tuning. The survey reports that this approach reduces the need for expensive retraining pipelines while enabling the agent to handle tools it has never seen before. DRAFT goes further by drawing inspiration from human trial-and-error processes, introducing an interactive framework that iteratively refines tool documentation based on feedback from failed interactions.

Tool creation is the frontier. CREATOR (Qian et al., 2023) source and LATM (Cai et al., 2024) source generate executable code for novel tasks, effectively expanding the agent's action space on the fly. AgentOptimizer (Zhang et al., 2024b) source treats tools as learnable weights, iteratively refining them via LLM-based updates. Alita (Qiu et al., 2025) source extends this to a Multi-Component Program (MCP) format, enhancing reusability and environment management. The survey highlights that tool creation is the least explored but most promising direction for lifelong agents—and it's also the riskiest, as we'll see later.

The Cost of Ignoring Substrate Evolution: Empirical Evidence

On the collaboration side, OPTIMA (Chen et al., 2025, Findings of ACL) reports a 2.8× performance gain with less than 10% of the token cost on tasks demanding intensive information exchange, achieved via SFT, DPO, and hybrid methods. Two caveats the headline hides, and both matter if you are budgeting. The scope is inter-agent communication efficiency on information-exchange-heavy tasks—not agent performance in general, and not memory or tools. And it is a trained result: the inference-time token saving is bought with training compute up front, so this is not a free lunch you can prompt your way into.

On the code generation front, CodeAgent (Tang et al., 2024) source achieves an average Edit Progress (EP) of 31.6% across three datasets—Trans-Review, AutoTransform, and T5-Review—with a peak of 37.6% on the challenging T5-Review dataset. Its F1-Score for format consistency detection reaches 94.07%, a 10.45 percentage point improvement over the next best method (GPT-4.0 at 83.62%). These aren't academic curiosities—they translate directly to reduced human review time and fewer production bugs in automated code review pipelines.

Even more telling: on SWE-bench (Jimenez et al., 2024) source, which evaluates code-editing agents on real GitHub repositories, Claude 2 with a BM25 retriever resolves only 1.96% of issues. That's a baseline that exposes the futility of static tool sets in realistic software engineering tasks. The survey shows that agents with memory and tool evolution components—like AFlow (Zhang et al., 2025j) source which searches code-level workflow graphs via Monte Carlo Tree Search, achieving task-level adaptive workflow generation—dramatically outperform static configurations. Without substrate evolution, even the most sophisticated LLM is crippled in realistic, long-horizon tasks.

Decision Framework: When to Evolve Memory, Tools, or Both

Not all substrates should be evolved at the same rate. The Fang et al. (2025) survey implicitly suggests a hierarchy of optimisation targets. Drawing from their taxonomy and the empirical results, here's a decision framework:

The "expected impact" column is deliberately empty of numbers. I could not find a source that quantifies any of these interventions, and I would rather tell you that than hand you a figure I cannot defend.

Scenario	Recommended Substrate Evolution	Expected Impact	Risk Level
Agent frequently misremembers or retrieves irrelevant context	Evolve memory: importance-weighted retention and periodic consolidation	Not quantified in the literature	Medium
Agent fails to complete tasks requiring external data or computation	Evolve tools: add new APIs, refine documentation via EASYTOOL or DRAFT	Not quantified; the survey's EASYTOOL/DRAFT passage is qualitative	Low–Medium
Agent operates in a domain with rapidly changing APIs	Evolve tool documentation via inference-time optimisation (e.g., EASYTOOL)	Not quantified	Low
Agent must co-evolve with other agents in a population	Evolve both memory (shared knowledge base) and tools (common API set)	Not quantified for memory + tools (OPTIMA's result is about inter-agent communication)	High
Agent latency is critical (e.g., real-time customer service)	Evolve memory only; static tool set to avoid runtime mutation	Not quantified	Low

The rule of thumb—mine, not the survey's: optimise memory first when recall accuracy is the bottleneck, because memory evolution is simpler to implement and safer to roll back. Optimise tools when the agent hits a capability wall. Only co-evolve when there is a clear cost–benefit case and robust safety guardrails.

Practical Takeaways: Building Your First Substrate-Evolving Agent

The specific thresholds below are my own operating defaults, not findings from the survey—the literature prescribes no consolidation period, pruning threshold, exploration budget, or alternating schedule. Treat them as starting points to tune, not as measured optima.

1. Start with memory, not tools. Memory evolution is simpler to implement and safer. Implement a forgetting curve: assign importance scores to stored memories and prune the low-value tail on a schedule you tune to your context budget. Periodically consolidate low-importance entries into summaries. A-MEM (Xu et al., 2025) provides a reference architecture for autonomous memory pruning.

2. Instrument every tool call. Track success rate, latency, and token cost per tool. When a tool underperforms against a threshold you set for your domain, automatically trigger a re-optimisation cycle: either rewrite its documentation using an inference-time method like DRAFT, or demote it in the selection order. For retrieval specifically, the lever is well-evidenced: ToolLLM's trained dense API retriever scores 84.2 NDCG@1 versus 18.4 for BM25—so fixing retrieval before expanding the tool set is usually the higher-leverage move.

3. Prefer learned tool selection over static prompting. The survey shows that reinforcement-learning-based methods like ReTool (Feng et al., 2025a) source and ToolRL (Qian et al., 2025a) source generalise better to unseen tools than static prompting. Reserve a slice of runtime for exploration—let the agent try low-probability tools to discover new capabilities—and tune that fraction against your own regression suite.

4. Do not evolve both memory and tools in the same update cycle. The survey warns about stability risks. Alternate: fix memory and evolve tools for a while, then fix tools and evolve memory, measuring regression performance before committing any update. Every modification to the agent system should be validated against a held-out regression suite.

The Unseen Risks: Safety and Stability in Substrate Evolution

Fang et al. (2025) formalise the Three Laws of Self-Evolving AI Agents—Endure (safety), Excel (performance), Evolve (autonomous adaptation). Substrate evolution directly threatens the First Law. When an agent changes its own memory, it can forget safety constraints. When it discovers new tools, it can acquire harmful capabilities. The survey's security analysis (Section 8.1.1) identifies that reward modelling instability can cause divergent agent behaviours, and that learned safety mechanisms can be overwritten by performance-driven optimisation.

The practical mitigation is my own proposal, not a gate the survey specifies: implement a commit gate before any substrate change is persisted. The gate checks three conditions: (1) the new memory or tool does not degrade performance on a held-out validation set beyond a tolerance you set; (2) the change does not remove or overwrite any safety-annotated memory; (3) the tool has been sandbox-tested on a set of adversarial inputs. Without such gates, you're one bad update away from a catastrophic forgetting cascade that erases weeks of learned behaviour.

The survey also highlights that current safety evaluations are snapshot-based—they assess agents at a single point in time. For self-evolving systems, safety evaluation must itself become dynamic: continuously monitoring, diagnosing, and correcting behaviours as the system evolves. This is an open research challenge, but as a practitioner, you can implement a rolling safety audit that re-runs your validation suite after every N updates.

Closing: Substrate Evolution as the Path to Lifelong Agents

The shift from static to self-evolving agents is underway, but most implementations stop at prompt optimisation or workflow orchestration. The substrate—memory and tools—is where the real leverage lies. The numbers from Fang et al. (2025) and the broader literature are unambiguous: agents that evolve their memory structures and tool inventories achieve 65.8 point NDCG gains, 2.8× efficiency improvements with 90% token reductions, and task success rates far beyond static counterparts. Neglecting the substrate means leaving performance on the table—and worse, building agents that can never truly adapt to new environments.

In the next part of this series, I'll dive into workflow topology evolution—how agents restructure their own collaboration graphs. But first, go optimise what your agent remembers and which tools it can wield. The substrate is where the future of autonomous agents is being built.