Memory Papers
Memory Architecture

Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory

AuthorsTianxin Wei, Noveen Sachdeva, Benjamin Coleman et al.

arXiv 20252025
arXivPDF

TL;DR

Evo-Memory uses the unified Search–Synthesis–Evolve loop with the ReMem Think–Act–Refine Memory mechanism to reach 0.92/0.96 success/progress on BabyAI and 0.83/0.95 on PDDL.

SharePost on XLinkedIn

Read our summary here, or open the publisher PDF on the next tab.

THE PROBLEM

Agents Remember Facts But Not Strategies Across Streaming Tasks

Evo-Memory highlights that existing systems “remember what was said but not what was learned,” focusing on static conversational recall instead of experience reuse.

This means long-term LLM agents repeatedly solve similar problems, failing to reuse reasoning strategies, which hurts test-time evolution in multi-turn and streaming environments.

HOW IT WORKS

Evo-Memory and the ReMem Think Act Refine Loop

Evo-Memory centers on Search, Synthesis, Evolve, ExpRAG, and ReMem to unify retrieval, context construction, and memory updates in a streaming loop.

You can think of Evo-Memory like a computer with RAM and disk, where Search is the index lookup, Synthesis is the scheduler, and Evolve is a background defragmenter.

This unified loop lets Evo-Memory with ReMem reason about and reorganize memory at test time, something a plain context window or static RAG pipeline cannot do.

DIAGRAM

Test time Search Synthesis Evolve Loop in Evo-Memory

This diagram shows how Evo-Memory processes a streaming sequence of tasks using the Search–Synthesis–Evolve loop at each time step.

DIAGRAM

Evo-Memory Evaluation Pipeline Across Benchmarks

This diagram shows how Evo-Memory restructures datasets into task streams and evaluates different memory architectures under a unified search–predict–evolve protocol.

PROCESS

How Evo-Memory Handles a Test time Evolution Step

  1. 01

    Search

    Evo-Memory uses Search to retrieve R_t = R(M_t, x_t) from the evolving memory state, capturing similar past tasks and strategies.

  2. 02

    Synthesis

    Evo-Memory applies Synthesis to build C_t = C(x_t, R_t), structuring retrieved experiences into a working context for the base LLM F.

  3. 03

    Evolve

    Evo-Memory constructs a new memory entry m_t from (x_t, y_hat_t, f_t) and updates M_{t+1} via the Evolve operator U.

  4. 04

    ReMem Think Act Refine

    Within each step, Evo-Memory’s ReMem agent alternates Think, Act, and Refine operations to decompose tasks and reorganize memory.

KEY CONTRIBUTIONS

Key Contributions

  • 01

    Benchmark

    Evo-Memory introduces a streaming benchmark that restructures datasets like MMLU-Pro, GPQA-Diamond, AIME-24, and BabyAI into task streams for Search, Synthesis, and Evolve evaluation.

  • 02

    Framework

    Evo-Memory provides a unified evaluation framework with memory-centric metrics and plug-in components such as ExpRAG and ReMem across Gemini 2.5 and Claude families.

  • 03

    Analysis and Insights

    Evo-Memory analyzes experience reuse using ExpRAG and the ReMem Think–Act–Refine pipeline, showing up to 0.92/0.96 success/progress on BabyAI and 0.83/0.95 on PDDL.

RESULTS

By the Numbers

BabyAI Success

0.92

+0.26 over History on Claude 3.7 Sonnet

BabyAI Progress

0.96

+0.23 over History on Claude 3.7 Sonnet

PDDL Success

0.83

+0.18 over AWM on Claude 3.7 Sonnet

ScienceWorld Success

0.62

+0.16 over Mem0 on Claude 3.7 Sonnet

These metrics come from multi-turn embodied reasoning benchmarks BabyAI, PDDL, and ScienceWorld, which test procedural planning and goal completion. The results show that Evo-Memory with ReMem can reuse and refine experience to achieve higher success and progress than History and ReAct baselines under the same search–predict–evolve protocol.

BENCHMARK

By the Numbers

These metrics come from multi-turn embodied reasoning benchmarks BabyAI, PDDL, and ScienceWorld, which test procedural planning and goal completion. The results show that Evo-Memory with ReMem can reuse and refine experience to achieve higher success and progress than History and ReAct baselines under the same search–predict–evolve protocol.

BENCHMARK

Multi turn embodied reasoning results on BabyAI with Claude 3.7 Sonnet

Success metric on BabyAI under the Evo-Memory benchmark.

KEY INSIGHT

The Counterintuitive Finding

Evo-Memory shows that lightweight ExpRAG often matches or exceeds more complex designs, for example reaching 0.87 success on Alf World with Claude 3.7 Sonnet.

This is surprising because many assume sophisticated memory controllers are necessary, yet simple task level retrieval and aggregation already yields large gains over History and Amem.

WHY IT MATTERS

What this unlocks for the field

Evo-Memory enables LLM agents to perform test-time evolution, reusing structured experiences across streaming tasks instead of re-solving each problem from scratch.

Builders can now prototype self improving agents that integrate ReMem style Think–Act–Refine loops, gaining continual adaptation without retraining or heavyweight online optimization.

~14 min read← Back to papers

Related papers

Memory Architecture

A Control Architecture for Training-Free Memory Use

Yanzhen Lu, Muchen Jiang et al.

· 2026

TAG routes low-confidence steps to uncertainty-based routing, filters them with guarded acceptance with rollback, chooses between bank selection across rule and exemplar memory, and prunes via evidence-based retirement inside a unified control loop. On SVAMP and ASDiv, TAG reaches 81.0% and 85.2% accuracy, improving over the 74.0% and 77.5% no-memory baselines while a compute-matched Retry baseline stays flat.

arXiv:2604.18206Read explainer

Questions about this paper?

Paper: Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory

Answers use this explainer on Memory Papers.

Checking…