AuthorsWei Zhou, Xuanhe Zhou, Shaokun Han et al.
TL;DR
Are We Ready For An Agent-Native Memory System? decomposes agent memory into four modules and benchmarks 12 systems on 11 datasets, showing no single architecture dominates and revealing sharp cost–performance trade-offs.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Are We Ready For An Agent-Native Memory System? shows that existing evaluations rely on end-to-end metrics like F1 and BLEU while ignoring system internals and operational costs.
Because agent memory is treated as a monolithic black box, critical issues like architectural trade-offs, dynamic update robustness, and latency remain underexplored, causing agents to suffer from contradictions and catastrophic forgetting.
HOW IT WORKS
Are We Ready For An Agent-Native Memory System? decomposes agent memory into Memory Representation and Storage, Memory Extraction, Memory Retrieval and Routing, and Memory Maintenance, then instantiates these with systems like Mem0, Zep, MemTree, LightMem, MemOS, MemoryOS, and A-MEM.
The design is analogous to a database stack where logical schemas, query planners, and storage engines replace a single flat context window, and each memory system chooses a different combination of engines.
This decomposition lets Are We Ready For An Agent-Native Memory System? expose which module fails first—representation fidelity, retrieval precision, update correctness, or lifecycle governance—rather than blaming a generic context window limit.
DIAGRAM
This diagram shows how Are We Ready For An Agent-Native Memory System? conceptualizes the interaction between a user, an LLM agent, and the four memory modules during a single query.
DIAGRAM
This diagram shows how Are We Ready For An Agent-Native Memory System? runs 12 memory systems over five workloads and aggregates multi-metric results.
PROCESS
Memory Representation and Storage
Are We Ready For An Agent-Native Memory System? first maps information into logical forms like token-level sequences, graph and tree-based topology, or heterogeneous composite representations, then binds them to transient registers, single-engine stores, or heterogeneous multi-engine storage.
Memory Extraction
Are We Ready For An Agent-Native Memory System? transforms raw multi-turn dialogues and tool logs via raw sequence concatenation, schema-free semantic extraction, or schema-constrained structured extraction into memory objects ready for indexing.
Memory Retrieval and Routing
Are We Ready For An Agent-Native Memory System? routes queries using native attention-based retrieval, semantic-based dense retrieval, topological subgraph traversal, autonomous agentic routing, or multi-stage hybrid execution before assembling context.
Memory Maintenance
Are We Ready For An Agent-Native Memory System? governs lifecycle with timestamp-based multi-versioning, capacity-driven physical eviction, LLM-driven semantic consolidation, and continuous parametric optimization to handle conflicts, growth, and consolidation.
KEY CONTRIBUTIONS
Technology Decomposition and Taxonomy
Are We Ready For An Agent-Native Memory System? decomposes agent memory into Memory Representation and Storage, Memory Extraction, Memory Retrieval and Routing, and Memory Maintenance, and catalogs systems like Mem0, Zep, MemTree, LightMem, MemOS, MemoryOS, and A-MEM into a structured taxonomy.
Overall End-to-End Performance Evaluation
Are We Ready For An Agent-Native Memory System? evaluates 12 representative memory systems and two baselines across five workloads and 11 datasets, reporting metrics such as 48.0 LLM Judge Accuracy for Zep and 55.4 Task Success Rate for MemoChat.
Fine-Grained Technical Component Evaluation
Are We Ready For An Agent-Native Memory System? runs controlled ablations on representation, extraction, retrieval, and maintenance, showing for example that LightMem User-Only Raw reaches 38.9 Answer F1 on LoCoMo while User-Only Summary drops to 15.6.
RESULTS
LLM Judge Acc.
48.0
+29.0 over Long Context
ROUGE-L F1
35.3
Cognee vs Long Context on LongMemEval
Answer F1
40.8
MemOS vs 32.8 for Long Context on LoCoMo
Task Success Rate
55.4
MemoChat vs 48.2 for Long Context on DB-Bench
On LongMemEval, which tests multi-session long-memory, Zep reaches 48.0 LLM Judge Accuracy while Long Context reaches 19.0, proving that structured temporal knowledge graphs can nearly double judged correctness. On LoCoMo and DB-Bench, Are We Ready For An Agent-Native Memory System? shows that MemOS reaches 40.8 Answer F1 and MemoChat reaches 55.4 Task Success Rate, quantifying how architecture choices affect conversational QA and procedural execution.
BENCHMARK
On LongMemEval, which tests multi-session long-memory, Zep reaches 48.0 LLM Judge Accuracy while Long Context reaches 19.0, proving that structured temporal knowledge graphs can nearly double judged correctness. On LoCoMo and DB-Bench, Are We Ready For An Agent-Native Memory System? shows that MemOS reaches 40.8 Answer F1 and MemoChat reaches 55.4 Task Success Rate, quantifying how architecture choices affect conversational QA and procedural execution.
BENCHMARK
LLM Judge Accuracy on LongMemEval, comparing structured memory systems against the Long Context baseline.
BENCHMARK
Answer F1 on LoCoMo, highlighting how MemOS compares to Long Context and other architectures.
KEY INSIGHT
Are We Ready For An Agent-Native Memory System? finds that simple LightMem achieves 48.3% normalized utility at only 3.67 s per query, while richer systems like Cognee exceed 84% utility only after 116.5 s.
This is counterintuitive because many assume more structured, multi-engine memory is always better, but the study shows global reorganization can erase efficiency gains despite higher accuracy.
WHY IT MATTERS
Are We Ready For An Agent-Native Memory System? gives builders a concrete four-module blueprint and real numbers to choose between sequential context, structural topological, and multi-paradigm hybrid memory.
Armed with these decomposed modules and cost–performance curves, developers can now design agent-native memory systems tuned to workloads like LoCoMo, LongMemEval, and DB-Bench instead of blindly scaling context windows.
Xingyu Lyu, Jianfeng He et al.
· 2026
ADAM combines Anchor extraction, Distribution estimation, Anchor selection, and Query generation to adaptively probe agent memory via an auxiliary generator and entropy based selection. On the EHRAgent benchmark with Llama2-7b-chat, ADAM reaches EQ=77 and ASR=1.00, compared to MEXTRA’s EQ=44 and ASR=0.89.
Okan Bursa
· 2026
Adaptive RAG Memory (ARM) augments a standard retriever–generator stack with a Dynamic Embedding Layer and Remembrance Engine that track usage statistics and apply selective remembrance and decay to embeddings. On a lightweight retrieval benchmark, ARM achieves NDCG@5 ≈ 0.9401 and Recall@5 = 1.000 with 22M parameters, matching larger baselines like gte-small while providing the best efficiency among ultra-efficient models.
Zhanghao Hu, Qinglin Zhu et al.
· 2026
xMemory organises agent memory via segments, memory components, groups, and dynamic maintenance with a sparsity plus semantic faithfulness objective. On LoCoMo with GPT-5 nano, xMemory reaches 38.71 BLEU and 50.00 F1 versus Nemori’s 36.65 BLEU and 48.17 F1, while cutting tokens per query from 9154.76 to 6581.20.
Answers use this explainer on Memory Papers.
Checking…