Best for
- • Long-term assistant memory
Memory benchmark
LongMemEval
A scalable benchmark for long-term interactive memory in chat assistants, covering information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention.
Evaluation style
- • Multi-session conversation QA
Memory horizon
- • Scalable histories; 500 questions
Scale
- • 500 curated questions across five long-term memory abilities, with histories that can be scaled in length.
Tasks
- • Information extraction
- • Multi-session reasoning
- • Temporal reasoning
- • Knowledge updates
- • Abstention
What it measures
- • Answer correctness
- • Evidence use
- • Cross-session recall
- • Temporal consistency
Original research
LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
This is the paper that introduced LongMemEval.
arXiv:2410.10813 ↗Used in the field
Papers evaluated on LongMemEval
Memory Architecture
AgentIR: A Workload-Adaptive Cascade Retrieval Substrate for Long-Term Conversational Memory
Aojie Yuan, Haiyue Zhang, Shahin Nazarian
arXiv 2026 · 2026
AgentIR combines a SIMD-Accelerated BM25 engine, a Temporal Partitioned Index, an Agent-Aware Fusion layer, and a Cascade Router over a shared CSR substrate to adapt retrieval per query. On LongMemEval, AgentIR’s confidence-triggered cascade skips dense on 63% of queries for a 2.67× latency reduction at parity LLM-judged accuracy, and on BEIR AgentIR’s BM25 core is up to 29× faster than Pyserini 8T at matching nDCG@10.
Benchmark
Agent Zero Memory: Provenance-Aware Long-Term Memory for LLM Agents
Ming Wu, Pengyuan Zhu
arXiv 2026 · 2026
Agent Zero Memory builds a triple store using a Memory Events timeline, Ontology Graph, Hierarchical Documentary Memory, and an Intent Gate with AgenticSearch to retrieve citation-locked evidence. On LongMemEval, Agent Zero Memory scores 95.60% accuracy, +0.73 over Mastra, and 93.60% on LoCoMo, +1.10 over Mem0.
Long-Term Memory
All-Mem: Agentic Lifelong Memory via Dynamic Topology Evolution
Can Lv, Heng Chang et al.
· 2026
All-Mem organizes long-term agent memory through Online/Offline Decoupling, Agentic Topology Consolidation, and Topology-Aware Retrieval over a topology-structured memory bank. On LoCoMo, All-Mem reaches 54.63 4o-J versus Mem0’s 48.91, and on LongMemEval-S All-Mem reaches 60.20 4o-J versus Mem0’s 55.80.
RAG
Are We Ready For An Agent-Native Memory System?
Wei Zhou, Xuanhe Zhou et al.
arXiv 2026 · 2026
Are We Ready For An Agent-Native Memory System? analyzes Memory Representation and Storage, Memory Extraction, Memory Retrieval and Routing, and Memory Maintenance across 12 real systems like Mem0, Zep, MemTree, LightMem, MemOS, MemoryOS, and A-MEM. The study’s main result is that structured systems such as Zep reach 48.0 LLM Judge Accuracy on LongMemEval while Long Context reaches 19.0, and that localized maintenance strategies like LightMem achieve 48.3% normalized utility at only 3.67 s per query.
BenchmarkAgent Memory
ATANT v1.1: Positioning Continuity Evaluation Against Memory, Long-Context, and Agentic-Memory Benchmarks
Samuel Sameer Tanguturi
· 2026
ATANT v1.1 structurally analyzes seven benchmarks using the 7 v1.0 continuity properties, the 10 checkpoints, a property-coverage matrix, and the Kenotic v1.0 reference implementation. ATANT v1.1 reports 96% ATANT cumulative-scale versus 8.8% LOCOMO substring accuracy, showing that LOCOMO, LongMemEval, BEAM, MemoryBench, Zep eval, MemGPT/Letta, and RULER measure different properties from continuity.
Agent Memory
Beyond Memory Majority: Latent-Source Reasoning for Multi-Agent Memory Arbitration
Chenchen Lin, Wenhao Yuan et al.
arXiv 2026 · 2026
CAMA combines Neuro-Symbolic Evidence Assignment, Effective Independent Evidence Estimation, Factor-Level Conflict Arbitration, and Active Independent-Evidence Recovery to decouple correlated memories into latent evidence slots and recover missing sources. On LongMemEval with DeepSeek V4 Flash, CAMA reaches 87.9 EM compared to MADAM RAG’s 86.8 EM while also cutting Replication Sensitivity from 15.3 to 7.8.