Evaluation map
Compare the benchmarks used to evaluate conversational, long-context, and agent memory systems, with links to their original papers and datasets.
Compare the field
| Benchmark | Best for | Evaluation | Memory horizon | Released |
|---|---|---|---|---|
| DolphinBench | Long-horizon memory-guided tool use | Tool-using agent tasks in simulated apps | Nearly five years of conversation history | 2026 |
| LoCoMo | Long-term conversational recall and reasoning | Conversation QA and generation | Up to 35 sessions | 2024 |
| BEAM | Extreme-length conversational memory | Long-context conversation QA | Up to 10M tokens | 2025 |
| MemoryArena | Memory-guided agent actions | Interactive agent tasks | Interdependent multi-session tasks | 2026 |
| LongMemEval | Long-term assistant memory | Multi-session conversation QA | Scalable histories; 500 questions | 2024 |
| MemBench | Multi-dimensional agent-memory evaluation | Agent interaction scenarios | Factual and reflective memory | 2025 |
| MemoryBench | Learning continuously from user feedback | Simulated service-time feedback | Stepwise continual learning | 2025 |
Benchmark atlas
Benchmark
An open benchmark for long-term agent memory that evaluates whether a system uses years of conversation history to take the correct tool action, while also reporting cost and latency.
600 verified tasks across three simulated users with roughly 500K tokens of message history per persona.
Explore DolphinBench →Official project ↗Benchmark
A dataset and evaluation benchmark for very long-term conversational memory across question answering, event summarization, and multimodal dialogue generation.
Very long conversations averaging 300 turns and 9K tokens across as many as 35 sessions.
Explore LoCoMo →Official project ↗Benchmark
A long-context conversational-memory benchmark with coherent conversations reaching 10M tokens and questions spanning multiple memory abilities.
100 conversations, up to 10M tokens each, with 2,000 human-validated questions.
Explore BEAM →Official project ↗Benchmark
An evaluation gym for agent memory in interdependent multi-session tasks where agents must learn from earlier actions and reuse that memory later.
Human-authored, interdependent task sequences evaluated through Memory-Agent-Environment loops.
Explore MemoryArena →Benchmark
A scalable benchmark for long-term interactive memory in chat assistants, covering information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention.
500 curated questions across five long-term memory abilities, with histories that can be scaled in length.
Explore LongMemEval →Official project ↗Benchmark
A benchmark for LLM-agent memory across factual and reflective memory, participation and observation scenarios, and effectiveness, efficiency, and capacity dimensions.
Multiple memory levels, interaction scenarios, and evaluation dimensions in one agent-memory suite.
Explore MemBench →Benchmark
A benchmark for memory and continual learning in LLM systems that tests declarative and procedural memory learned from explicit and implicit user feedback.
28 benchmarks across three domains, four task formats, and two languages, with eight published baselines.
Explore MemoryBench →Official project ↗How to use this atlas
Use conversational QA benchmarks for recall, interactive suites for tool-using agents, and feedback benchmarks for continual learning.
Model, judge, retrieval depth, prompt, data version, and cost accounting can make apparently similar scores incomparable.
Every atlas page names what the benchmark does not test, so a leaderboard result is not mistaken for production readiness.