AuthorsYe Shen, Dun Pei, Yiqiu Guo et al.
TL;DR
EvolMem uses a hybrid dialogue synthesis framework grounded in cognitive memory types to show that Gemini-3-Pro reaches 67.47% overall memory performance while no LLM dominates all seven abilities.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Existing long context benchmarks mostly test single turn recall and declarative memory, neglecting non declarative systems that support habits and implicit knowledge.
This leaves LLMs and agent memory systems under evaluated on multi session dialogue tasks, where missing habits and learning harms long term coherence, user preference retention, and procedural adaptation.
HOW IT WORKS
EvolMem centers on Topic-Initiated Generation, Narrative-Inspired Transformation, Filtering, and Challenge Injection to synthesize multi session dialogues aligned with seven cognitive memory abilities.
Think of EvolMem as a lab where long texts are sliced like chapters, then re staged into conversations, while a curator injects conflicts and removes easy cases.
This KEY_MECHANISM lets EvolMem stress memory evolution, non declarative learning, and habituation in ways a plain context window or static long text benchmark cannot.
DIAGRAM
This diagram shows how EvolMem organizes cognitive memory into declarative and non declarative types with seven fine grained abilities.
DIAGRAM
This diagram shows how EvolMem evaluates model answers using result oriented, process oriented, and open ended metrics.
PROCESS
Topic Initiated Generation
EvolMem uses Topic-Initiated Generation with seed task categories and topics to call fLLM under RPLA, producing structured dialogue overviews with sessions, turns, final query, reference answer, and evaluation checklist.
Narrative Inspired Transformation
EvolMem applies Narrative-Inspired Transformation to long text benchmarks like Variable Tracking in Ruler, segmenting paragraphs into sessions and turns, then generating human like agent responses via fLLM under RNIT.
Filtering
EvolMem runs DeepSeek-V3.2 through its evaluation pipeline on generated dialogues and filters out all samples with scores greater than or equal to 0.8, retaining only challenging and discriminative memory cases.
Challenge Injection
EvolMem performs Challenge Injection by inserting plausible but antithetical information and integrating high difficulty benchmarks, forcing models to handle non linear reasoning, conflicts, and long range dependencies.
KEY CONTRIBUTIONS
Cognitive driven memory benchmark
EvolMem formalizes memory into declarative and non declarative types with seven abilities including Retrieval, Summarization, Isolation, Inference, Reproduction, Learning, and Habituation, enabling fine grained evaluation.
Multi source data synthesis framework
EvolMem introduces Topic-Initiated Generation and Narrative-Inspired Transformation plus Filtering and Challenge Injection to build 1,600 multi session dialogues averaging 6.82 sessions and 29.49 turns.
Systematic evaluation of LLMs and agents
EvolMem shows Gemini-3-Pro reaches 67.47% overall memory performance while MemoryOS, Mem0, HippoRAG, and A-MEM all fail to surpass their DeepSeek-V3.2 base, revealing weaknesses in non declarative memory and efficiency.
RESULTS
Overall memory
67.47%
+14.94 over Llama-4-Maverick
Habituation ability
84.16%
+69.37 over GPT-5.1
Retrieval ability
58.37%
-22.74 vs Mistral-Large
Agent speed Mem0
30.17s
baseline vs A-MEM at 492.60s
These numbers come from EvolMem’s main LLM and agent tables, which test seven memory abilities over 1,600 multi session dialogues. The results show EvolMem can separate strengths like Gemini-3-Pro’s 84.16% habituation from weaknesses such as GPT-5.1’s 14.79% habituation and A-MEM’s 16.3× latency over Mem0.
BENCHMARK
These numbers come from EvolMem’s main LLM and agent tables, which test seven memory abilities over 1,600 multi session dialogues. The results show EvolMem can separate strengths like Gemini-3-Pro’s 84.16% habituation from weaknesses such as GPT-5.1’s 14.79% habituation and A-MEM’s 16.3× latency over Mem0.
BENCHMARK
Overall averaged memory score across declarative and non declarative abilities.
KEY INSIGHT
EvolMem reveals that MemoryOS, Mem0, HippoRAG, and A-MEM all score below their DeepSeek-V3.2 base, with MemoryOS at 48.66% overall versus 52.18%.
This is surprising because agent memory mechanisms are designed to enhance long term reasoning, yet EvolMem shows they can reduce effectiveness while adding latency, challenging assumptions about agentic memory benefits.
WHY IT MATTERS
EvolMem unlocks a way to stress test LLMs and agents on realistic multi session memory, including non declarative learning and habituation that prior benchmarks ignored.
Builders can now diagnose specific memory weaknesses, tune agent memory mechanisms, and track evolving performance as sessions and difficulty increase, rather than relying on static long context scores.
Peilin Feng, Suorong Yang, Soujanya Poria
arXiv 2026 · 2026
Σ-Mem maintains historical competence evidence matrices M_p, a peer relationship matrix G, a memory readout r_{p,t}, and residual steering δ_{p,t} to guide multi-agent trust decisions. On the mixed counterfactual benchmark, Σ-Mem raises Qwen3-0.6B accuracy at CF@90 from 46.22% with the base model to 71.10% with Σ w/ G.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xiaohui Zhang, Zequn Sun et al.
· 2026
ActMem transforms dialogue history into atomic facts via Memory Fact Extraction, groups them with Fact Clustering, links them through a Memory KG Construction module, and uses Counterfactual-based Retrieval and Reasoning for action-aware answers. On ActMemEval, ActMem reaches 76.52% QA accuracy with DeepSeek-V3, beating LightMem’s 63.97% by 12.55 points and NaiveRAG’s 61.54%.
Answers use this explainer on Memory Papers.
Checking…