AuthorsYihao Wang, Haoran Xu, Renjie Gu et al.
TL;DR
MedMemoryBench uses a streaming evaluate while constructing protocol with noise augmented trajectories to reveal memory saturation, where accuracy on IG drops from 43.60% to 25.60% under Mixed noise.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
MedMemoryBench shows that when Mixed noise is injected, IG accuracy for Unstructured Long Context falls from 43.60% to 25.60%, and MCD from 24.50% to 10.25%.
In personalized healthcare, this memory saturation means agents mis handle safety critical personalization, degrading complex reasoning exactly when long term tracking is most needed.
HOW IT WORKS
MedMemoryBench centers on Patient Profile Construction, Disease Progression Event Generation, Multi turn Sessions Simulation, and Memory Extraction and Query Construction to create year long, clinically grounded trajectories.
You can think of MedMemoryBench like a hospital wide memory stress test, where patient histories accumulate like medical charts while noisy notes and family consultations clutter the file.
This design lets MedMemoryBench expose memory saturation and retrieval failures that a plain context window or static benchmark cannot, especially under streaming evaluate while constructing conditions.
DIAGRAM
This diagram shows how MedMemoryBench feeds chronological sessions into memory methods and evaluates at streaming checkpoints without future leakage.
DIAGRAM
This diagram shows how MedMemoryBench constructs patient trajectories, simulates sessions, extracts memories, and injects noise for Efficient and Mixed modes.
PROCESS
Patient Profile Construction
MedMemoryBench builds 20 chronic disease Patient Profile Construction cases from de identified real scenarios, expanding family background and lifestyle with LLM assistance.
Disease Progression Event Generation
MedMemoryBench performs Disease Progression Event Generation to create expert verified one year diagnostic reports and a temporal event graph with injected trap events.
Multi turn Sessions Simulation
MedMemoryBench runs Multi turn Sessions Simulation where patient and physician agents converse along the event graph, recording summaries as accumulated contextual memory.
Memory Extraction and Query Construction
MedMemoryBench applies Memory Extraction and Query Construction with Trap Agent, Query Agent, and Critique Agent, then evaluates memory methods using the streaming STREAMEVAL protocol.
KEY CONTRIBUTIONS
MedMemoryBench benchmark for personalized healthcare
MedMemoryBench introduces a four stage pipeline with Patient Profile Construction, Disease Progression Event Generation, Multi turn Sessions Simulation, and Memory Extraction and Query Construction, yielding about 2,000 sessions and 16,000 turns.
Streaming evaluate while constructing protocol
MedMemoryBench defines STREAMEVAL with checkpoints every Δ=10 sessions, enforcing src(q) session ids ≤ t to avoid future leakage while assessing evolving memory states.
First study of memory saturation in healthcare agents
MedMemoryBench adds Noise Augmentation with general health consultations and proxy family sessions, showing IG accuracy drops from 43.60% to 25.60% and MCD from 24.50% to 10.25% under Mixed noise.
RESULTS
Avg. accuracy Efficient
51.21%
+9.63 over BM25 Efficient (41.58% vs 34.70%)
Avg. accuracy Mixed
41.55%
+12.69 over BM25 Mixed (41.55% vs 28.86%)
IG Efficient Letta
52.87%
+31.71 over BM25 Efficient IG (52.87% vs 21.16%)
MCD Efficient A Mem
31.93%
+13.08 over BM25 Efficient MCD (31.93% vs 18.85%)
MedMemoryBench reports accuracy across six query types under Efficient and Mixed modes, testing retrieval, temporal updates, and reasoning. These numbers show how MedMemoryBench exposes large drops under noise and highlights methods like Letta and A Mem on specific tasks.
BENCHMARK
MedMemoryBench reports accuracy across six query types under Efficient and Mixed modes, testing retrieval, temporal updates, and reasoning. These numbers show how MedMemoryBench exposes large drops under noise and highlights methods like Letta and A Mem on specific tasks.
BENCHMARK
Average response accuracy (%) across six query types on MedMemoryBench Efficient.
BENCHMARK
Average response accuracy (%) across six query types on MedMemoryBench Mixed with noise.
KEY INSIGHT
MedMemoryBench shows that adding more memory through Mixed noise consistently reduces performance, for example IG accuracy 43.60% → 25.60% and MCD 24.50% → 10.25% for Unstructured Long Context.
This is counterintuitive because developers expect larger memory stores to help, but MedMemoryBench demonstrates that redundant, homogeneous entries trigger memory saturation and harm reasoning robustness.
WHY IT MATTERS
MedMemoryBench gives researchers a medically grounded way to stress test memory mechanisms under streaming accumulation, trap events, and realistic noise, rather than static, clean histories.
Builders can now design and compare memory systems that explicitly manage saturation, retrieval granularity, and safety critical personalization, moving toward production ready healthcare agents with reliable long term memory.
Peilin Feng, Suorong Yang, Soujanya Poria
arXiv 2026 · 2026
Σ-Mem maintains historical competence evidence matrices M_p, a peer relationship matrix G, a memory readout r_{p,t}, and residual steering δ_{p,t} to guide multi-agent trust decisions. On the mixed counterfactual benchmark, Σ-Mem raises Qwen3-0.6B accuracy at CF@90 from 46.22% with the base model to 71.10% with Σ w/ G.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xiaohui Zhang, Zequn Sun et al.
· 2026
ActMem transforms dialogue history into atomic facts via Memory Fact Extraction, groups them with Fact Clustering, links them through a Memory KG Construction module, and uses Counterfactual-based Retrieval and Reasoning for action-aware answers. On ActMemEval, ActMem reaches 76.52% QA accuracy with DeepSeek-V3, beating LightMem’s 63.97% by 12.55 points and NaiveRAG’s 61.54%.
Answers use this explainer on Memory Papers.
Checking…