AuthorsWeiwei Xie, Shaoxiong Guo, Fan Zhang et al.
TL;DR
MemEvoBench uses multi-round biased-memory interactions plus a memory correction tool to show ASR often exceeds 80% and that active memory editing can cut QA-style risks for Gemini-2.5-Pro from 67.0% to 19.0% in Round 1.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
MemEvoBench shows that under Vanilla prompting, average Attack Success Rate reaches 75.9% in Round 1 and climbs to 87.8% by Round 3 with biased feedback.
In long-horizon QA and workflow tasks, contaminated memory causes behavioral drift, leading to unsafe recommendations and security violations even when intrinsic model knowledge is adequate.
HOW IT WORKS
MemEvoBench centers on Misleading Memory Injection, Noisy Tool Returns, Biased User Feedback, and a Memory Modification Tool (+ModTool) to simulate memory misevolution in realistic agents.
You can think of MemEvoBench as stress-testing an agent’s long-term memory like a corrupted hard drive that keeps getting reinforced by a biased user clicking “like” on unsafe behaviors.
This design lets MemEvoBench expose how cumulative memory drift and biased reinforcement create stable unsafe reasoning paths that a plain context window or static safety prompt cannot reveal.
DIAGRAM
This diagram shows how MemEvoBench runs three related rounds per case, appending responses and biased feedback into memory to simulate misevolution.
DIAGRAM
This diagram shows how MemEvoBench builds contaminated memory pools, runs QA and workflow tasks, and computes ASR with GPT-5.2 judging.
PROCESS
Misleading Memory Injection
MemEvoBench first constructs QA Style cases by mixing correct and misleading entries into a hybrid memory pool for 7 domains and 36 risk types.
Noisy Tool Returns
MemEvoBench then builds Workflow Style cases from 20 AgentSafetyBench environments, encoding correct and contaminated workflows as procedural memories.
Memory Evolution and Biased Feedback
Across three rounds, MemEvoBench appends each agent response and simulated biased user feedback into the memory pool to model misevolution.
Evaluation and Memory Ablation
MemEvoBench computes ASR with GPT-5.2 judging, and compares runs with no memory, initial memory, and A-MEM to isolate memory-driven safety failures.
KEY CONTRIBUTIONS
MemEvoBench benchmark for contaminated memory evolution
MemEvoBench introduces QA Style and Workflow Style scenarios spanning 7 domains, 36 risk types, and 20 tool environments, with 108 QA and 83 workflow test cases.
Revealing evolutionary dynamics of memory misevolution
MemEvoBench shows average Vanilla ASR of 75.9% in Round 1 rising to 87.8% by Round 3 with biased feedback, linking accumulated memory drift to safety failures.
Hybrid defense with external knowledge and ModTool
MemEvoBench equips agents with a correct_memory tool and web_search, showing Gemini-2.5-Pro QA ASR drops from 67.0% to 19.0% in Round 1 under +ModTool.
RESULTS
QA Style ASR Rd.1 Vanilla avg
75.9%
+20.4 percentage points over QA Style ASR Rd.1 +SafePrompt avg 55.5%
QA Style ASR Rd.3 GPT 5 w/feedback Vanilla
78.0%
vs GPT 5 QA Style ASR Rd.3 Vanilla 59.0% without feedback
QA Style ASR Rd.1 Gemini 2.5 Pro
67.0%
drops by 48.0 percentage points to 19.0% with +ModTool
Workflow Style ASR Rd.1 Gemini 2.5 Pro
74.7%
reduced to 54.2% with +ModTool under the same round
MemEvoBench evaluates ASR on QA Style (108 cases) and Workflow Style (83 cases), showing how contaminated memory and biased feedback drive high attack success and how +ModTool can sharply reduce QA risks for some models.
BENCHMARK
MemEvoBench evaluates ASR on QA Style (108 cases) and Workflow Style (83 cases), showing how contaminated memory and biased feedback drive high attack success and how +ModTool can sharply reduce QA risks for some models.
BENCHMARK
Attack Success Rate (%) on MemEvoBench QA Style Round 1 for Gemini-2.5-Pro under different memory safety configurations.
KEY INSIGHT
MemEvoBench shows that adding biased user feedback can push average ASR from 71.6% in Round 1 to 87.8% in Round 3, even with strong base models.
This is surprising because many assume better intrinsic knowledge guarantees safety, yet MemEvoBench reveals memory evolution and feedback loops can override that knowledge and stabilize unsafe behavior.
WHY IT MATTERS
MemEvoBench gives researchers a concrete way to quantify how long-term memory, noisy tools, and biased feedback jointly degrade safety across realistic agent deployments.
With MemEvoBench, builders can now design and test active memory defenses like +ModTool and A-MEM integration, targeting memory dynamics directly instead of relying solely on static safety prompts.
Guilin Zhang, Wei Jiang et al.
· 2026
A-MAC scores candidate memories using Utility, Confidence, Novelty, Recency, and Type Prior combined by a learned linear admission policy with Algorithm 1 A-MAC Memory Admission. On the LoCoMo benchmark, A-MAC achieves F1 0.583 and 2644 ms latency, improving F1 by 0.042 and reducing latency by 1187 ms compared to A-mem.
Yi Yu, Liuyi Yao et al.
arXiv 2026 · 2026
Agentic Memory (AgeMem) exposes memory management tools, a three-stage progressive RL strategy, and step-wise GRPO directly inside the agent policy to jointly control long-term and short-term memory. On Qwen3-4B-Instruct, AgeMem attains 54.31% average performance across ALFWorld, SciWorld, PDDL, BabyAI, and HotpotQA, exceeding the best baseline A-Mem at 45.74%.
Can Lv, Heng Chang et al.
· 2026
All-Mem organizes long-term agent memory through Online/Offline Decoupling, Agentic Topology Consolidation, and Topology-Aware Retrieval over a topology-structured memory bank. On LoCoMo, All-Mem reaches 54.63 4o-J versus Mem0’s 48.91, and on LongMemEval-S All-Mem reaches 60.20 4o-J versus Mem0’s 55.80.
Answers use this explainer on Memory Papers.
Checking…