AuthorsXuanze Chen, Xukang Xie, Wentao Fu et al.
TL;DR
MemSecBench tracks malicious agent memories across Write–Execute–Forget checkpoints, showing 84.2% poisoning and 50.3% end to end attacks with lifecycle-aware evidence gates.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
MemSecBench shows that across 24 agent configurations, malicious memory persists in 84.2% of all benchmark cases, even after Write filtering.
These poisoned memories later drive unsafe actions, with 50.3% of all cases completing the full Write–Execute chain and producing a verified external consequence.
HOW IT WORKS
MemSecBench centers on Lifecycle Task Packages, the Build-MemSecBench-Case Skill, a Lifecycle Evaluation Workflow, and Evidence-Based LLM Judging to track malicious semantics end to end.
You can think of MemSecBench like a security lab where each agent configuration is a different machine, and the benchmark injects, traces, and repairs toxic memory across its internal storage.
This lifecycle design lets MemSecBench expose how memory backends differ from a plain context window, revealing poisoning, downstream consequences, and selective repair under matched harness and model settings.
DIAGRAM
This diagram shows how MemSecBench models attacker content entering in one session and later causing consequences and repair in another.
DIAGRAM
This diagram shows how MemSecBench evaluates 2×4×3 configurations across Write, Execute, and Forget checkpoints.
PROCESS
Lifecycle Task Packages
MemSecBench builds Lifecycle Task Packages that define target malicious semantics, benign memories, and Write, Execute, Forget tasks for each case.
Lifecycle Evaluation Workflow
The Lifecycle Evaluation Workflow initializes clean memory, loads stage inputs, and runs Write, Execute, and Forget under a pinned agent configuration.
Evidence-Based LLM Judging
Evidence-Based LLM Judging uses DeepSeek-V4-Pro and checkpoint-specific rubrics to adjudicate W2, E1–E3, and F1–F2 from admissible evidence packs.
Memory Poisoning and Selective Repair Metrics
MemSecBench aggregates checkpoint outcomes into Memory Poisoning Success Rate, Memory Exploitation Success Rate, End-to-End Attack Success Rate, and Selective Repair Success Rate.
KEY CONTRIBUTIONS
Memory Security Benchmark
MemSecBench introduces 310 Write–Execute–Forget Lifecycle Task Packages across 48 contexts, enabling end to end tracking of malicious semantics through seven checkpoints.
Lifecycle Evaluation Framework
MemSecBench’s Lifecycle Evaluation Workflow and Evidence-Based LLM Judging support controlled comparisons across 2 agent harnesses, 4 memory backends, and 3 LLM backends.
Agent Memory Risks
MemSecBench reveals pervasive vulnerabilities, with malicious memory persisting in 84.2% of cases and full Write–Execute attacks succeeding in 50.3% across all configurations.
RESULTS
Memory Poisoning Success Rate
84.2% macro average
shows high poisoning persistence across 24 configurations
End-to-End Attack Success Rate
50.3% macro average
half of all cases complete Write–Execute chain
Selective Repair Success Rate
56.1% macro average
conditional on successful poisoning across configurations
Judge Model Accuracy
91.80% label match
DeepSeek-V4-Pro vs human annotations on 500 records
MemSecBench evaluates 310 cases on a 2×4×3 configuration matrix, measuring lifecycle security rather than task accuracy. These numbers show that MemSecBench exposes both high attack success and only moderate selective repair capability in current agent memory stacks.
BENCHMARK
MemSecBench evaluates 310 cases on a 2×4×3 configuration matrix, measuring lifecycle security rather than task accuracy. These numbers show that MemSecBench exposes both high attack success and only moderate selective repair capability in current agent memory stacks.
BENCHMARK
Attack and repair rates (MPSR, MESR, E2E-ASR, SRSR) for MemSecBench under OpenClaw with different LLM backends.
KEY INSIGHT
MemSecBench finds that adoption, not persistence or recall, is the main bottleneck: E2 drops to 53.7% while E1 remains at 76.1%.
This is surprising because many defenses focus on blocking writes or retrieval, yet MemSecBench shows that once malicious content is adopted, 50.3% of cases complete the harmful chain.
WHY IT MATTERS
MemSecBench gives practitioners a way to quantify memory poisoning risk and selective repair across full agent configurations, not just isolated LLM prompts.
Builders can now compare Native, Mem0, Mem0-Graph, and A-MEM under identical tasks, designing backend-specific defenses that target adoption and benign-memory preservation rather than only write filters.
Peilin Feng, Suorong Yang, Soujanya Poria
arXiv 2026 · 2026
Σ-Mem maintains historical competence evidence matrices M_p, a peer relationship matrix G, a memory readout r_{p,t}, and residual steering δ_{p,t} to guide multi-agent trust decisions. On the mixed counterfactual benchmark, Σ-Mem raises Qwen3-0.6B accuracy at CF@90 from 46.22% with the base model to 71.10% with Σ w/ G.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xiaohui Zhang, Zequn Sun et al.
· 2026
ActMem transforms dialogue history into atomic facts via Memory Fact Extraction, groups them with Fact Clustering, links them through a Memory KG Construction module, and uses Counterfactual-based Retrieval and Reasoning for action-aware answers. On ActMemEval, ActMem reaches 76.52% QA accuracy with DeepSeek-V3, beating LightMem’s 63.97% by 12.55 points and NaiveRAG’s 61.54%.
Answers use this explainer on Memory Papers.
Checking…