AuthorsZhewen Tan, Yilun Yao, Huiyan Jin et al.
TL;DR
MemAudit uses counterfactual memory influence plus a global consistency graph to audit poisoned agent memory, cutting GPT-4o QA attack success from 70.0% to 0.0%.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Memory-augmented agents under MINJA see QA attack success reach 70.0% on GPT-4o and 83.3% on RAP before any auditing.
These poisoned memories steer LLM agents toward harmful outputs, so unsafe behavior persists across sessions and can corrupt long-horizon reasoning trajectories.
HOW IT WORKS
MemAudit combines Counterfactual Memory Influence Score, a global Memory Consistency Graph, and a fused Detoxification Score computed via Algorithm 1 to rank suspicious memories.
You can think of MemAudit as a security-grade RAM scrubber plus a library card catalog that flags books whose content contradicts the rest of the shelf.
This dual-signal design lets MemAudit repair poisoned agent memory post-hoc, something a plain context window or online filter cannot do once harmful traces are already stored.
DIAGRAM
This diagram shows how MemAudit replays harmful events to compute Counterfactual Memory Influence Scores and aggregate them across events.
DIAGRAM
This diagram shows how MemAudit is evaluated under different contamination ratios using the three stage post hoc batch auditing protocol.
PROCESS
Post hoc batch auditing protocol
MemAudit first runs the post hoc batch auditing protocol, replaying the attacked agent to collect harmful events E with queries, outputs, and retrieved memories.
Counterfactual Memory Influence Score
MemAudit computes Counterfactual Memory Influence Score by removing each memory mi from M, replaying the event, and measuring harm reduction via the harm scorer h.
Memory Consistency Graph
MemAudit builds the Memory Consistency Graph G over all memories, using semantic similarity and DeBERTa v3 natural language inference to derive the structural anomaly score CAS(mi).
Detoxification Score and removal set S
MemAudit fuses normalized CMIS and CAS into the Detoxification Score DS(mi), ranks memories, and selects the top ranked set S for removal to detoxify the agent.
KEY CONTRIBUTIONS
Post hoc causal memory auditing formulation
MemAudit is the first to formulate post hoc causal memory auditing for interaction induced poisoning, targeting HarmAfter(M \ S) minimization after failures in MINJA QA and RAP.
Dual signal MemAudit framework
MemAudit introduces Counterfactual Memory Influence Score and Memory Consistency Graph, fused into a Detoxification Score that removes memories and drops GPT-4o QA ASR from 70.0% to 0.0%.
Operating boundary under contamination ratio
MemAudit analyzes contamination ratio ρ, showing full recovery up to ρ = 0.20 on QA and ρ = 0.15 on RAP, but sharp degradation once poisoned memories become dense and mutually reinforcing.
RESULTS
ASRbefore
70.0%
+20.0 over GPT-4o-mini QA baseline ASRbefore 50.0%
ASRafter
0.0%
-70.0 vs GPT-4o QA ASRbefore 70.0%
ASRafter
10.0%
DeepSeek QA after MemAudit vs 70.0% ASRbefore
ASRafter
0.0%
RAP GPT-4o after MemAudit vs 83.3% ASRbefore
Table 1 reports attack success rate on MINJA QA and RAP for GPT-4o, GPT-4o-mini, and DeepSeek. MemAudit’s MAIN_RESULT shows that targeted memory removal can completely eliminate attacks on GPT-4o and GPT-4o-mini and nearly eliminate them on DeepSeek QA.
BENCHMARK
Table 1 reports attack success rate on MINJA QA and RAP for GPT-4o, GPT-4o-mini, and DeepSeek. MemAudit’s MAIN_RESULT shows that targeted memory removal can completely eliminate attacks on GPT-4o and GPT-4o-mini and nearly eliminate them on DeepSeek QA.
BENCHMARK
Attack Success Rate after auditing on MINJA QA and RAP for GPT-4o with different deletion strategies.
BENCHMARK
Attack Success Rate after auditing on MINJA QA GPT-4o for CMISonly, MCGonly, and MemAudit.
KEY INSIGHT
MemAudit drives GPT-4o QA attack success from 70.0% to 0.0%, while random deletion actually increases ASRafter to 74.0%.
It is surprising that deleting memories naively can worsen attacks, showing that poisoned traces are structurally entangled and require causal plus structural auditing rather than simple pruning.
WHY IT MATTERS
MemAudit unlocks practical post hoc repair of memory augmented agents, letting auditors identify and remove harmful memories without poison labels or online control.
Builders can now treat persistent memory as a first class security surface, adding MemAudit style replay and consistency graphs to safely detoxify long lived agents after deployment failures.
Peilin Feng, Suorong Yang, Soujanya Poria
arXiv 2026 · 2026
Σ-Mem maintains historical competence evidence matrices M_p, a peer relationship matrix G, a memory readout r_{p,t}, and residual steering δ_{p,t} to guide multi-agent trust decisions. On the mixed counterfactual benchmark, Σ-Mem raises Qwen3-0.6B accuracy at CF@90 from 46.22% with the base model to 71.10% with Σ w/ G.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xiaohui Zhang, Zequn Sun et al.
· 2026
ActMem transforms dialogue history into atomic facts via Memory Fact Extraction, groups them with Fact Clustering, links them through a Memory KG Construction module, and uses Counterfactual-based Retrieval and Reasoning for action-aware answers. On ActMemEval, ActMem reaches 76.52% QA accuracy with DeepSeek-V3, beating LightMem’s 63.97% by 12.55 points and NaiveRAG’s 61.54%.
Answers use this explainer on Memory Papers.
Checking…