AuthorsChen Shen
TL;DR
What Eviction Destroys uses a restore counterfactual over agent memory to decompose eviction errors, showing irreversible loss reaches 1.00 at an 8k-token budget.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Agent memory systems must discard stored information when their history exceeds a fixed token budget, but budget–accuracy frontiers conflate distinct failure modes. Under an 8k-token budget on LongMemEval-S, the irreversible share among errors corrected by restoration reaches 1.00 for all four eviction policies.
This means a capacity-bounded eviction policy can permanently destroy gold evidence, leaving questions unanswerable even with better retrieval. For multi-session conversational agents, this destruction directly reduces end-task accuracy in ways standard evaluations cannot see.
HOW IT WORKS
What Eviction Destroys centers on the restore counterfactual, the recoverable/irreversible/residual decomposition, and a controlled LongMemEval-S study of four eviction policies. These components operate over a capacity-bounded external store and a frozen GPT-4o-mini reader and judge.
You can think of What Eviction Destroys like testing RAM under memory pressure: it removes entries, then temporarily plugs the exact needed card back into the catalog to see if the computation recovers. This is a per-question, leave-one-out intervention that isolates what eviction destroyed versus what retrieval missed.
By reinstating benchmark-labeled gold evidence and re-answering each question, What Eviction Destroys exposes irreversible losses that a plain context window or aggregate accuracy cannot distinguish from recoverable retrieval failures.
DIAGRAM
This diagram shows how What Eviction Destroys runs the per-question restore counterfactual to classify each oracle-answerable error.
DIAGRAM
This diagram shows how What Eviction Destroys evaluates eviction policies across budgets and retrieval regimes on LongMemEval-S.
PROCESS
The restore counterfactual metric
What Eviction Destroys first defines restore gain(q) by answering each question with the post-eviction store and then with full gold evidence restored.
The recoverability decomposition
Using restore gain and whether gold evidence was retained, What Eviction Destroys classifies each oracle-answerable error as recoverable, irreversible, or residual.
Experimental setup
What Eviction Destroys applies this decomposition to FIFO, random, redundancy-aware, LLM-importance, and a destructive control across 8k, 30k, and 80k budgets.
Results and matched accuracy analysis
What Eviction Destroys reports irreversible shares and runs a matched-accuracy analysis, finding no dissociation among policies while detecting the destructive control.
KEY CONTRIBUTIONS
The restore counterfactual metric
What Eviction Destroys introduces a per-question restore counterfactual over external agent-memory stores, using GPT-4o-mini to measure restore gain(q) ∈ {−1,0,1}.
Recoverable irreversible residual decomposition
What Eviction Destroys defines a three-bin decomposition on an oracle-answerable denominator, separating destruction, retrieval misses, and utilization failures for each policy.
Controlled LongMemEval S eviction study
What Eviction Destroys runs a controlled study of four eviction policies and a destructive control, showing irreversible shares of 0.67–0.73 at 80k and 1.00 at 8k tokens.
RESULTS
Irreversible share 80k FIFO
0.71 share
+0.71 over no-evict irreversible share 0.00
Irreversible share 80k random
0.73 share
+0.73 over no-evict irreversible share 0.00
Irreversible share 80k redundancy
0.67 share
+0.67 over no-evict irreversible share 0.00
Irreversible share 80k LLM importance
0.60 share
+0.60 over no-evict irreversible share 0.00
On LongMemEval-S under the realistic top-k regime, What Eviction Destroys shows that destruction dominates eviction loss, with irreversible shares above 0.5 at 80k tokens for all baseline policies. This proves that budget–accuracy frontiers hide large irreversible components unless retrieval regime and restore-based decomposition are reported.
BENCHMARK
On LongMemEval-S under the realistic top-k regime, What Eviction Destroys shows that destruction dominates eviction loss, with irreversible shares above 0.5 at 80k tokens for all baseline policies. This proves that budget–accuracy frontiers hide large irreversible components unless retrieval regime and restore-based decomposition are reported.
BENCHMARK
Irreversible share irr/(irr+rec) for eviction policies on LongMemEval-S at an 80k-token budget.
KEY INSIGHT
At an 80k-token budget under top-k retrieval, irreversible shares among corrected errors are 0.67–0.73 for FIFO, random, and redundancy-aware eviction. Even the LLM-importance policy still has an irreversible share of 0.60, while at 8k tokens all four policies reach an irreversible share of 1.00.
This is surprising because many memory systems assume retrieval is the main bottleneck, yet What Eviction Destroys shows that under realistic budgets, destruction dominates and better retrieval alone cannot recover most lost accuracy.
WHY IT MATTERS
What Eviction Destroys unlocks a way to audit agent memory by separating what eviction destroyed, what retrieval missed, and what the reader failed to use. Developers can now quantify how much task loss is truly irreversible for each eviction policy and budget.
This lets builders design memory systems and privacy-driven forgetting policies with explicit awareness of irreversible loss, making budget–accuracy frontiers and retention strategies comparable only when paired with a restore-counterfactual decomposition.
Peilin Feng, Suorong Yang, Soujanya Poria
arXiv 2026 · 2026
Σ-Mem maintains historical competence evidence matrices M_p, a peer relationship matrix G, a memory readout r_{p,t}, and residual steering δ_{p,t} to guide multi-agent trust decisions. On the mixed counterfactual benchmark, Σ-Mem raises Qwen3-0.6B accuracy at CF@90 from 46.22% with the base model to 71.10% with Σ w/ G.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xiaohui Zhang, Zequn Sun et al.
· 2026
ActMem transforms dialogue history into atomic facts via Memory Fact Extraction, groups them with Fact Clustering, links them through a Memory KG Construction module, and uses Counterfactual-based Retrieval and Reasoning for action-aware answers. On ActMemEval, ActMem reaches 76.52% QA accuracy with DeepSeek-V3, beating LightMem’s 63.97% by 12.55 points and NaiveRAG’s 61.54%.
Answers use this explainer on Memory Papers.
Checking…