AuthorsJundong Hu, Shekar Ramachandran
TL;DR
The Memory Trust Gap uses a frozen Benefit/Safety benchmark and a 4-feature factorial to show stale memories are over-trusted (reliance up to 1.00, ∆mem down to −1.00) in a capability-dependent way.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Persistent-memory agents can treat a stale stored fact as authoritative, overriding current evidence; in the Benefit suite, stale-value reliance is 0.92–1.00 across 0.6–8B.
When this happens in the Safety suite, a stale memory can drag accuracy below the no-memory baseline, with ∆mem as low as −1.00 once the agent is capable enough.
HOW IT WORKS
The Memory Trust Gap combines a Benefit suite, a Safety suite, a trap sweep, and a 2×2×2×2 memory feature factorial to isolate when stale memory harms persistent agents.
Think of the Benefit and Safety suites as two test tracks: one where memory is required to solve the task, and one where an authoritative tool is like a trusted disk that should override any stale notes.
This design lets The Memory Trust Gap expose capability-dependent failures that a plain context window or single-shot benchmark cannot, by measuring both stale-value reliance and paired net harm ∆mem against a no-memory baseline.
DIAGRAM
This diagram shows how The Memory Trust Gap routes a Safety-suite scenario through no-memory, clean, stale, and explicit_conflict conditions to measure net harm ∆mem.
DIAGRAM
This diagram shows how The Memory Trust Gap runs trap levels and a 4-feature factorial across Qwen3 0.6B–8B to compute reliance and ∆mem.
PROCESS
Benchmark and 2 Suites
The Memory Trust Gap defines a frozen benchmark with a Benefit suite and a Safety suite, each with 150 base scenarios and SHA-256 pinning.
Conditions and Scoring
The Memory Trust Gap applies four conditions (no_memory, clean, stale, explicit_conflict) and scores constrained actions via exact or regex match without any LLM judge.
Trap Sweep
The Memory Trust Gap hardens stale notes through trap levels L0–L3, varying plausibility to find the capability threshold ρ* where ∆mem first becomes significantly negative.
Memory Feature Factorial
The Memory Trust Gap runs a 2×2×2×2 factorial over label, recency, authority, and position, computing main effects and cross-size interactions on stale-value reliance.
KEY CONTRIBUTIONS
A 2-suite closed-set benchmark
The Memory Trust Gap introduces a frozen, SHA-256 pinned Benefit and Safety benchmark with 300 base scenarios, separating floored and ceilinged no-memory baselines.
Capability-gated harm analysis
The Memory Trust Gap shows stale-value reliance stays 0.92–1.00 across Qwen3 0.6–8B, while Safety-suite ∆mem becomes negative only once models are accurate enough for stale memory to hurt.
Feature-level trigger and mitigation study
The Memory Trust Gap decomposes triggers via a 4-feature factorial and compares representations, finding recency and position interactions change with scale and that metadata vs pre-resolution help different model sizes.
RESULTS
Benefit stale reliance
1.00
+0.65 over Benefit no_memory baseline 0.35 at 4B
Benefit ∆mem stale
-0.37
vs Benefit no_memory 0.37 at 8B (CI [−.43, −.31])
Safety ∆mem stale
-0.18
below Safety no_memory 0.98 at 0.6B (CI [−.23, −.14])
Trap L3 reliance
1.00
with ∆mem −1.00 at 8B when stale note looks newest
The Memory Trust Gap runs on a frozen synthetic benchmark v1 with 150 Benefit and 150 Safety scenarios, plus external RGB and MisBench datasets. These numbers show stale memories are almost always followed, and once models are capable, this over-trust can drive accuracy far below a no-memory baseline.
BENCHMARK
The Memory Trust Gap runs on a frozen synthetic benchmark v1 with 150 Benefit and 150 Safety scenarios, plus external RGB and MisBench datasets. These numbers show stale memories are almost always followed, and once models are capable, this over-trust can drive accuracy far below a no-memory baseline.
BENCHMARK
Accuracy in Benefit suite under no_memory, clean, and stale conditions (∆mem = acc(cond) − acc(no_memory)).
KEY INSIGHT
The Memory Trust Gap finds stale-value reliance is 0.92, 0.99, 1.00, and 1.00 from 0.6B to 8B in the Benefit suite, never dropping with scale.
This breaks the intuition that more capable agents will automatically distrust stale memory; instead, capability increases the amount of accuracy they can lose when they over-trust it.
WHY IT MATTERS
The Memory Trust Gap gives builders a concrete way to measure when persistent memory starts harming agents, with capability thresholds and feature-level triggers.
With The Memory Trust Gap, developers can design memory systems that keep labels, expose metadata, or pre-resolve conflicts, tailoring mitigation to model size instead of assuming memory is always beneficial.
Guilin Zhang, Wei Jiang et al.
· 2026
A-MAC scores candidate memories using Utility, Confidence, Novelty, Recency, and Type Prior combined by a learned linear admission policy with Algorithm 1 A-MAC Memory Admission. On the LoCoMo benchmark, A-MAC achieves F1 0.583 and 2644 ms latency, improving F1 by 0.042 and reducing latency by 1187 ms compared to A-mem.
Robbyant Team, Zelin Gao et al.
arXiv 2026 · 2026
LingBot-World combines a Data Engine, Fundamental World Model, Action-Conditioned World Model, and Post-Training causal adaptation to turn a 28B-parameter video generator into a real-time interactive world simulator. On the VBench benchmark, LingBot-World achieves a dynamic degree of 0.8857 versus 0.7612 for Yume-1.5, while also improving imaging quality to 0.6683.
Manoj Madushanka Perera, Adnan Mahmood et al.
· 2026
AgenticAI-DialogGen chains ChatPreprocessor, KnowledgeExtractor, TopicAnalyzer, KnowledgeGraphBuilder, PersonaGenerator, DuelingChat Agent, ConversationValidator, ConversationRefiner, QAGeneration, and PostProcessing to turn raw multi-session chats into topic-guided, persona-grounded conversations with explicit short- and long-term memories. On the TGC / KG memory QA benchmark, Mistral-7B fine-tuned within AgenticAI-DialogGen achieves 87.36 F1, compared to GPT-4’s 83.77 F1 in a zero-shot setting on the same task.
Answers use this explainer on Memory Papers.
Checking…