AuthorsXianxuan Long, Zhikai Chen, Shenglai Zeng et al.
TL;DR
MemTrace uses per-fact knowledge-point probes across memory age, question type, and evidence condition to show that failures come from evidence use, not retrieval, with errors 10× more often on reachable evidence.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Aggregating accuracy over question rows treats multiple queries about one fact as independent, so it cannot show how that fact behaves as conditions change.
MemTrace shows that systems can answer a current role correctly while failing historical and trajectory questions on the same knowledge point, breaking user-facing coherence.
HOW IT WORKS
MemTrace centers four components: Knowledge points, Probe construction, Metrics, and Diagnostic views, all built from multi-session user histories in HaluMem-Medium.
Like a card catalog for user facts, MemTrace files each fact as a card, then repeatedly checks that card under different time windows and evidence conditions.
This per-fact protocol lets MemTrace expose retention, trajectory tracking, and safe behavior under missing or conflicting evidence that a plain context window benchmark cannot reveal.
DIAGRAM
This diagram shows how MemTrace probes each knowledge point across memory age, question type, and evidence condition while keeping the underlying fact fixed.
DIAGRAM
This diagram shows how MemTrace builds memory windows, scores responses, and then decomposes failures into retrieval reach versus evidence use.
PROCESS
Probe Construction
MemTrace pairs each knowledge point with a memory window and evidence condition, forming base probes that reflect realistic multi-session queries.
Memory Age Windows
MemTrace evaluates each user history at eight chronological checkpoints W1 to W8, mapping a retention trace from fresh to saturated memory.
Question Type Expansion
MemTrace expands each base probe into current, historical, and trajectory question rows so the same fact is tested under varied query intents.
Diagnostic Views
MemTrace aggregates scores into memory maintenance, evidence-condition behavior, and reach versus use attribution to explain where and why systems fail.
KEY CONTRIBUTIONS
Knowledge-point benchmark design
MemTrace introduces a knowledge-point benchmark over 835 typed knowledge points and 15,422 question rows, using Probe construction and Diagnostic views to analyze fact-level behavior.
Three probing dimensions
MemTrace systematically varies memory age, question type, and evidence condition, showing that recovering current and earlier states does not imply tracking trajectory or correcting false premises.
Failure attribution: reach vs use
MemTrace’s reach versus use analysis shows that when systems fail, gold evidence is retrievable about 10× more often than missing, making evidence use the dominant bottleneck.
RESULTS
Overall Saturated Gist
36.5%
+3.9 over Mem-T
Current Fresh Gist
69.7%
+3.7 over Qwen3.5-35B
Historical Saturated Gist
50.9%
+3.8 over Qwen3.5-35B
Trajectory Saturated Gist
19.8%
+6.4 over HippoRAG-v2
On MemTrace’s 835 knowledge points and 5,677 base probes, HippoRAG-v2 and Mem-T illustrate how endpoint accuracy and trajectory questions diverge. The main result shows that similar pooled scores mask large gaps, such as Qwen3.5-35B dropping from 49.0% fresh trajectory Gist to 6.7% saturated while Mem-T retains 19.8%.
BENCHMARK
On MemTrace’s 835 knowledge points and 5,677 base probes, HippoRAG-v2 and Mem-T illustrate how endpoint accuracy and trajectory questions diverge. The main result shows that similar pooled scores mask large gaps, such as Qwen3.5-35B dropping from 49.0% fresh trajectory Gist to 6.7% saturated while Mem-T retains 19.8%.
BENCHMARK
Saturated Gist accuracy (%) for current questions across representative memory paradigms.
KEY INSIGHT
MemTrace’s reach versus use replay shows that only 7.0% of sampled failures are reach misses, while 73.3% are retriever-reached but unsolved.
This is counterintuitive because many builders assume retrieval is the main bottleneck, yet MemTrace demonstrates that evidence use dominates by roughly an order of magnitude.
WHY IT MATTERS
MemTrace unlocks fact-level diagnostics for long-term memory systems, revealing how individual user facts decay, mis-update, or cause unsafe behavior over time.
Builders can now design memory architectures that explicitly organize temporal and conflict structure around knowledge points, instead of only scaling context windows or retrievers.
Guilin Zhang, Wei Jiang et al.
· 2026
A-MAC scores candidate memories using Utility, Confidence, Novelty, Recency, and Type Prior combined by a learned linear admission policy with Algorithm 1 A-MAC Memory Admission. On the LoCoMo benchmark, A-MAC achieves F1 0.583 and 2644 ms latency, improving F1 by 0.042 and reducing latency by 1187 ms compared to A-mem.
Robbyant Team, Zelin Gao et al.
arXiv 2026 · 2026
LingBot-World combines a Data Engine, Fundamental World Model, Action-Conditioned World Model, and Post-Training causal adaptation to turn a 28B-parameter video generator into a real-time interactive world simulator. On the VBench benchmark, LingBot-World achieves a dynamic degree of 0.8857 versus 0.7612 for Yume-1.5, while also improving imaging quality to 0.6683.
Manoj Madushanka Perera, Adnan Mahmood et al.
· 2026
AgenticAI-DialogGen chains ChatPreprocessor, KnowledgeExtractor, TopicAnalyzer, KnowledgeGraphBuilder, PersonaGenerator, DuelingChat Agent, ConversationValidator, ConversationRefiner, QAGeneration, and PostProcessing to turn raw multi-session chats into topic-guided, persona-grounded conversations with explicit short- and long-term memories. On the TGC / KG memory QA benchmark, Mistral-7B fine-tuned within AgenticAI-DialogGen achieves 87.36 F1, compared to GPT-4’s 83.77 F1 in a zero-shot setting on the same task.
Answers use this explainer on Memory Papers.
Checking…