AuthorsMd Nayem Uddin, Kumar Shubham, Eduardo Blanco et al.
TL;DR
Memora uses explicit memory traces plus Forgetting-Aware Memory Accuracy (FAMA) to show long-term memory agents drop from 71.82 to 15.18 on remembering as timelines grow.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Existing long-term memory benchmarks mostly require grounding from around one prior session, with PerLTQA at 1.0 and LOCOMO at 1.3 average consolidation.
Under these settings, personalized agents rarely face frequent updates or deletions, so they appear competent while still failing to maintain coherent long-term user states.
HOW IT WORKS
Memora builds on Seed Data Design, Session Simulation, Conversation Generation, and Questions and Evaluation Criteria to generate long-horizon, memory-mutating conversations with explicit traces.
Think of Memora like a long-running personal diary plus ledger: it logs every preference, activity, and goal change, then checks whether agents read the latest entries instead of stale ones.
This explicit memory tracing and FAMA scoring lets Memora test consolidation and forgetting over up to 1991 sessions and 1171.4 memory operations, far beyond what a plain context window can handle.
DIAGRAM
This diagram shows how Memora’s Remembering, Reasoning, and Recommending tasks depend on temporally distant, non-contiguous sessions.
DIAGRAM
This diagram shows how Memora evaluates model and agent outputs using criterion-level LLM judging and FAMA aggregation.
PROCESS
Seed Data Design
Memora defines ten persona profiles and structured preference, activity, and goal memory types, setting constraints for how memories can evolve.
Session Simulation
Memora simulates weeks to months of interactions, applying add, update, and delete operations to build explicit memory traces across up to 1991 sessions.
Conversation Generation
Memora converts each simulated session into multi-turn dialogues, mixing memory-neutral and memory-grounded turns while enforcing grounding via an auto-eval loop.
Questions and Evaluation Criteria
Memora derives Remembering, Reasoning, and Recommending questions plus memory presence and forgetting absence criteria directly from the memory traces.
KEY CONTRIBUTIONS
Memora Long-Term Memory Benchmark
Memora introduces weekly, monthly, and quarterly settings with up to 28.4 average consolidation and 14.8 average mutation, far exceeding prior benchmarks like PerLTQA at 1.0.
Forgetting-Aware Memory Accuracy (FAMA)
Memora proposes FAMA, combining memory presence and forgetting absence criteria with a per-question λ weighting to penalize reliance on obsolete or deleted memories.
Empirical Evaluation of LLMs and Memory Agents
Memora evaluates four LLMs and six agents, showing remembering FAMA for MemoBase drops from 43.60 weekly to 15.18 quarterly, despite explicit memory mechanisms.
RESULTS
Avg. consolidation Memora Quarterly
28.4 sessions
+27.4 over PerLTQA (1.0)
Avg. mutation Memora Quarterly
14.8 operations
+12.8 over LongMemEval (2.0)
MemoBase Remembering Weekly FAMA
43.60 score
explicit memory agent performance at short horizon
MemoBase Remembering Quarterly FAMA
15.18 score
−28.42 from weekly under heavy mutation
Memora’s Table 1 and Table 3 quantify consolidation and mutation demands plus FAMA scores across weekly, monthly, and quarterly timelines. These results show Memora forces agents to reconcile up to 309 prior sessions and 94 mutations, revealing brittleness that short-range benchmarks cannot expose.
BENCHMARK
Memora’s Table 1 and Table 3 quantify consolidation and mutation demands plus FAMA scores across weekly, monthly, and quarterly timelines. These results show Memora forces agents to reconcile up to 309 prior sessions and 94 mutations, revealing brittleness that short-range benchmarks cannot expose.
BENCHMARK
Aggregated FAMA across Remembering for long-term memory agents (Table 4).
KEY INSIGHT
Long-term memory agents like MemoBase achieve 154.4 memory presence accuracy weekly but lose 23.8 points once forgetting-aware penalties are applied.
This is surprising because explicit memory stores are expected to help, yet Memora shows they often reuse obsolete memories, breaking the assumption that more stored context automatically improves personalization.
WHY IT MATTERS
Memora unlocks a way to stress-test agents on remembering, reasoning, and recommending over months-long, heavily mutated user histories with explicit forgetting penalties.
Builders can now design and benchmark memory mechanisms that actively forget and reconcile user state, rather than just expanding context windows or appending more data to vector stores.
Guilin Zhang, Wei Jiang et al.
· 2026
A-MAC scores candidate memories using Utility, Confidence, Novelty, Recency, and Type Prior combined by a learned linear admission policy with Algorithm 1 A-MAC Memory Admission. On the LoCoMo benchmark, A-MAC achieves F1 0.583 and 2644 ms latency, improving F1 by 0.042 and reducing latency by 1187 ms compared to A-mem.
Robbyant Team, Zelin Gao et al.
arXiv 2026 · 2026
LingBot-World combines a Data Engine, Fundamental World Model, Action-Conditioned World Model, and Post-Training causal adaptation to turn a 28B-parameter video generator into a real-time interactive world simulator. On the VBench benchmark, LingBot-World achieves a dynamic degree of 0.8857 versus 0.7612 for Yume-1.5, while also improving imaging quality to 0.6683.
Manoj Madushanka Perera, Adnan Mahmood et al.
· 2026
AgenticAI-DialogGen chains ChatPreprocessor, KnowledgeExtractor, TopicAnalyzer, KnowledgeGraphBuilder, PersonaGenerator, DuelingChat Agent, ConversationValidator, ConversationRefiner, QAGeneration, and PostProcessing to turn raw multi-session chats into topic-guided, persona-grounded conversations with explicit short- and long-term memories. On the TGC / KG memory QA benchmark, Mistral-7B fine-tuned within AgenticAI-DialogGen achieves 87.36 F1, compared to GPT-4’s 83.77 F1 in a zero-shot setting on the same task.
Answers use this explainer on Memory Papers.
Checking…