AuthorsJiadong Zhang, Xiaosong Ma
TL;DR
MEMARENA uses the MASIM multi-agent simulator and ego-centric evidence projection to show that MemSearch beats Memobase by +32.5 pp recall on MEMARENA-L while adding only 48 ms latency.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Existing long-context benchmarks provide only around 0.3K tokens per speaker per dated session or undefined daily rates, far below the ∼10K human baseline.
This under-tests ego-centric, globally coherent, and activity-dense personal memory, so on-device assistants cannot be evaluated for long-horizon recall, reasoning, and permission-aware disclosure.
HOW IT WORKS
MEMARENA builds on MASIM world simulation, PP sessions, PA sessions, ego-centric projection, and benchmark question generation to generate dense, provenance-tracked personal histories.
Think of MASIM as a shared social "disk" of events and dialogs, while ego-centric projection is each agent’s private "RAM" view filtered by visibility and participation.
This design lets MEMARENA test recall, reasoning, abstention, and permission-aware access over millions of tokens in ways a plain context window or omniscient transcript cannot.
DIAGRAM
This diagram shows how MEMARENA uses MASIM to generate PP and PA dialogs, project ego-centric views, and derive evaluation instances across six dimensions.
DIAGRAM
This diagram shows how MEMARENA-L pairs readers with memory backends, runs queries, and scores accuracy and latency.
PROCESS
MASIM world simulation
MASIM world simulation creates persona-driven agents, Dunbar-layered social ties, and events that feed PP sessions and PA sessions for MEMARENA.
Dense PP and PA session generation
PP sessions and PA sessions generate around 24.1K ego-observed tokens per agent per day, with provenance and access metadata logged in the shared state.
Ego-centric projection and benchmark question generation
Ego-centric projection filters the shared log to each agent’s visible evidence, and benchmark question generation builds D1–D6 tasks from this view.
Reader and memory backend evaluation
MEMARENA pairs open-weight readers with Vanilla, BM25-RAG, Oracle retrieval, Memobase, and MemSearch, then scores recall, reasoning, abstention, and permission-aware access.
KEY CONTRIBUTIONS
MEMARENA ego-centric benchmark design
MEMARENA introduces the first ego-centric, globally coherent, activity-dense conversational memory benchmark with 50 agents over 15 days and 10.3M dialog-text tokens.
MASIM multi-agent simulator
MASIM world simulation generates PP sessions and PA sessions with day-batched synchronization, producing around 24.1K text-only ego-observed tokens per agent per day.
Systematic evaluation of memory backends
MEMARENA evaluates Vanilla, BM25-RAG, Oracle retrieval, Memobase, and MemSearch with five readers, revealing MemSearch’s +32.5 pp recall and +19.2 pp reasoning gains over Memobase at Qwen3-0.6B.
RESULTS
Recall Qwen3 0.6B MemSearch
56.2%
+32.5 pp over Memobase at 23.7%
Reasoning Qwen3 0.6B MemSearch
41.4%
+19.2 pp over Memobase at 22.2%
Avg accuracy Qwen3 0.6B MemSearch
36.1%
+2.9 pp over Vanilla at 33.2%
MemSearch latency overhead
48 ms
fixed TTFT add on Spark GB10 compared to 0 ms for Vanilla
MEMARENA-L is a 15-day, 50-agent benchmark testing Recall, Reasoning, and Trustworthiness for ego-centric personal memory assistants. The main result shows that MemSearch, within MEMARENA, improves content accuracy far more than scaling Qwen3 from 0.6B to 32B, while adding only modest latency.
BENCHMARK
MEMARENA-L is a 15-day, 50-agent benchmark testing Recall, Reasoning, and Trustworthiness for ego-centric personal memory assistants. The main result shows that MemSearch, within MEMARENA, improves content accuracy far more than scaling Qwen3 from 0.6B to 32B, while adding only modest latency.
BENCHMARK
Avg accuracy (%) on MEMARENA-L across six dimensions for Qwen3-0.6B with different memory backends.
KEY INSIGHT
Within MEMARENA, switching from Memobase to MemSearch at Qwen3-0.6B boosts Recall by +32.5 pp and Reasoning by +19.2 pp, dwarfing reader scaling gains.
This is surprising because many assume larger readers matter most, yet MEMARENA shows memory-backend choice dominates content accuracy while adding only 48 ms search latency.
WHY IT MATTERS
MEMARENA gives the community a realistic, ego-centric, activity-dense benchmark to stress-test personal memory assistants under on-device constraints and permission-aware access.
Builders can now compare memory backends like MemSearch and Memobase on long-horizon recall, reasoning, abstention, and disclosure, instead of relying on context-window size or synthetic single-task transcripts.
Guilin Zhang, Wei Jiang et al.
· 2026
A-MAC scores candidate memories using Utility, Confidence, Novelty, Recency, and Type Prior combined by a learned linear admission policy with Algorithm 1 A-MAC Memory Admission. On the LoCoMo benchmark, A-MAC achieves F1 0.583 and 2644 ms latency, improving F1 by 0.042 and reducing latency by 1187 ms compared to A-mem.
Robbyant Team, Zelin Gao et al.
arXiv 2026 · 2026
LingBot-World combines a Data Engine, Fundamental World Model, Action-Conditioned World Model, and Post-Training causal adaptation to turn a 28B-parameter video generator into a real-time interactive world simulator. On the VBench benchmark, LingBot-World achieves a dynamic degree of 0.8857 versus 0.7612 for Yume-1.5, while also improving imaging quality to 0.6683.
Manoj Madushanka Perera, Adnan Mahmood et al.
· 2026
AgenticAI-DialogGen chains ChatPreprocessor, KnowledgeExtractor, TopicAnalyzer, KnowledgeGraphBuilder, PersonaGenerator, DuelingChat Agent, ConversationValidator, ConversationRefiner, QAGeneration, and PostProcessing to turn raw multi-session chats into topic-guided, persona-grounded conversations with explicit short- and long-term memories. On the TGC / KG memory QA benchmark, Mistral-7B fine-tuned within AgenticAI-DialogGen achieves 87.36 F1, compared to GPT-4’s 83.77 F1 in a zero-shot setting on the same task.
Answers use this explainer on Memory Papers.
Checking…