AuthorsXinle Deng, Yida Xue, Xiangyuan Ru et al.
TL;DR
MobileMem uses the KEME knowledge‑guided synthesis engine over year‑scale mobile trajectories to benchmark long‑term on‑device memory, showing A‑MEM and HippoRAG2 reaching up to 80.06% LLM‑Judge accuracy while exposing large cost–performance tradeoffs.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
MobileMem highlights that existing long-term memory benchmarks “fail to capture the heterogeneous, multimodal, and continuously evolving nature of mobile interactions,” leaving on-device memory systems under-evaluated.
Without realistic year-scale trajectories, persistent personal assistants mis-handle multi-hop, temporal, and preference reasoning, undermining continuous personalization on smartphones and wearables.
HOW IT WORKS
MobileMem centers on User Prior Knowledge Construction, KEME, User Trajectory Synthesis, QA Pair Synthesis, and Quality Control to generate coherent year-scale mobile interaction trajectories and questions.
You can think of MobileMem like a personal “experience black box recorder,” where KEME organizes raw persona facts and app events into a structured flight log instead of a flat transcript.
This design lets MobileMem test how memory systems evolve user models over time, something a plain context window or naive RAG over dialogues cannot capture.
DIAGRAM
This diagram shows how MobileMem situates the system-level memory layer and application-specific memories in the envisioned on-device memory ecosystem.
DIAGRAM
This diagram shows how KEME alternates top-down planning and bottom-up evolution to synthesize trajectories and then generates hierarchical QA pairs.
PROCESS
User Prior Knowledge Construction
MobileMem first runs User Prior Knowledge Construction to build personas and app schemas from PersonaHub, OPeRA, interviews, and real OPPO trajectories.
KEME User Trajectory Synthesis
MobileMem then invokes KEME User Trajectory Synthesis, where the Knowledge-Guided Planner, Knowledge Anchor Grounder, Experience Realizer, and Experience-Driven Reviser generate temporally coherent sessions.
Question-Answer Pair Synthesis
MobileMem applies Question-Answer Pair Synthesis, using a bottom-up algorithm over profile leaves and leaf events to create multi-hop, temporal, and preference questions.
Quality Control
Finally, MobileMem runs Quality Control with automatic validation, multimodal checks, and human sampling to filter inconsistent trajectories, images, and QA pairs.
KEY CONTRIBUTIONS
MobileMem Benchmark
MobileMem introduces a long-horizon on-device memory benchmark built from User Prior Knowledge Construction, KEME, User Trajectory Synthesis, QA Pair Synthesis, and Quality Control, covering multi-hop, temporal, and preference reasoning.
KEME Synthesis Framework
MobileMem presents KEME, a Knowledge-guided Experience synthesis engine that combines a Knowledge-Guided Planner, Knowledge Anchor Grounder, Experience Realizer, and Experience-Driven Reviser to generate evolving trajectories.
MobileMem-Omni Dataset
MobileMem extends to MobileMem-Omni, adding multimodal trajectories with 19,060 images and 7,415 QA pairs, plus personal knowledge graphs and visual memory points for on-device multimodal memory evaluation.
RESULTS
Overall LLM-Judge (GPT-5.4-mini, A-MEM)
78.39%
+33.20 over Long Context (45.19%)
Overall LLM-Judge (GPT-5.4-mini, HippoRAG2)
80.06%
+34.87 over Long Context (45.19%)
Token Cost A-MEM (GPT-5.4-mini)
11170.44k tokens
More than 4x HippoRAG2 at 2741.05k tokens
MobileMem-Omni Scale
7,415 QA, 19,060 images
16 users with 1.72M tokens per user on average
These results come from the MobileMem and MobileMem-Omni benchmarks, which test long-horizon recall, multi-hop reasoning, temporal reasoning, knowledge updating, and multimodal memory. The MAIN_RESULT shows that A-MEM and HippoRAG2 can reach around 80% LLM-Judge accuracy but at very different token construction costs, highlighting efficiency–accuracy tradeoffs for on-device memory.
BENCHMARK
These results come from the MobileMem and MobileMem-Omni benchmarks, which test long-horizon recall, multi-hop reasoning, temporal reasoning, knowledge updating, and multimodal memory. The MAIN_RESULT shows that A-MEM and HippoRAG2 can reach around 80% LLM-Judge accuracy but at very different token construction costs, highlighting efficiency–accuracy tradeoffs for on-device memory.
BENCHMARK
Overall LLM-Judge score (%) for GPT-5.4-mini backbone on MobileMem.
KEY INSIGHT
MobileMem shows that LangMem has the lowest overall accuracy yet achieves the best adversarial question performance, despite stronger systems reaching up to 80.06% overall.
This is surprising because you would expect better memory systems to be safer, but MobileMem reveals that richer retrieval can actually increase overconfident answers on unanswerable questions.
WHY IT MATTERS
MobileMem unlocks realistic evaluation of year-scale, multimodal, on-device memory systems that must integrate app logs, screenshots, and evolving personas.
Builders can now prototype assistants that learn from a year of mobile life, stress-test memory update and retrieval policies, and compare architectures like A-MEM, HippoRAG2, and EverMemOS under identical long-horizon conditions.
Guilin Zhang, Wei Jiang et al.
· 2026
A-MAC scores candidate memories using Utility, Confidence, Novelty, Recency, and Type Prior combined by a learned linear admission policy with Algorithm 1 A-MAC Memory Admission. On the LoCoMo benchmark, A-MAC achieves F1 0.583 and 2644 ms latency, improving F1 by 0.042 and reducing latency by 1187 ms compared to A-mem.
Robbyant Team, Zelin Gao et al.
arXiv 2026 · 2026
LingBot-World combines a Data Engine, Fundamental World Model, Action-Conditioned World Model, and Post-Training causal adaptation to turn a 28B-parameter video generator into a real-time interactive world simulator. On the VBench benchmark, LingBot-World achieves a dynamic degree of 0.8857 versus 0.7612 for Yume-1.5, while also improving imaging quality to 0.6683.
Manoj Madushanka Perera, Adnan Mahmood et al.
· 2026
AgenticAI-DialogGen chains ChatPreprocessor, KnowledgeExtractor, TopicAnalyzer, KnowledgeGraphBuilder, PersonaGenerator, DuelingChat Agent, ConversationValidator, ConversationRefiner, QAGeneration, and PostProcessing to turn raw multi-session chats into topic-guided, persona-grounded conversations with explicit short- and long-term memories. On the TGC / KG memory QA benchmark, Mistral-7B fine-tuned within AgenticAI-DialogGen achieves 87.36 F1, compared to GPT-4’s 83.77 F1 in a zero-shot setting on the same task.
Answers use this explainer on Memory Papers.
Checking…