AuthorsYihang Ding, Wanke Xia, Yiting Zhao et al.
TL;DR
MemGround uses a three-tier game-based memory framework plus multi-dimensional metrics to show GPT-5.2 falls to 23.61% QA Overall in Reasoning-Based Memory.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
MemGround argues that current long-term memory evaluations are "fundamentally static", fixating on simple retrieval and short-context inference while ignoring dynamic state tracking.
This failure mode appears in interactive assistants and investigative agents, where missing dynamic, hierarchical, and associative memory leads to broken timelines and incoherent multi-step reasoning.
HOW IT WORKS
MemGround’s core mechanism is a three-tier hierarchical memory evaluation framework plus a human-grounded data construction pipeline and unified interactive evaluation framework over TRPG, No Case Should Remain Unsolved, and Type Help.
You can think of MemGround like a layered memory lab: TRPG tests RAM-like Surface State Memory, No Case Should Remain Unsolved probes timeline stitching, and Type Help stresses disk-like Reasoning-Based Memory over a clue graph.
This design lets MemGround expose failures in dynamic tracking, temporal association, and DAG-style reasoning that a plain context window benchmark cannot reveal.
DIAGRAM
This diagram shows how MemGround runs an agent-environment loop to test long-term memory across TRPG, No Case Should Remain Unsolved, and Type Help.
DIAGRAM
This diagram shows how MemGround constructs human-grounded JSON cases and then evaluates LLMs with multi-dimensional metrics.
PROCESS
Environment Initialization
MemGround uses Environment Initialization to load a specific TRPG, No Case Should Remain Unsolved, or Type Help JSON scenario and build prompts for the agent.
Agent Interaction Loop
In the Agent Interaction Loop, MemGround repeatedly feeds observations, lets the agent choose retrieval or actions, and updates game states to simulate long-term interaction.
Memory Management
During Memory Management, MemGround applies StoreMemory, CompressMemory, RetrieveMemory, and IndexMemory to track fragments under realistic context window constraints.
End Evaluation
At End Evaluation, MemGround computes QA Overall, MFU, MFCO, and ETD to summarize the agent’s dynamic tracking, temporal ordering, and reasoning-based memory.
KEY CONTRIBUTIONS
Three-tier hierarchical memory evaluation framework
MemGround introduces Surface State Memory, Temporal Associative Memory, and Reasoning-Based Memory, each grounded in a distinct game scenario and evaluated with QA Overall.
Multi-dimensional metric suite for memory
MemGround defines QA Overall, Memory Fragments Unlocked (MFU), Memory Fragments with Correct Order (MFCO), and Exploration Trajectory Diagrams (ETD) to capture both outcomes and trajectories.
Game Ground with human-grounded pipeline
MemGround builds Game Ground via human annotation, calibration, and structured JSON, then evaluates closed-source and open-source models plus Mem0 and A-MEM memory agents.
RESULTS
QA Overall TRPG
51.51%
+25.73 over Qwen3-VL-32B-Instruct In-context
QA Overall No Case
47.76%
vs Gemini-3-Pro-Preview 41.61% in Surface and Temporal tasks
MFCO No Case
45.46%
temporal ordering accuracy for Gemini-3-Pro-Preview in No Case Should Remain Unsolved
MFU Type Help
50.91%
file discovery rate for Gemini-3-Pro-Preview in Reasoning-Based Memory scenario
MemGround’s main table reports QA Overall, MFU, and MFCO across TRPG, No Case Should Remain Unsolved, and Type Help. These results show that even GPT-5.2 and Gemini-3-Pro-Preview struggle with Reasoning-Based Memory despite stronger Surface State Memory performance.
BENCHMARK
MemGround’s main table reports QA Overall, MFU, and MFCO across TRPG, No Case Should Remain Unsolved, and Type Help. These results show that even GPT-5.2 and Gemini-3-Pro-Preview struggle with Reasoning-Based Memory despite stronger Surface State Memory performance.
BENCHMARK
QA Overall (%) for GPT-5.2 configurations on MemGround’s three game scenarios.
KEY INSIGHT
MemGround shows GPT-5.2 drops from 51.51% QA Overall in TRPG to just 23.61% QA Overall in Type Help, despite being a frontier model.
This is counterintuitive because GPT-5.2 is strong at long-context reasoning, yet MemGround reveals that dynamic reasoning-based memory over fragmented clues remains severely limited.
WHY IT MATTERS
MemGround unlocks a way to stress-test hierarchical memory in realistic interactive games, exposing failures in state tracking, temporal association, and DAG-style reasoning.
With MemGround, builders can design and debug memory agents like Mem0 and A-MEM against rich trajectories, rather than relying on static QA benchmarks that hide long-term reasoning breakdowns.
Guilin Zhang, Wei Jiang et al.
· 2026
A-MAC scores candidate memories using Utility, Confidence, Novelty, Recency, and Type Prior combined by a learned linear admission policy with Algorithm 1 A-MAC Memory Admission. On the LoCoMo benchmark, A-MAC achieves F1 0.583 and 2644 ms latency, improving F1 by 0.042 and reducing latency by 1187 ms compared to A-mem.
Robbyant Team, Zelin Gao et al.
arXiv 2026 · 2026
LingBot-World combines a Data Engine, Fundamental World Model, Action-Conditioned World Model, and Post-Training causal adaptation to turn a 28B-parameter video generator into a real-time interactive world simulator. On the VBench benchmark, LingBot-World achieves a dynamic degree of 0.8857 versus 0.7612 for Yume-1.5, while also improving imaging quality to 0.6683.
Manoj Madushanka Perera, Adnan Mahmood et al.
· 2026
AgenticAI-DialogGen chains ChatPreprocessor, KnowledgeExtractor, TopicAnalyzer, KnowledgeGraphBuilder, PersonaGenerator, DuelingChat Agent, ConversationValidator, ConversationRefiner, QAGeneration, and PostProcessing to turn raw multi-session chats into topic-guided, persona-grounded conversations with explicit short- and long-term memories. On the TGC / KG memory QA benchmark, Mistral-7B fine-tuned within AgenticAI-DialogGen achieves 87.36 F1, compared to GPT-4’s 83.77 F1 in a zero-shot setting on the same task.
Answers use this explainer on Memory Papers.
Checking…