AuthorsBaichuan Li, Junyi Yao, Zihao Zheng
TL;DR
Memory–Clarification Boundary (MCB) benchmark forces agents to choose between remember, verify, or ask, revealing Qwen3.5-9B accuracy drops from 0.557 to 0.343 when decisions become tool calls.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Persistent memory in LLM agents can silently corrupt future behavior when a wrong update is durably stored instead of clarified or re-verified. MCB shows that bare Qwen3.5-9B verifies 12/18 freshness items but asks on 0/12 clarification items, confusing world checks with user intent.
This failure hits long-term personalization and safety-critical policy adherence, where mis-scoped preferences or stale facts can propagate unnoticed across sessions and tasks.
HOW IT WORKS
Memory–Clarification Boundary (MCB) defines four actions—persist, ephemeral, verify, and clarify—and pairs them with the MCB-Act tool-call variant to test memory commitment. MCB also introduces over-memory and under-memory metrics plus a category-majority oracle to diagnose behavior beyond raw accuracy.
You can think of MCB like a gatekeeper between RAM and disk: ephemeral is short-lived RAM, persist is disk, verify is a sensor check, and clarify is asking the operator. The MCB-Act variant then forces the gatekeeper to actually flip a concrete switch instead of just naming the intended action.
This design lets Memory–Clarification Boundary (MCB) expose when agents misroute uncertainty, such as verifying the world instead of clarifying user intent, which a plain context window or recall-only benchmark cannot reveal.
DIAGRAM
This diagram shows how Memory–Clarification Boundary (MCB) routes an acquired update through the four actions and their corresponding tools in MCB-Act.
DIAGRAM
This diagram shows how Memory–Clarification Boundary (MCB) constructs datasets, collects non-author labels, and evaluates Claude and Qwen under multiple prompts.
PROCESS
Task and Labels
Memory–Clarification Boundary (MCB) defines the four-way persist, ephemeral, verify, and clarify label space and encodes the weaker-action tie-breaker to favor safer commitments.
Benchmark and Method
Memory–Clarification Boundary (MCB) constructs 140 primary scenarios plus a 70-item contrast set, each with acquire context, candidate update, and reuse context.
Label and Tool-Call Evaluation
Memory–Clarification Boundary (MCB) runs label-mode prompts and the MCB-Act tool-call variant, mapping memory_write, use_now, check_source, and ask_user to the four actions.
Metrics and Statistics
Memory–Clarification Boundary (MCB) computes accuracy, macro-F1, over-memory, under-memory, and class recalls, using paired exact McNemar tests with Holm correction.
KEY CONTRIBUTIONS
Memory–Clarification Boundary benchmark
Memory–Clarification Boundary (MCB) introduces a four-action benchmark with 140 primary items and a 70-item contrast set, plus non-author-adjudicated labels with 0.971 agreement (Cohen’s κ = 0.962).
Two-family experiment
Memory–Clarification Boundary (MCB) evaluates Claude Haiku 4.5, Claude Sonnet 4.6, and Qwen3.5-9B under bare, policy, and few-shot prompts, revealing cross-family under-asking and prompt-sensitive over-memory.
MCB-Act tool-call variant
Memory–Clarification Boundary (MCB) adds MCB-Act, where agents must emit memory_write, use_now, check_source, or ask_user calls, exposing a Qwen3.5-9B accuracy drop from 0.557 to 0.343.
RESULTS
Accuracy
0.771
+0.214 over Qwen3.5-9B bare
Macro-F1
0.846
vs Claude Haiku 4.5 policy
Over-memory OM
0.100
-0.143 OM vs Qwen3.5-9B bare
Clarification recall
0.333
Qwen3.5-9B few-shot vs 0.000 bare
These metrics come from the 70-item held-out Memory–Clarification Boundary (MCB) test, which targets memory commitment decisions rather than retrieval. The main result shows that Qwen3.5-9B few-shot prompting raises accuracy from 0.557 to 0.771 while the policy prompt cuts erroneous persistence from 0.243 to 0.100.
BENCHMARK
These metrics come from the 70-item held-out Memory–Clarification Boundary (MCB) test, which targets memory commitment decisions rather than retrieval. The main result shows that Qwen3.5-9B few-shot prompting raises accuracy from 0.557 to 0.771 while the policy prompt cuts erroneous persistence from 0.243 to 0.100.
BENCHMARK
Accuracy on the Memory–Clarification Boundary (MCB) held-out set.
KEY INSIGHT
Memory–Clarification Boundary (MCB) shows that Qwen3.5-9B verifies 12/18 freshness items but asks users on 0/12 clarification items in bare label mode. When forced into MCB-Act tool calls, Qwen3.5-9B accuracy drops from 0.557 to 0.343 and verification recall collapses from 0.667 to 0.056.
This is surprising because many assume agents will ask when unsure, yet Memory–Clarification Boundary (MCB) reveals a strong bias toward verify or persist instead of clarify, even when clarify is the only correct action.
WHY IT MATTERS
Memory–Clarification Boundary (MCB) gives researchers a concrete way to measure when agents should remember, limit, verify, or ask, with behavior-specific metrics like over-memory and clarification recall. This lets developers tune prompts or policies to reduce silent miscommitments without relying solely on aggregate accuracy.
With Memory–Clarification Boundary (MCB), builders can now test both label decisions and structured tool-call choices, catching cases where an agent says it will clarify but actually calls verify or persist, which was previously invisible in recall-only benchmarks.
Guilin Zhang, Wei Jiang et al.
· 2026
A-MAC scores candidate memories using Utility, Confidence, Novelty, Recency, and Type Prior combined by a learned linear admission policy with Algorithm 1 A-MAC Memory Admission. On the LoCoMo benchmark, A-MAC achieves F1 0.583 and 2644 ms latency, improving F1 by 0.042 and reducing latency by 1187 ms compared to A-mem.
Robbyant Team, Zelin Gao et al.
arXiv 2026 · 2026
LingBot-World combines a Data Engine, Fundamental World Model, Action-Conditioned World Model, and Post-Training causal adaptation to turn a 28B-parameter video generator into a real-time interactive world simulator. On the VBench benchmark, LingBot-World achieves a dynamic degree of 0.8857 versus 0.7612 for Yume-1.5, while also improving imaging quality to 0.6683.
Manoj Madushanka Perera, Adnan Mahmood et al.
· 2026
AgenticAI-DialogGen chains ChatPreprocessor, KnowledgeExtractor, TopicAnalyzer, KnowledgeGraphBuilder, PersonaGenerator, DuelingChat Agent, ConversationValidator, ConversationRefiner, QAGeneration, and PostProcessing to turn raw multi-session chats into topic-guided, persona-grounded conversations with explicit short- and long-term memories. On the TGC / KG memory QA benchmark, Mistral-7B fine-tuned within AgenticAI-DialogGen achieves 87.36 F1, compared to GPT-4’s 83.77 F1 in a zero-shot setting on the same task.
Answers use this explainer on Memory Papers.
Checking…