AuthorsZachary Johnson, Nigel Boachie Kumankumah, Somya Chatterjee et al.
TL;DR
AIM uses multi-label intent detection plus privacy-aware visibility classification to reach 96.0% visibility accuracy and 70.5% state-aware operation accuracy on MUMBench.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Traditional LLM agents are scoped to single sessions, so they cannot learn evolving user preferences or safely share cross-user knowledge.
In multi-user, multi-agent systems, this leads to fragmented context, weak personalization, and no way to enforce private versus public access across 672 interactions from 93 users.
HOW IT WORKS
AIM chains Multi-Label Intent Detection, Structured Memory Extraction, Privacy-Aware Visibility Classification, In-Context Semantic Deduplication, and Cascaded Memory Retrieval over a shared memory store.
You can think of AIM like an operating system with user accounts: agents are apps, memories are files, and visibility labels are file permissions.
This design lets AIM enforce index-level access control and multi-label operations, enabling behaviors beyond what a plain context window or single-user memory can support.
DIAGRAM
This diagram shows how AIM processes each user turn through write and read paths with visibility-constrained retrieval.
DIAGRAM
This diagram shows how MUMBench interactions are replayed through AIM and scored with strict and state-aware metrics.
PROCESS
Multi-Label Intent Detection
AIM uses the Multi-Label Intent Detection component to map each user message to create, read, update, delete, or no-op operations, allowing multi-operation turns.
Structured Memory Extraction
When writes are needed, AIM runs Structured Memory Extraction to produce content, tags, and metadata tuples suitable for indexing.
Privacy-Aware Visibility Classification
AIM applies Privacy-Aware Visibility Classification to label each extracted memory as PUBLIC or PRIVATE, driving access control and deduplication scope.
Cascaded Memory Retrieval
For reads, AIM performs Cascaded Memory Retrieval with visibility-constrained dense search, tag-level rescoring, and LLM reranking fused with recency.
KEY CONTRIBUTIONS
Agentic Interoperable Memory Framework
AIM introduces a unified multi-agent, multi-user memory framework combining Multi-Label Intent Detection and Cascaded Memory Retrieval over a shared vector index with index-level access control.
Privacy-Aware Classification Engine
AIM’s Privacy-Aware Visibility Classification and visibility predicate V(u, mi) achieve up to 96.0% visibility accuracy on MUMBench while structurally blocking unauthorized access.
MUMBench Multi-User Benchmark
AIM is evaluated on MUMBench, a 672-interaction, 93-user dataset across four domains that tests create, read, update, delete, and privacy constraints end-to-end.
RESULTS
Strict accuracy
58.8% avg
+6.1 over Mistral-Large-3
State-Aware
70.5% avg
+1.1 over DeepSeek-V3.2
Visibility
96.0% avg
+5.2 over Llama-3.3-70B-Instruct
Content Quality
91.6% avg
+7.3 over Llama-3.3-70B-Instruct
On MUMBench, which spans Coding, Customer Support, Education, and Travel Planning, AIM is run end-to-end with different backbone models. These results show AIM can accurately choose operations, preserve privacy, and extract high-quality memories in a realistic multi-user setting.
BENCHMARK
On MUMBench, which spans Coding, Customer Support, Education, and Travel Planning, AIM is run end-to-end with different backbone models. These results show AIM can accurately choose operations, preserve privacy, and extract high-quality memories in a realistic multi-user setting.
BENCHMARK
Macro-averaged Strict accuracy across models on MUMBench.
KEY INSIGHT
AIM with GPT-5.4-mini reaches 58.8% strict and 70.5% state-aware accuracy, beating larger GPT-5.4 on both metrics.
This is surprising because many expect larger models to dominate, yet AIM shows smaller models can handle complex memory lifecycles competitively.
WHY IT MATTERS
AIM enables multi-agent, multi-user systems to share a single memory registry while enforcing strict private versus public access at the index level.
Builders can now deploy cross-user, cross-agent assistants that remember over time without separate stores per user or ad hoc privacy hacks.
Jingbiao Mei, Jinghong Chen et al.
arXiv 2026 · 2026
ATM-Bench structurally evaluates long-term multimodal personal memory using Memory Ingestion, Retrieval, and Answer Generation with Schema-Guided Memory and Descriptive Memory variants. On ATM-Bench-Hard, Oracle with SGM reaches 47.3% QS while the best full system stays under 20% accuracy, revealing a large gap.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xingyu Lyu, Jianfeng He et al.
· 2026
ADAM combines Anchor extraction, Distribution estimation, Anchor selection, and Query generation to adaptively probe agent memory via an auxiliary generator and entropy based selection. On the EHRAgent benchmark with Llama2-7b-chat, ADAM reaches EQ=77 and ASR=1.00, compared to MEXTRA’s EQ=44 and ASR=0.89.
Answers use this explainer on Memory Papers.
Checking…