AuthorsYeonjun In, Wonjoong Kim, Sangwu Park et al.
TL;DR
Personalize-then-Store uses session-level storage gating on PerMem-Bench to learn personalized memory policies, revealing large Oracle–Universal retention gaps under tight budgets.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Existing LLM memory systems use universal, static policies that ignore user-specific agent use patterns and personalized worth-storing contexts.
This misalignment causes long-horizon agents to evict essential context, over-store transient interactions, and ultimately fail on multi-year, multi-domain tasks that require persistent personalized memory.
HOW IT WORKS
Personalize-then-Store introduces PerMem-Bench, PerMem-Benchs, PerMem-Benchd, and session-level storage gating with a structural note to benchmark and learn user-specific worth-storing sessions.
You can think of Personalize-then-Store as adding a smart circuit breaker in front of RAM, deciding which conversations deserve to be written to long-term disk.
This KEY_MECHANISM lets Personalize-then-Store concentrate a fixed memory budget on long-horizon projects, something a plain context window or universal storage policy cannot achieve.
DIAGRAM
This diagram shows how Personalize-then-Store constructs PerMem-Bench dialogues from personas through life skeletons into dual-simulator conversations.
DIAGRAM
This diagram shows how Personalize-then-Store evaluates memory systems on PerMem-Bench using session gating and Memory Retention Rate.
PROCESS
User-Specific Agent Use Profiling
Personalize-then-Store infers per-user domain participation, frequency, and memory necessity, creating Mp,d triplets that define which domains need persistent storage.
Life Skeleton and Timeline Construction
Using the profile, Personalize-then-Store builds projects and events with reference memories, then arranges them into a unified multi-month timeline.
Dialogue Generation via Dual-Simulator
Personalize-then-Store runs a user simulator and agent simulator over the life skeleton to synthesize realistic long-horizon sessions with embedded worth-storing facts.
Session-Level Storage Gating and Evaluation
Personalize-then-Store applies Greedy, Context-aware, or Structure-aware gating, runs Mem0, Memory-R1, and RMM, and scores Memory Retention Rate over reference memories.
KEY CONTRIBUTIONS
PerMem-Bench for personalized memory
Personalize-then-Store introduces PerMem-Bench with PerMem-Benchs and PerMem-Benchd, covering up to 32 month timelines, 1M tokens per user, and up to 146 reference memories.
Session-level storage gating paradigm
Personalize-then-Store proposes session-level storage gating with Greedy, Context-aware, and Structure-aware methods that wrap existing systems like Mem0, Memory-R1, and RMM.
Empirical study of personalization limits
Personalize-then-Store shows Structure-aware gating reaches F1 up to 0.844 on PerMem-Benchs, yet still leaves a large Oracle–Universal Memory Retention Rate gap under fixed budgets.
RESULTS
F1 on PerMem-Benchs
0.844
+0.184 over Qwen3 14B Greedy
FNR on PerMem-Benchs
0.115
Structure-aware vs 0.457 for Qwen3 14B Greedy
F1 on PerMem-Benchd
0.805
+0.148 over Qwen3 14B Greedy
Dialogue quality score
98.4%
with 96.9% Gwet AC1 agreement in meta evaluation
On PerMem-Benchs and PerMem-Benchd, Personalize-then-Store evaluates session-level gating quality and memory behavior, showing Structure-aware gating can reach 0.844 F1 while the data pipeline maintains 98.4% dialogue quality.
BENCHMARK
On PerMem-Benchs and PerMem-Benchd, Personalize-then-Store evaluates session-level gating quality and memory behavior, showing Structure-aware gating can reach 0.844 F1 while the data pipeline maintains 98.4% dialogue quality.
BENCHMARK
F1 on PerMem-Benchs for different session-level storage gating methods and LLM backbones.
KEY INSIGHT
Personalize-then-Store shows Structure-aware gating achieves 0.844 F1 on PerMem-Benchs, yet only yields marginal Memory Retention Rate gains over Universal policies.
This is surprising because such high classification accuracy should translate directly into better retention, but compounding false positives and false negatives still squander the fixed memory budget.
WHY IT MATTERS
Personalize-then-Store unlocks a concrete way to study and benchmark personalized memory policies, separating user-specific worth-storing decisions from raw memory mechanisms.
Builders can now plug session-level storage gating into existing systems like Mem0 or RMM and quantify how much Oracle-level personalization could improve long-horizon retention under realistic memory budgets.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xingyu Lyu, Jianfeng He et al.
· 2026
ADAM combines Anchor extraction, Distribution estimation, Anchor selection, and Query generation to adaptively probe agent memory via an auxiliary generator and entropy based selection. On the EHRAgent benchmark with Llama2-7b-chat, ADAM reaches EQ=77 and ASR=1.00, compared to MEXTRA’s EQ=44 and ASR=0.89.
Shannan Yan, Jingchen Ni et al.
· 2026
AdaMem organizes dialogue history into Working Memory, Episodic Memory, Persona Memory, and Graph Memory coordinated by a Memory Agent, Research Agent, and Working Agent. On LoCoMo with GPT-4.1-mini, AdaMem achieves 44.65 F1 overall, beating the best baseline LangMem at 41.76 F1 by +2.89.
Answers use this explainer on Memory Papers.
Checking…