AuthorsBen Wang, Kang Zhou, Lifan Guo et al.
TL;DR
FinPerMA uses a three-layer Impact Model with Post-Shock checkpoints to show full-context LLM agents still cap out at ≈46.9% accuracy and ≈39% MCQ on personalized finance memory.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Existing personalized-memory benchmarks mostly test factual retention or long-context recall, not event-driven preference adaptation after shocks.
LLM-based financial assistants often recall past facts yet fail to update a persistent user model, leaving recommendations misaligned after major market or life events.
HOW IT WORKS
FinPerMA builds a controlled pipeline with Persona Synthesis, Event Timeline Construction, a three-layer Impact Model, Dialogue Synthesis, and Evaluation Task Design to freeze investor trajectories.
You can think of FinPerMA like a financial memory lab: personas and events are the "hardware," the Impact Model is the "OS," and dialogues are the "logs" agents must reason over.
This design lets FinPerMA separate recall from state updating, so Post-Shock checkpoints directly test whether agents integrate consequential events into a persistent, individualized user state beyond a plain context window.
DIAGRAM
This diagram shows how FinPerMA's Impact Model transforms a persona and event into a validated event reaction used in dialogues.
DIAGRAM
This diagram shows how FinPerMA evaluates LLM agents across memory configurations and four temporal checkpoints on the v8gold corpus.
PROCESS
Persona Synthesis
FinPerMA samples demographic, financial, and psychometric attributes, then classifies each persona into Behavioral Investor Types using Persona Synthesis rules.
Event Timeline Construction
FinPerMA builds a 5–8 event timeline per persona from 97 dated macro, industry, and personal events with category diversification and spacing constraints.
Impact Model
FinPerMA applies the three-layer Impact Model to each persona–event pair, generating deterministic ImpactConstraint objects and validated EventReaction outputs.
Dialogue Synthesis
FinPerMA feeds persona, events, and reactions into Dialogue Synthesis, creating multi-session investor–advisor conversations that implicitly surface preferences for later Evaluation Task Design.
KEY CONTRIBUTIONS
Dynamic personalized-memory benchmark
FinPerMA jointly tests stable-profile recall and event-conditioned preference adaptation using Persona Synthesis and Event Timeline Construction over 97 dated events and ≈300 personas.
Post-Shock checkpoint protocol
FinPerMA introduces the Post-Shock checkpoint in Evaluation Task Design to isolate whether agents integrate high-severity events into persistent user models rather than relying on stale profiles.
Auditable Impact Model and frozen corpus
FinPerMA's three-layer Impact Model combines deterministic rules, constrained narration, and automated validation to produce the v8gold corpus of 2,994 questions from 276 personas for reproducible memory evaluation.
RESULTS
Accfull
46.9%
+19.6 points over Qwen-3.8 Accno-mem
MCQ
38.7%
vs Qwen-3.8 MCQ under no_memory near chance
Acc
40.6%
+22.4 points over Qwen3.7-Max no_memory with Bge_rag
Ctxk
1.40
context tokens per query for Bge_rag vs 12.8 for full_context
FinPerMA evaluates LLM agents on 2,994 questions from 276 personas, including 2,494 six-option MCQs and 500 open-ended items. These results show FinPerMA exposes large, unsaturated gaps in personalized memory, with full-context Qwen-3.8 reaching 46.9% accuracy and dense retrieval on Qwen3.7-Max recovering ≈88.2% of the memory gap using only 1.40k tokens of context.
BENCHMARK
FinPerMA evaluates LLM agents on 2,994 questions from 276 personas, including 2,494 six-option MCQs and 500 open-ended items. These results show FinPerMA exposes large, unsaturated gaps in personalized memory, with full-context Qwen-3.8 reaching 46.9% accuracy and dense retrieval on Qwen3.7-Max recovering ≈88.2% of the memory gap using only 1.40k tokens of context.
BENCHMARK
Accfull on FinPerMA across seven frontier LLM backbones.
BENCHMARK
Acc on FinPerMA for different memory configurations with Qwen3.7-Max.
KEY INSIGHT
On Qwen3.7-Max, simple Bge_rag retrieval reaches 40.6% Acc and recovers ≈88.2% of the no_memory–full_context gap with only 1.40k tokens.
This is surprising because purpose-built structured memory systems like MemOS and Mem0, designed for agent memory, trail in accuracy (32.0% and 36.6%) despite their specialized architectures.
WHY IT MATTERS
FinPerMA gives the field a frozen, event-grounded benchmark that cleanly separates recall, preference reasoning, and context adaptation across four temporal checkpoints.
Builders can now stress-test memory systems on Post-Shock preference updates and anti-typical personas, designing hybrid retrieval–profile architectures that preserve the signals needed for genuine long-horizon personalization.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xingyu Lyu, Jianfeng He et al.
· 2026
ADAM combines Anchor extraction, Distribution estimation, Anchor selection, and Query generation to adaptively probe agent memory via an auxiliary generator and entropy based selection. On the EHRAgent benchmark with Llama2-7b-chat, ADAM reaches EQ=77 and ASR=1.00, compared to MEXTRA’s EQ=44 and ASR=0.89.
Shannan Yan, Jingchen Ni et al.
· 2026
AdaMem organizes dialogue history into Working Memory, Episodic Memory, Persona Memory, and Graph Memory coordinated by a Memory Agent, Research Agent, and Working Agent. On LoCoMo with GPT-4.1-mini, AdaMem achieves 44.65 F1 overall, beating the best baseline LangMem at 41.76 F1 by +2.89.
Answers use this explainer on Memory Papers.
Checking…