AuthorsYifei Li, Weidong Guo, Lingling Zhang et al.
TL;DR
LoCoMo-Plus reframes long-term memory as constraint-consistency under cue–trigger semantic disconnect, revealing up to 26.06% cognitive memory accuracy where factual scores exceed 71.78%.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
LoCoMo-Plus shows that gemini-2.5-pro drops from 71.78 average factual accuracy on LoCoMo to only 26.06 on cognitive memory tasks.
Under this setting, long-term conversational agents fail to respect implicit state, goal, and value constraints, leading to behaviorally inconsistent and sometimes harmful responses.
HOW IT WORKS
LoCoMo-Plus builds cognitive memory cases through Implicit Cue Dialogue Generation, Memory-Worthy Verification, Cue–Trigger Query Construction, Semantic Filtering, and Cue Memory Elicitation Validation before insertion into LoCoMo dialogues.
You can think of LoCoMo-Plus like a cognitive RAM plus a judge: cues write latent constraints, triggers read them back, and an LLM judge checks behavioral consistency.
This constraint-consistency mechanism lets LoCoMo-Plus test whether agents preserve and apply implicit constraints across long contexts, something a plain context window and string-matching metrics cannot reliably measure.
DIAGRAM
This diagram shows how LoCoMo-Plus models the interaction between user, agent, cue, trigger, and LLM judge during evaluation.
DIAGRAM
This diagram shows the end-to-end LoCoMo-Plus pipeline from cue generation to constraint-consistency scoring.
PROCESS
Implicit Cue Dialogue Generation
LoCoMo-Plus first uses Implicit Cue Dialogue Generation to create short dialogues that encode latent state, goal, preference, or value information.
Memory-Worthy Verification
In Memory-Worthy Verification, annotators filter cues to keep only persistent, behaviorally constraining information that truly benefits long-term conversational assistance.
Cue–Trigger Query Construction
Cue–Trigger Query Construction then generates underspecified trigger queries whose correct resolution depends on the cue while maintaining low semantic similarity.
Semantic Filtering and Cue Memory Elicitation Validation
Semantic Filtering removes shortcut pairs, and Cue Memory Elicitation Validation ensures each remaining instance genuinely requires recalling and applying the implicit cue.
KEY CONTRIBUTIONS
LoCoMo-Plus Cognitive Memory Benchmark
LoCoMo-Plus extends LoCoMo by adding cognitive memory QA instances decomposed into causal, state, goal, and value constraints, with gemini-2.5-pro scoring only 26.06% on this axis.
Constraint-Consistency Evaluation Framework
LoCoMo-Plus replaces task-disclosed prompts and string-matching metrics with a unified conversational query and LLM-based constraint-consistency judging aligned with implicit memory use.
Empirical Analysis of Memory and Metric Biases
LoCoMo-Plus empirically shows prompt bias, length bias, and rapid cognitive memory collapse, including PDR dropping to 5.88% and RTR to 18% for cognitive cases as context grows.
RESULTS
LoCoMo average
71.78%
+26.06 over LoCoMo-Plus cognitive accuracy for gemini-2.5-pro
LoCoMo-Plus cognitive
26.06%
vs gemini-2.5-pro factual average of 71.78% on LoCoMo
Object memory PDR
92.85%
high precision drop rate for object memory under increasing dialogue length
Cognitive memory PDR
5.88%
rapid collapse of cognitive memory compared to 67.39% episodic PDR
LoCoMo-Plus is evaluated alongside LoCoMo, where LoCoMo tests factual memory and LoCoMo-Plus tests cognitive memory under cue–trigger semantic disconnect. These numbers show that LoCoMo-Plus exposes a large gap between robust factual recall and fragile cognitive constraint preservation.
BENCHMARK
LoCoMo-Plus is evaluated alongside LoCoMo, where LoCoMo tests factual memory and LoCoMo-Plus tests cognitive memory under cue–trigger semantic disconnect. These numbers show that LoCoMo-Plus exposes a large gap between robust factual recall and fragile cognitive constraint preservation.
BENCHMARK
Average accuracy on LoCoMo (factual memory) versus LoCoMo-Plus (cognitive memory) for selected systems.
BENCHMARK
Accuracy on LoCoMo-Plus cognitive memory tasks for selected systems, showing the performance drop from factual LoCoMo.
KEY INSIGHT
Even the strongest system, gemini-2.5-pro, achieves 71.78% average accuracy on LoCoMo but only 26.06% on LoCoMo-Plus cognitive memory tasks.
This is surprising because developers often assume that better long-context factual recall automatically yields strong cognitive memory, but LoCoMo-Plus shows these abilities can diverge sharply.
WHY IT MATTERS
LoCoMo-Plus unlocks a way to stress-test agents on implicit constraint preservation, not just explicit fact retrieval, using realistic cue–trigger disconnects.
Builders can now design and compare memory architectures that explicitly target cognitive behaviors like respecting user goals and values over time, which was previously impractical to measure.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xingyu Lyu, Jianfeng He et al.
· 2026
ADAM combines Anchor extraction, Distribution estimation, Anchor selection, and Query generation to adaptively probe agent memory via an auxiliary generator and entropy based selection. On the EHRAgent benchmark with Llama2-7b-chat, ADAM reaches EQ=77 and ASR=1.00, compared to MEXTRA’s EQ=44 and ASR=0.89.
Shannan Yan, Jingchen Ni et al.
· 2026
AdaMem organizes dialogue history into Working Memory, Episodic Memory, Persona Memory, and Graph Memory coordinated by a Memory Agent, Research Agent, and Working Agent. On LoCoMo with GPT-4.1-mini, AdaMem achieves 44.65 F1 overall, beating the best baseline LangMem at 41.76 F1 by +2.89.
Answers use this explainer on Memory Papers.
Checking…