AuthorsXiucheng Xu, Bingbing Xu, Xueyun Tian et al.
TL;DR
Chain-of-Memory (CoM) uses Dynamic Memory Chain Evolution with adaptive truncation to reach 76.40% accuracy on LongMemEval, +10.40 points over turn-level RAG with Qwen3-32B.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Existing structured memories push token consumption up to 332k on LongMemEval while yielding negligible accuracy gains over naive RAG baselines.
On long-term QA for LLM agents, such heavy memory construction plus context concatenation inflates latency and still leaves a gap between retrieval recall and reasoning accuracy.
HOW IT WORKS
Chain-of-Memory (CoM) builds on Memory Construction and Retrieval, Dynamic Memory Chain Evolution, State-Aware Gating Evolution, and Adaptive Path Truncation to organize flat turns into chains.
You can think of Chain-of-Memory (CoM) like a librarian building a custom reading path from index cards, pruning cards once their relevance drops off a cliff.
This design lets Chain-of-Memory (CoM) maintain long-horizon, multi-hop reasoning that a plain context window or naive retrieve-and-concatenate RAG cannot sustain.
DIAGRAM
This diagram shows how Chain-of-Memory (CoM) evolves multiple memory chains from top K retrieved nodes using state-aware gating and adaptive path truncation.
DIAGRAM
This diagram shows how Chain-of-Memory (CoM) is evaluated across LongMemEval and LoCoMo with different backbones, metrics, and ablations.
PROCESS
Memory Construction and Retrieval
Chain-of-Memory (CoM) turns each conversation turn into a memory node with text, role, time, and embedding, then retrieves top K nodes via semantic similarity.
Dynamic Memory Chain Evolution
Chain-of-Memory (CoM) initializes L chains from the top L anchors and iteratively extends each using Dynamic Memory Chain Evolution over the candidate pool.
State-Aware Gating Evolution
Chain-of-Memory (CoM) uses State-Aware Gating Evolution to score candidates by multiplying global relevance to the query and contextual consistency with the current chain.
Adaptive Path Truncation
Chain-of-Memory (CoM) applies Adaptive Path Truncation, stopping chain growth when Sgate drops below β times the previous score, then feeds the refined chains to the backbone LLM.
KEY CONTRIBUTIONS
Paradigm Shifting
Chain-of-Memory (CoM) replaces heavy graph or tree construction with lightweight Memory Construction and Retrieval plus Dynamic Memory Chain Evolution, cutting token usage to about 2.7% of complex architectures.
Methodologically Novel
Chain-of-Memory (CoM) introduces State-Aware Gating Evolution and Adaptive Path Truncation to build coherent inference paths that jointly optimize query relevance and contextual consistency.
Empirically Effective
Chain-of-Memory (CoM) achieves 76.40% accuracy on LongMemEval and 70.97% on LoCoMo with Qwen3-32B, delivering 7.5%–10.4% absolute gains while using only 6.0% of the runtime of complex memory structures.
RESULTS
Accuracy LongMemEval Total
76.40%
+10.40 over RAG (turn) with Qwen3-32B
Accuracy LoCoMo Total
70.97%
+9.09 over RAG (turn) with Qwen3-32B
Token Consumption LongMemEval
8.8k tokens
vs 119.6k for Full-Context with Qwen3-32B
Total Runtime LongMemEval
2002.45 s
vs 32949 s for A-Mem with Qwen3-32B
On LongMemEval and LoCoMo, which test long-term single-hop, multi-hop, temporal, and knowledge QA, Chain-of-Memory (CoM) shows that dynamic memory chains can close much of the reasoning gap while slashing token and time costs compared to Full-Context and structured memories.
BENCHMARK
On LongMemEval and LoCoMo, which test long-term single-hop, multi-hop, temporal, and knowledge QA, Chain-of-Memory (CoM) shows that dynamic memory chains can close much of the reasoning gap while slashing token and time costs compared to Full-Context and structured memories.
BENCHMARK
Total Accuracy (%) on LongMemEval across baselines and Chain-of-Memory (CoM).
BENCHMARK
Total Accuracy (%) with different gating formulations using Qwen3-32B.
KEY INSIGHT
Chain-of-Memory (CoM) uses only about 2.7% of the token consumption and 6.0% of the latency of complex memory architectures while improving accuracy by up to 10.4 points.
This is surprising because prior work assumed that more elaborate trees or graphs were necessary for better reasoning, yet Chain-of-Memory (CoM) shows that smarter utilization of flat memories beats heavier structure.
WHY IT MATTERS
Chain-of-Memory (CoM) unlocks long-horizon, multi-hop, and temporal reasoning for LLM agents using cheap flat memories plus Dynamic Memory Chain Evolution.
Builders can now deploy persistent, high-quality agent memory in real-time systems without paying full-context or graph-construction costs, making long-term interactive agents practically viable.
Xingyu Lyu, Jianfeng He et al.
· 2026
ADAM combines Anchor extraction, Distribution estimation, Anchor selection, and Query generation to adaptively probe agent memory via an auxiliary generator and entropy based selection. On the EHRAgent benchmark with Llama2-7b-chat, ADAM reaches EQ=77 and ASR=1.00, compared to MEXTRA’s EQ=44 and ASR=0.89.
Okan Bursa
· 2026
Adaptive RAG Memory (ARM) augments a standard retriever–generator stack with a Dynamic Embedding Layer and Remembrance Engine that track usage statistics and apply selective remembrance and decay to embeddings. On a lightweight retrieval benchmark, ARM achieves NDCG@5 ≈ 0.9401 and Recall@5 = 1.000 with 22M parameters, matching larger baselines like gte-small while providing the best efficiency among ultra-efficient models.
Zhanghao Hu, Qinglin Zhu et al.
· 2026
xMemory organises agent memory via segments, memory components, groups, and dynamic maintenance with a sparsity plus semantic faithfulness objective. On LoCoMo with GPT-5 nano, xMemory reaches 38.71 BLEU and 50.00 F1 versus Nemori’s 36.65 BLEU and 48.17 F1, while cutting tokens per query from 9154.76 to 6581.20.
Answers use this explainer on Memory Papers.
Checking…