Rethinking How to Remember: Beyond Atomic Facts in Lifelong LLM Agent Memory

AuthorsJingwei Sun, Jianing Zhu, Jiangchao Yao et al.

arXiv 20262026

TL;DR

TriMem uses raw dialogue links, multi-dimensional facts, and entity profiles with TextGrad-evolved prompts to reach 54.26 F1 on LoCoMo, +3.96 over SimpleMem.

SharePost on XLinkedIn

Read our summary here, or open the publisher PDF on the next tab.

THE PROBLEM

Fact Centric Memory Loses 14.5 Percent of Answer Tokens

Existing fact based memory systems compress dialogues into atomic facts, discarding fine grained details and losing 14.5% more information than original dialogue.

When agents rely only on these lossy facts, long term tasks suffer broken context logic, shallow reasoning on multi evidence questions, and unstable performance across diverse dialogue styles.

HOW IT WORKS

Tri Granularity Memory Architecture

TriMem introduces dense storage with efficient retrieval, scattered fact with integrated profile, and lifelong evolution with optimized prompt as its core memory components.

You can think of TriMem like a library: raw dialogues are the full books, extracted facts are index cards, and profiles are curated biographies that summarize a person’s story.

This design lets TriMem recover precise details from source windows, integrate dispersed facts into entity understanding, and adapt its prompts over time, beyond what a fixed context window can support.

DIAGRAM

TriMem Query Time Memory Retrieval Pipeline

This diagram shows how TriMem analyzes a question, builds a search query, and retrieves facts, raw dialogues, and profiles for reasoning.

DIAGRAM

TriMem Training and Evaluation Loop

This diagram shows how TriMem constructs memory, optimizes prompts with TextGrad, and is evaluated on LoCoMo and PerLTQA.

PROCESS

How TriMem Handles a Long Term Question

  1. 01

    Dialogue Partition

    TriMem applies a sliding window segmentation function to split historical dialogue into overlapping windows, preserving local coherence across turns for later extraction.

  2. 02

    Dense Storage with Efficient Retrieval

    TriMem performs multidimensional extraction on each window, binding facts with a source dialogue identifier and building a structured fact memory bank for similarity based retrieval.

  3. 03

    Scattered Fact with Integrated Profile

    TriMem groups facts by person, then uses a profile prompt to synthesize identity, personality, career, interests, and relationships into entity profiles for deep reasoning.

  4. 04

    Lifelong Evolution with Optimized Prompt

    TriMem evaluates answers against references, uses TextGrad to generate natural language gradients, and iteratively refines extraction and profile prompts without updating model parameters.

KEY CONTRIBUTIONS

Key Contributions

  • 01

    Three Dimensional Memory Perspective

    TriMem revisits agent memory across storage fidelity, indexing efficiency, and reasoning quality, showing extracted facts lose 14.5% more information than raw dialogue and collapse on multi evidence questions.

  • 02

    TriMem Architecture with Entity Profiles

    TriMem introduces entity profile construction and raw dialogue identifiers, enabling integrated understanding and detail recovery, and reaches 43.79 BLEU and 54.26 F1 on LoCoMo with GPT-4.1-mini.

  • 03

    TextGrad Based Prompt Evolution

    TriMem applies TextGrad to jointly optimize extraction and profile prompts over four evolution steps, improving performance until 4 steps and avoiding degradation from excessive refinement.

RESULTS

By the Numbers

Average F1 on LoCoMo GPT-4.1-mini

54.26 F1

+3.96 over SimpleMem

Average BLEU on LoCoMo GPT-4.1-mini

43.79 BLEU

+3.82 over SimpleMem

Average F1 on LoCoMo GPT-5-nano

57.04 F1

+14.39 over SimpleMem

Profile Task Accuracy on PerLTQA Qwen3-8B

92.46 %

+4.34 over SimpleMem

LoCoMo tests multi hop, temporal, open domain, and single hop long term QA, while PerLTQA evaluates profiles, social relationships, events, and dialogues. These results show TriMem’s tri granularity memory and prompt evolution substantially improve long horizon reasoning over baselines like SimpleMem and LightMem.

BENCHMARK

By the Numbers

LoCoMo tests multi hop, temporal, open domain, and single hop long term QA, while PerLTQA evaluates profiles, social relationships, events, and dialogues. These results show TriMem’s tri granularity memory and prompt evolution substantially improve long horizon reasoning over baselines like SimpleMem and LightMem.

BENCHMARK

LoCoMo Average F1 with GPT-4.1-mini

F1 on LoCoMo across memory systems using GPT-4.1-mini.

BENCHMARK

PerLTQA Profile Task Correctness Qwen3-8B

LLM judged correctness on PerLTQA profile questions with Qwen3-8B.

KEY INSIGHT

The Counterintuitive Finding

TriMem shows that adding raw dialogue and profiles, which increases context tokens to about 1.2k, still improves F1 over lighter fact only systems like SimpleMem.

This is surprising because many assume fewer tokens always help reasoning, but TriMem demonstrates that richer, well structured memory can beat aggressively compressed facts despite higher token cost.

WHY IT MATTERS

What this unlocks for the field

TriMem unlocks long term agents that can both recall precise details and reason about people’s identities, personalities, and relationships over many sessions.

Builders can now design memory systems that keep verbatim dialogue, efficient fact indices, and evolving profiles together, enabling deeper social and temporal reasoning without retraining LLM backbones.

~12 min read← Back to papers

Related papers

BenchmarkAgent Memory

Active Context Compression: Autonomous Memory Management in LLM Agents

Nikhil Verma

· 2026

Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.

Agent Memory

ActMem: Bridging the Gap Between Memory Retrieval and Reasoning in LLM Agents

Xiaohui Zhang, Zequn Sun et al.

· 2026

ActMem transforms dialogue history into atomic facts via Memory Fact Extraction, groups them with Fact Clustering, links them through a Memory KG Construction module, and uses Counterfactual-based Retrieval and Reasoning for action-aware answers. On ActMemEval, ActMem reaches 76.52% QA accuracy with DeepSeek-V3, beating LightMem’s 63.97% by 12.55 points and NaiveRAG’s 61.54%.

Questions about this paper?

Paper: Rethinking How to Remember: Beyond Atomic Facts in Lifelong LLM Agent Memory

Answers use this explainer on Memory Papers.

Checking…