HiMem: Hierarchical Long-Term Memory for LLM Long-Horizon Agents

AuthorsNingning Zhang, Xingxing Yang, Zhizhong Tan et al.

2026

TL;DR

HiMem uses hierarchical Episode Memory plus Note Memory with conflict-aware Memory Reconsolidation to reach 80.71% GPT-Score on LoCoMo vs 69.03% for SeCom (+11.68 points).

SharePost on XLinkedIn

Read our summary here, or open the publisher PDF on the next tab.

THE PROBLEM

Long-horizon agents lose context and semantic alignment over extended dialogues

Existing long-term memory systems still show clear limitations in adaptability, scalability, and self-evolution under continuous interaction settings.

When long-horizon conversational agents lose semantic alignment, they mis-handle temporal references and coreference, degrading long-term consistency and personalization over sustained interactions.

HOW IT WORKS

HiMem — Hierarchical Episode and Note Memory with Reconsolidation

HiMem’s core mechanism combines Episode Memory, Note Memory, Knowledge Alignment, and Memory Reconsolidation into a hierarchical long-term memory framework for dialogues.

You can think of HiMem like a brain with a hippocampus for detailed events and a card catalog of abstract notes, linked through alignment and reconsolidation.

This hierarchical design lets HiMem keep both concrete episodes and abstract knowledge, enabling conflict-aware updates that a plain context window cannot support.

DIAGRAM

Best Effort Retrieval and Self-Evolution Flow

This diagram shows how HiMem performs best-effort retrieval from Note Memory and Episode Memory and triggers Memory Reconsolidation when evidence is insufficient.

DIAGRAM

LoCoMo Evaluation and Ablation Design

This diagram shows how HiMem is evaluated on LoCoMo with different memory configurations and metrics.

PROCESS

How HiMem Handles a Long Horizon Dialogue Session

  1. 01

    Memory Construction

    HiMem first runs Dual Channel Segmentation to build Episode Memory and applies Multi Stage Information Extraction to populate Note Memory from dialogue logs.

  2. 02

    Knowledge Alignment

    HiMem performs Knowledge Alignment to normalize time, resolve coreference, and link Episode Memory with Note Memory in a unified semantic space.

  3. 03

    Memory Retrieval

    HiMem uses the hierarchical memory retrieval module, starting with Note First Retrieval and optionally descending to Episode Retrieval under best effort retrieval.

  4. 04

    Memory Reconsolidation

    When Note First Retrieval fails but episodes suffice, HiMem triggers Memory Reconsolidation to ADD, UPDATE, or DELETE notes based on episodic evidence.

KEY CONTRIBUTIONS

Key Contributions

  • 01

    Hierarchical Long Term Memory Framework

    HiMem integrates Episode Memory and Note Memory into a hierarchical structure that bridges concrete events and abstract knowledge for long horizon conversational agents.

  • 02

    Topic Aware Dual Channel Segmentation

    HiMem introduces Topic Aware Event Surprise Dual Channel Segmentation and Multi Stage Information Extraction to build cognitively consistent Episode Memory and Note Memory.

  • 03

    Conflict Aware Memory Reconsolidation

    HiMem designs conflict aware Memory Reconsolidation with ADD, UPDATE, DELETE operations, improving Note Memory GPT Score by about 5.85% when Memory Self Evolution is enabled.

RESULTS

By the Numbers

Overall GPT-Score

80.71% (mean)

+11.68 over SeCom

Overall F1

34.95% (mean)

vs SeCom 16.77% overall F1

Multi Hop GPT-Score

70.92% (mean)

+11.82 over Mem0 on Multi Hop

Temporal GPT-Score

74.77% (mean)

+6.23 over Mem0 on Temporal

On the LoCoMo long horizon dialogue benchmark, which averages 600 turns and 16K tokens per session, HiMem’s 80.71% overall GPT-Score shows more reliable long term reasoning than SeCom and Mem0. These gains demonstrate that HiMem’s hierarchical memory and reconsolidation design improves multi hop and temporal reasoning under extended interactions.

BENCHMARK

By the Numbers

On the LoCoMo long horizon dialogue benchmark, which averages 600 turns and 16K tokens per session, HiMem’s 80.71% overall GPT-Score shows more reliable long term reasoning than SeCom and Mem0. These gains demonstrate that HiMem’s hierarchical memory and reconsolidation design improves multi hop and temporal reasoning under extended interactions.

BENCHMARK

Performance comparison of HiMem and baseline methods on LoCoMo

Overall GPT-Score on LoCoMo for HiMem and baseline long term memory systems.

BENCHMARK

Ablation study on memory components in HiMem

Overall GPT-Score for HiMem and its variants without Episode Memory or Note Memory.

KEY INSIGHT

The Counterintuitive Finding

Disabling Knowledge Alignment for Episode Memory slightly improves average GPT-Score from 78.12% to 79.63%, despite alignment helping Note Memory substantially.

This is surprising because you might expect more alignment to always help, but HiMem shows that over aligning episodic context can dilute useful raw dialogue signals.

WHY IT MATTERS

What this unlocks for the field

HiMem unlocks long horizon agents that maintain both detailed episodic histories and compact, self evolving knowledge without sacrificing retrieval efficiency.

Builders can now deploy conversational agents that adapt their Note Memory over months of interaction, correcting contradictions instead of blindly accumulating stale or inconsistent facts.

~14 min read← Back to papers

Related papers

Agent MemoryLong-Term Memory

Adaptive Memory Admission Control for LLM Agents

Guilin Zhang, Wei Jiang et al.

· 2026

A-MAC scores candidate memories using Utility, Confidence, Novelty, Recency, and Type Prior combined by a learned linear admission policy with Algorithm 1 A-MAC Memory Admission. On the LoCoMo benchmark, A-MAC achieves F1 0.583 and 2644 ms latency, improving F1 by 0.042 and reducing latency by 1187 ms compared to A-mem.

Long-Term Memory

Advancing Open-source World Models

Robbyant Team, Zelin Gao et al.

arXiv 2026 · 2026

LingBot-World combines a Data Engine, Fundamental World Model, Action-Conditioned World Model, and Post-Training causal adaptation to turn a 28B-parameter video generator into a real-time interactive world simulator. On the VBench benchmark, LingBot-World achieves a dynamic degree of 0.8857 versus 0.7612 for Yume-1.5, while also improving imaging quality to 0.6683.

BenchmarkBenchmarkLong-Term Memory

AgenticAI-DialogGen: Topic-Guided Conversation Generation for Fine-Tuning and Evaluating Short- and Long-Term Memories of LLMs

Manoj Madushanka Perera, Adnan Mahmood et al.

· 2026

AgenticAI-DialogGen chains ChatPreprocessor, KnowledgeExtractor, TopicAnalyzer, KnowledgeGraphBuilder, PersonaGenerator, DuelingChat Agent, ConversationValidator, ConversationRefiner, QAGeneration, and PostProcessing to turn raw multi-session chats into topic-guided, persona-grounded conversations with explicit short- and long-term memories. On the TGC / KG memory QA benchmark, Mistral-7B fine-tuned within AgenticAI-DialogGen achieves 87.36 F1, compared to GPT-4’s 83.77 F1 in a zero-shot setting on the same task.

Questions about this paper?

Paper: HiMem: Hierarchical Long-Term Memory for LLM Long-Horizon Agents

Answers use this explainer on Memory Papers.

Checking…