HiGMem: A Hierarchical and LLM-Guided Memory System for Long-Term Conversational Agents

AuthorsShuqi Cao, Jingyi He, Fei Tan

2026

TL;DR

HiGMem uses an LLM-guided event–turn hierarchy to filter memories, boosting adversarial F1 from 0.54 to 0.78 on LoCoMo10 while retrieving 8.09 vs 99.84 turns.

SharePost on XLinkedIn

Read our summary here, or open the publisher PDF on the next tab.

THE PROBLEM

Bloated evidence sets in long conversations hurt precision and cost

Existing memory systems relying on vector similarity produce bloated evidence sets, where extra similar turns yield little recall but erode precision and inflate context.

In long-term conversational agents, this makes retrieval precision drop, raises answer-stage context cost, and leaves evidence harder to inspect and manage over extended histories.

HOW IT WORKS

HiGMem: Hierarchical and LLM-Guided Memory System

HiGMem’s core mechanism combines a Turn Layer, Event Layer, Memory Construction Mechanism, and LLM-Guided Retrieval to structure and filter conversational memories.

You can think of HiGMem like a library card catalog: events are shelves with concise cards, and turns are the individual books the librarian pulls on demand.

This event–turn hierarchy lets HiGMem reason over summaries to pick only turns worth reading, something a plain context window or pure vector store cannot achieve.

DIAGRAM

HiGMem Memory Construction and Event Affiliation Flow

This diagram shows how HiGMem constructs Turn nodes, affiliates them to Event nodes, and updates summaries and links when a new dialogue turn arrives.

DIAGRAM

HiGMem Query-Time LLM-Guided Retrieval Pipeline

This diagram shows how HiGMem processes a query, retrieves semantic events and turns, predicts useful turns, filters candidates, and generates the final response.

PROCESS

How HiGMem Handles a Question Answering Session

  1. 01

    Memory Construction Mechanism

    HiGMem uses the Memory Construction Mechanism to analyze each new dialogue turn with a sliding window and build enriched Turn nodes with metadata.

  2. 02

    Event Affiliation and Update

    HiGMem performs Event Affiliation and Update by embedding Turn nodes, selecting top-kevent Event candidates, and refreshing or appending summaries and fact sheets based on volume threshold τ.

  3. 03

    LLM-Guided Retrieval

    During LLM-Guided Retrieval, HiGMem generates query keywords, retrieves semantic Turn and Event nodes, and lets the LLM reason over event-linked turns to form Tpred.

  4. 04

    Final Evidence Set and Response

    HiGMem merges Tsemantic and Tpred into Tcand, applies an LLM-based Filter to obtain Tfinal, and then feeds Tfinal with the query into GPT-4o-mini or GPT-5 to generate the final response.

KEY CONTRIBUTIONS

Key Contributions

  • 01

    Hierarchical event–turn memory architecture

    HiGMem introduces a two-level Turn Layer and Event Layer with explicit links, enabling multi-granularity reasoning and improving F1 from 0.39 to 0.49 over a non-hierarchical variant.

  • 02

    LLM-guided Hierarchical Retrieval strategy

    HiGMem’s LLM-Guided Retrieval uses event summaries as semantic anchors to predict useful turns, reaching adversarial F1 0.78 vs 0.54 for A-Mem on LoCoMo10.

  • 03

    Efficient long-term conversational memory baseline

    HiGMem provides an open-source system that retrieves on average 8.09 turns with Precision@K 0.1909 while keeping Recall@K 0.7241, compared to A-Mem’s 99.84 turns and Precision@K 0.0101.

RESULTS

By the Numbers

Adversarial F1

0.78

+0.24 over A-Mem

Single-Hop F1

0.49

+0.07 over A-Mem

Avg K

8.09 turns

−91.75 turns vs A-Mem

Precision@K

0.1909

+0.1808 over A-Mem

On the LoCoMo10 benchmark, which tests long multi-session conversational QA across Single-Hop, Multi-Hop, Temporal, Open-Domain, and Adversarial questions, HiGMem shows that hierarchical LLM-guided memory can raise adversarial F1 to 0.78 while shrinking the evidence set from 99.84 to 8.09 turns.

BENCHMARK

By the Numbers

On the LoCoMo10 benchmark, which tests long multi-session conversational QA across Single-Hop, Multi-Hop, Temporal, Open-Domain, and Adversarial questions, HiGMem shows that hierarchical LLM-guided memory can raise adversarial F1 to 0.78 while shrinking the evidence set from 99.84 to 8.09 turns.

BENCHMARK

Experimental results on the LoCoMo10 dataset across five question categories

F1 score on LoCoMo10 adversarial questions.

BENCHMARK

Retrieval effectiveness on LoCoMo10

Average number of turns in the final evidence set (Avg K).

KEY INSIGHT

The Counterintuitive Finding

HiGMem retrieves only 8.09 turns on average yet maintains Recall@K 0.7241, nearly matching A-Mem’s 0.7502 with 99.84 turns.

This is surprising because many assume higher recall requires flooding the answer model with more context, but HiGMem shows LLM-guided selection can keep recall high with far fewer turns.

WHY IT MATTERS

What this unlocks for the field

HiGMem unlocks compact, high-precision evidence retrieval for long-term conversational agents, enabling multi-session reasoning without overwhelming answer-stage context windows.

Builders can now design agents that maintain rich long-term memory while keeping token cost and latency manageable, making hybrid deployments with expensive answer models like GPT-5 practical.

~10 min read← Back to papers

Related papers

BenchmarkAgent Memory

Active Context Compression: Autonomous Memory Management in LLM Agents

Nikhil Verma

· 2026

Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.

Agent Memory

ActMem: Bridging the Gap Between Memory Retrieval and Reasoning in LLM Agents

Xiaohui Zhang, Zequn Sun et al.

· 2026

ActMem transforms dialogue history into atomic facts via Memory Fact Extraction, groups them with Fact Clustering, links them through a Memory KG Construction module, and uses Counterfactual-based Retrieval and Reasoning for action-aware answers. On ActMemEval, ActMem reaches 76.52% QA accuracy with DeepSeek-V3, beating LightMem’s 63.97% by 12.55 points and NaiveRAG’s 61.54%.

Questions about this paper?

Paper: HiGMem: A Hierarchical and LLM-Guided Memory System for Long-Term Conversational Agents

Answers use this explainer on Memory Papers.

Checking…