LightMem: Lightweight and Efficient Memory-Augmented Generation

THE PROBLEM

LLM agents waste tokens and miss context in long interactions

HOW IT WORKS

LightMem — sensory compression, topic-aware STM, and sleep-time LTM

DIAGRAM

Online interaction and memory update flow in LightMem

This diagram shows how LightMem processes each dialogue turn online, performs soft LTM inserts, and later runs offline sleep-time updates.

DIAGRAM

Evaluation setup for LightMem on LongMemEval and LoCoMo

This diagram shows how LightMem is evaluated across datasets, backbones, and baselines, and where summary and update costs are measured.

PROCESS

How LightMem Handles an Incremental Dialogue Turn Feeding Session

1. 01

   Pre-Compressing Submodule

   LightMem uses the Pre-Compressing Submodule in Light1 with LLMLingua-2 to retain only high-information tokens based on retention probabilities and cross-entropy.

2. 02

   Topic Segmentation Submodule

   LightMem applies the Topic Segmentation Submodule, combining attention-based boundaries B1 and similarity-based boundaries B2 to form topic segments B in the sensory buffer.

3. 03

   Light2 Topic-Aware Short-Term Memory

   LightMem groups topic segments into the STM buffer, and when token capacity th is reached, LightMem calls fsum to summarize segments into sumi and embeddings ei.

4. 04

   Light3 Long-Term Memory with Sleep-Time Update

   LightMem inserts new entries softly into LTM at test time, then later builds update queues Q(ei) and runs offline parallel fupdate operations during sleep-time.

KEY CONTRIBUTIONS

Key Contributions

• 01

  Light1: Cognitive-Inspired Sensory Memory

  LightMem introduces a Light1 Sensory Memory Module with a Pre-Compressing Submodule and Topic Segmentation Submodule, using LLMLingua-2 to cut prompt tokens by up to 50–60% without hurting QA accuracy.

• 02

  Light2: Topic-Aware Short-Term Memory

  LightMem designs Light2 STM to group turns into topic segments, reducing summarization calls from O(N) to O(N r x T / th) and improving segmentation accuracy above 80% across compression ratios.

• 03

  Light3: Long-Term Memory with Sleep-Time Update

  LightMem proposes Light3 with soft online inserts and offline parallel updates, achieving up to 105.9× online token reduction and 159.4× fewer API calls on LongMemEval-S with GPT-4o-mini.

RESULTS

By the Numbers

On LongMemEval-S, which tests long-horizon conversational QA, LightMem with Qwen3-30B-A3B-Instruct-2507 reaches 70.20% ACC versus 65.20% for A-MEM while using far fewer tokens and API calls. On LoCoMo, which stresses long-context memory and reasoning, LightMem with GPT-4o-mini attains 72.99% ACC compared to 64.16% for A-MEM, confirming that LightMem’s lightweight memory pipeline improves both effectiveness and efficiency.

BENCHMARK

By the Numbers

On LongMemEval-S, which tests long-horizon conversational QA, LightMem with Qwen3-30B-A3B-Instruct-2507 reaches 70.20% ACC versus 65.20% for A-MEM while using far fewer tokens and API calls. On LoCoMo, which stresses long-context memory and reasoning, LightMem with GPT-4o-mini attains 72.99% ACC compared to 64.16% for A-MEM, confirming that LightMem’s lightweight memory pipeline improves both effectiveness and efficiency.

BENCHMARK

Effectiveness comparison on LongMemEval-S with GPT-4o-mini

ACC (%) on LongMemEval-S for LightMem and key memory baselines using GPT-4o-mini.

BENCHMARK

Effectiveness comparison on LoCoMo with GPT-4o-mini

ACC (%) on LoCoMo for LightMem and key memory baselines using GPT-4o-mini.

KEY INSIGHT

WHY IT MATTERS

What this unlocks for the field