Echo: A Large Language Model with Temporal Episodic Memory

AuthorsWenTao Liu, Ruohua Zhang, Aimin Zhou et al.

2025

TL;DR

Echo uses a user–time–assistant training paradigm with EM-Train and EM-Test to reach 6.7 vs 5.8 human score on easy episodic memory queries (+0.9 over GPT-4).

SharePost on XLinkedIn

Read our summary here, or open the publisher PDF on the next tab.

THE PROBLEM

LLMs lack episodic memory for multi-turn dialogue tasks

Most LLM work focuses on semantic memory-based question answering, neglecting episodic memory-related queries, especially in multi-turn scenarios.

Even advanced systems like GPT-4 still show logical inconsistencies and hallucinations in episodic memory tasks, harming emotional companionship, personal AI assistants, and AI teachers.

HOW IT WORKS

Multi-Agent Data Generation Framework and temporal training for Echo

Echo centers on Multi-Agent Data Generation Framework (MADGF), EM-Train, EM-Test, and a modified user–time–assistant training paradigm to inject temporal episodic memory into ChatGLM3-6B.

Think of MADGF as a scripted stage play: characters, plots, and environments coordinate multiple agents to act out rich, time-stamped life stories that Echo later learns from.

By inserting explicit observation time stamps between user and assistant turns, Echo learns to reconstruct events over “just now” to “several decades” spans in ways a plain context window cannot.

DIAGRAM

Echo’s episodic dialogue generation loop in MADGF

This diagram shows how MADGF alternates human and Echo agents, injects time observations, and produces EM-Train dialogues.

DIAGRAM

Echo training and evaluation pipeline with temporal observations

This diagram shows how EM-Train and EM-Test feed into Echo’s user–time–assistant training paradigm and evaluation.

PROCESS

How Echo Handles an EM-Test multi-turn episodic memory query

  1. 01

    Multi-Agent Data Generation Framework

    Echo uses MADGF with designed characters, plots, and environments to simulate multi-turn human–assistant dialogues enriched with episodic events.

  2. 02

    EM-Train construction

    Echo aggregates 15,533 dialogues from MADGF, averaging 16.75 rounds and 8,597 characters, and removes assistant prompts Pa to form EM-Train.

  3. 03

    User–time–assistant training paradigm

    Echo fine-tunes ChatGLM3-6B with EM-Train using a user–time–assistant chat template where observation tokens carry temporal information but do not update gradients.

  4. 04

    EM-Test evaluation

    Echo receives full historical dialogue plus a tagged Observation and Question, then is scored by humans and a Sentence Transformer similarity metric across eight time spans and two difficulty levels.

KEY CONTRIBUTIONS

Key Contributions

  • 01

    Multi-Agent Data Generation Framework (MADGF)

    Echo introduces MADGF to simulate multi-turn episodic dialogues, yielding EM-Train with 15,533 entries, 16.75 rounds per entry, and 8,597 characters on average.

  • 02

    Temporal user–time–assistant training paradigm

    Echo modifies the standard user–assistant template by inserting a non-trainable observation role carrying explicit time stamps for each dialogue round.

  • 03

    EM-Test and EM-Test-Without-Time benchmarks

    Echo is evaluated on EM-Test with 106 time-tagged test points across eight time spans and on EM-Test-Without-Time with 123 points, revealing improved episodic memory.

RESULTS

By the Numbers

Human Score Easy Overall

6.7

+0.9 over GPT-4 (5.8)

Human Score Hard Overall

5.9

+0.4 over LLAMA3-8B (5.5)

Similarity Metric Easy Overall

84.0

+9.2 over GPT-3.5-turbo (74.8)

Similarity Metric Hard Overall

74.5

+6.8 over GPT-4 (67.7)

Echo is evaluated on EM-Test, a 106-point multi-turn episodic memory benchmark with eight time spans and two difficulty levels. These numbers show Echo turning ChatGLM3-6B from the worst baseline (2.7 easy human score) into the best episodic memory system on both human and similarity metrics.

BENCHMARK

By the Numbers

Echo is evaluated on EM-Test, a 106-point multi-turn episodic memory benchmark with eight time spans and two difficulty levels. These numbers show Echo turning ChatGLM3-6B from the worst baseline (2.7 easy human score) into the best episodic memory system on both human and similarity metrics.

BENCHMARK

EM-Test Overall Human Scores (Easy Level)

Average human score (1–10) on EM-Test easy-level episodic memory questions.

KEY INSIGHT

The Counterintuitive Finding

Echo’s similarity-based automatic metric correlates with human scores at R = 0.935 on easy EM-Test and R = 0.842 on hard EM-Test.

This is surprising because semantic similarity was expected to miss nuanced episodic reasoning, yet it tracks human judgments closely enough to replace costly manual scoring.

WHY IT MATTERS

What this unlocks for the field

Echo shows that injecting explicit time observations and training on EM-Train can give LLMs constructive, human-like episodic memory over months and decades.

Builders can now design assistants, counselors, and teachers that reliably recall personal histories across 8 time spans, instead of relying on brittle semantic memory and external databases.

~12 min read← Back to papers

Related papers

BenchmarkAgent Memory

Active Context Compression: Autonomous Memory Management in LLM Agents

Nikhil Verma

· 2026

Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.

Questions about this paper?

Paper: Echo: A Large Language Model with Temporal Episodic Memory

Answers use this explainer on Memory Papers.

Checking…