TA-Mem: Tool-Augmented Autonomous Memory Retrieval for LLM in Long-Term Conversational QA

AuthorsMengwei Yuan, Jianan Liu, Jing Yang et al.

2026

TL;DR

TA-Mem uses a tool-augmented autonomous memory retrieval loop over a multi-indexed database to reach 55.95 F1 on LoCoMo temporal questions, +10.10 over Mem0.

SharePost on XLinkedIn

Read our summary here, or open the publisher PDF on the next tab.

THE PROBLEM

Long-Term Conversational QA Needs Flexible Memory Retrieval

Existing memory systems mostly rely on static similarity top-k over embeddings, which introduces information redundancy and limits adaptive access to context.

This rigid retrieval hurts long-term conversational QA, where agents must track temporal and multi-hop dependencies across many sessions without exceeding the context window.

HOW IT WORKS

Tool-Augmented Autonomous Memory Retrieval in TA-Mem

TA-Mem combines an Epsodic Memory Constructor, Multi-Indexed Database with Tools, and Memory Retrieval Agent to build and query structured memory pages from long conversations.

You can think of TA-Mem like a computer with RAM and disk plus a smart card catalog, where tools are query APIs that let the agent navigate memory by keys and embeddings.

This tool-augmented loop lets TA-Mem iteratively explore memory beyond a single context window, deciding when to fetch more pages or finalize answers based on reasoning over retrieved notes.

DIAGRAM

TA-Mem Memory Retrieval Loop Over Tools

This diagram shows how TA-Mem's Memory Retrieval Agent interacts with the multi-indexed database tools over multiple iterations to answer a question.

DIAGRAM

LoCoMo Evaluation and Ablation Pipeline for TA-Mem

This diagram shows how TA-Mem is evaluated on LoCoMo, including baseline comparison and iteration budget ablation.

PROCESS

How TA-Mem Handles a Long-Term Conversational QA Session

  1. 01

    Epsodic Memory Constructor

    TA-Mem uses the Episodic Memory Constructor to detect topic shifts, segment conversation history, and extract structured notes with summaries, keywords, people, facts, events, and tags.

  2. 02

    Multi-Indexed Database with Tools

    TA-Mem stores memory pages in a multi-indexed database keyed by names, tags, keywords, and embeddings, exposing query interfaces as tools like string queries, top-k similarity, and person profile fetches.

  3. 03

    Memory Retrieval Agent

    The Memory Retrieval Agent selects tools based on the user question, issues queries to the database, reasons over returned pages and profiles, and maintains a per-session cache to avoid duplicate context.

  4. 04

    Agentic Loop for QA

    Within the agentic loop, TA-Mem repeatedly decides whether more context is needed, performs additional tool calls up to 7 iterations, and then finalizes the response once sufficient memory has been explored.

KEY CONTRIBUTIONS

Key Contributions

  • 01

    Tool-Augmented Memory Retrieval Framework TA-Mem

    TA-Mem introduces a tool-augmented memory framework where the Memory Retrieval Agent autonomously explores a multi-indexed database instead of relying on static similarity top-k retrieval, achieving 55.95 F1 on temporal questions.

  • 02

    One-Shot Multi-Task Episodic Memory Constructor

    TA-Mem implements a one-shot multi-task prompting mechanism that simultaneously chunks context by topic shift and extracts summaries, keywords, people, facts, events, and tags into episodic memory notes.

  • 03

    Statistical Evaluation on LoCoMo with Tool Usage Analysis

    TA-Mem is evaluated on the LoCoMo dataset, showing varied tool usage distributions across question types and an average 2.71 iterations per question, demonstrating adaptive memory exploration strategies.

RESULTS

By the Numbers

Temporal F1

55.95

+10.10 over Mem0

Temporal BLEU-1

51.47

+10.96 over Mem0

Multi-Hop BLEU-1

27.84

+0.71 over Mem0

Avg Token Usage

3755

vs 1764 Mem0 and 3874 MemoryOS

TA-Mem is evaluated on the LoCoMo dataset, which contains 10 very long-term conversations and 1986 questions across multiple categories. The 55.95 F1 and 51.47 BLEU-1 on temporal questions show that TA-Mem can track time-sensitive events more accurately than Mem0 and MemoryOS while keeping token usage moderate.

BENCHMARK

By the Numbers

TA-Mem is evaluated on the LoCoMo dataset, which contains 10 very long-term conversations and 1986 questions across multiple categories. The 55.95 F1 and 51.47 BLEU-1 on temporal questions show that TA-Mem can track time-sensitive events more accurately than Mem0 and MemoryOS while keeping token usage moderate.

BENCHMARK

Performance Comparison with Existing Benchmarks on LoCoMo Dataset

Temporal F1 on LoCoMo temporal questions.

BENCHMARK

Iteration Budget Ablation on TA-Mem

Success rate vs iteration budget for TA-Mem (described qualitatively in Figure 3).

KEY INSIGHT

The Counterintuitive Finding

TA-Mem reaches a 55.95 F1 and 51.47 BLEU-1 on temporal questions while using only 3755 tokens on average per question.

This is surprising because a multi-iteration agentic loop with up to 7 tool calls would normally be expected to inflate token usage far beyond simpler baselines.

WHY IT MATTERS

What this unlocks for the field

TA-Mem unlocks adaptive, tool-driven memory retrieval where agents can choose between string keys, similarity search, and person profiles to match question structure.

Builders can now design long-term conversational agents that reason over rich episodic memory without flooding the context window, balancing retrieval depth, latency, and token efficiency.

~12 min read← Back to papers

Related papers

BenchmarkAgent Memory

Active Context Compression: Autonomous Memory Management in LLM Agents

Nikhil Verma

· 2026

Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.

Agent Memory

ActMem: Bridging the Gap Between Memory Retrieval and Reasoning in LLM Agents

Xiaohui Zhang, Zequn Sun et al.

· 2026

ActMem transforms dialogue history into atomic facts via Memory Fact Extraction, groups them with Fact Clustering, links them through a Memory KG Construction module, and uses Counterfactual-based Retrieval and Reasoning for action-aware answers. On ActMemEval, ActMem reaches 76.52% QA accuracy with DeepSeek-V3, beating LightMem’s 63.97% by 12.55 points and NaiveRAG’s 61.54%.

Questions about this paper?

Paper: TA-Mem: Tool-Augmented Autonomous Memory Retrieval for LLM in Long-Term Conversational QA

Answers use this explainer on Memory Papers.

Checking…