Memory Papers
Memory Architecture

MemRouter: Memory-as-Embedding Routing for Long-Term Conversational Agents

AuthorsTianyu Hu, Weikai Lin, Weizhi Zhang et al.

2026
arXivPDF

TL;DR

MemRouter uses an embedding-routed Memory Router Architecture on a frozen LLM backbone to reach 52.0 F1 vs 45.6 for an LLM manager on LoCoMo while cutting write-side p50 latency from 970 ms to 58 ms.

SharePost on XLinkedIn

Read our summary here, or open the publisher PDF on the next tab.

THE PROBLEM

Write-side memory management dominated by 1,200 LLM calls per 600-turn chat

Long-term agents currently require full LLM generation for each memory decision, with Memory-R1 needing about 1,200 generation calls for a single 600-turn conversation.

This per-turn overhead makes memory admission more expensive than answering questions, limiting real-world long-horizon QA deployments and coupling memory policies tightly to specific LLM backbones.

HOW IT WORKS

MemRouter: Embedding-based Memory Router Architecture

MemRouter centers on a Memory Router Architecture that uses Contextual Embedding and Projection, Backbone Contextualization, and Operation Classification to decide ADD or NOOP for each turn.

Think of MemRouter as a smart RAM controller: embeddings act like cache lines, the frozen backbone is the CPU, and the router heads are a hardware gate deciding which lines persist to disk-like long-term memory.

This Memory-as-Embedding Routing lets MemRouter learn write-side admission with only ∼12M trainable parameters, avoiding per-turn decoding while enabling backbone-agnostic reuse that plain context windows and LLM managers cannot provide.

DIAGRAM

Write Path vs Read Path in MemRouter

This diagram shows how MemRouter separates the per-turn write path from the question-time read and answer path.

DIAGRAM

MemRouter Evaluation and Baseline Comparison Pipeline

This diagram shows how MemRouter is trained, then evaluated under a matched harness against an LLM-based memory manager on LoCoMo.

PROCESS

How MemRouter Handles a Long-Term Conversational QA Session

  1. 01

    Contextual Embedding and Projection

    MemRouter chunks recent dialogue into up to 13 context segments and encodes them with BGE-large-en-v1.5, then uses the Projection Module to map 1024-d embeddings into the 3584-d backbone space.

  2. 02

    Backbone Contextualization

    MemRouter feeds projected chunk embeddings directly into the frozen Qwen2.5-7B transformer body, letting Backbone Contextualization capture dependencies between the current turn and its recent conversational history.

  3. 03

    Operation Classification

    MemRouter applies lightweight Operation Classification heads over the current chunk representation to predict ADD vs NOOP and the content type among key_facts, emotional, preference, plan, and routine.

  4. 04

    Memory Store and Retrieval plus Answer Agent

    When MemRouter decides ADD, it writes mt tuples into the Memory Store, later using hybrid dense plus BM25 retrieval to feed top-60 memories to the Answer Agent for category-specific prompted QA.

KEY CONTRIBUTIONS

Key Contributions

  • 01

    Embedding-based Memory Router Architecture

    MemRouter introduces a Memory Router Architecture that replaces per-turn decoding with Contextual Embedding and Projection, Backbone Contextualization, and Operation Classification, training only ∼12M parameters on top of a frozen 7B backbone.

  • 02

    Supervised Router Training Pipeline

    MemRouter uses teacher-generated turn-level labels from Qwen3.5-35B-A3B over LoCoMo, LongMemEval, and MSC, preserving clean conversation-level splits and avoiding reinforcement learning instability.

  • 03

    Controlled LoCoMo Evaluation and Factor Analysis

    MemRouter is evaluated under a matched harness on LoCoMo, showing 52.0 vs 45.6 overall F1 against an LLM manager and descriptive factor averaging where learned admission adds +10.3 F1 over random storage.

RESULTS

By the Numbers

Overall F1

52.0 F1

+8.9 over Memory-R1-GRPO

Single hop F1

57.5 F1

+23.9 over LoCoMo RAG

Multi hop F1

52.4 F1

+28.8 over Memory-R1-GRPO

Memory-management p50 latency

58 ms

-912 ms vs LLM manager p50 970 ms

On the LoCoMo long-term conversational QA benchmark, MemRouter with Qwen2.5-7B-Instruct reaches 52.0 overall F1 and 57.5 single-hop F1, compared to 43.1 overall F1 for Memory-R1-GRPO and 9.0 for LoCoMo RAG. These results show that MemRouter’s embedding-based admission policy improves accuracy while reducing write-side latency from 970 ms to 58 ms in the matched-harness comparison.

BENCHMARK

By the Numbers

On the LoCoMo long-term conversational QA benchmark, MemRouter with Qwen2.5-7B-Instruct reaches 52.0 overall F1 and 57.5 single-hop F1, compared to 43.1 overall F1 for Memory-R1-GRPO and 9.0 for LoCoMo RAG. These results show that MemRouter’s embedding-based admission policy improves accuracy while reducing write-side latency from 970 ms to 58 ms in the matched-harness comparison.

BENCHMARK

End-to-end LoCoMo F1 with Qwen2.5-7B-Instruct

Overall F1 on LoCoMo test set (excluding adversarial questions) for MemRouter and baselines.

KEY INSIGHT

The Counterintuitive Finding

MemRouter’s descriptive factor averaging shows learned admission improves mean F1 by +10.3 over random storage, while retrieval contributes only +0.7 F1 despite sophisticated hybrid scoring.

This is surprising because many systems focus on retrieval tricks, yet MemRouter reveals that write-side admission decisions dominate performance gains compared to incremental retrieval improvements.

WHY IT MATTERS

What this unlocks for the field

MemRouter unlocks backbone-agnostic, low-latency memory admission, letting agents process hundreds of conversational turns using embedding routing instead of expensive per-turn generation.

Builders can now attach MemRouter as a reusable write-side module, swap answer backbones like Qwen2.5-7B or Qwen3.5-35B-A3B without retraining memory policies, and scale long-horizon conversational QA with controllable storage budgets.

~12 min read← Back to papers

Related papers

Memory Architecture

A Control Architecture for Training-Free Memory Use

Yanzhen Lu, Muchen Jiang et al.

· 2026

TAG routes low-confidence steps to uncertainty-based routing, filters them with guarded acceptance with rollback, chooses between bank selection across rule and exemplar memory, and prunes via evidence-based retirement inside a unified control loop. On SVAMP and ASDiv, TAG reaches 81.0% and 85.2% accuracy, improving over the 74.0% and 77.5% no-memory baselines while a compute-matched Retry baseline stays flat.

arXiv:2604.18206Read explainer

Questions about this paper?

Paper: MemRouter: Memory-as-Embedding Routing for Long-Term Conversational Agents

Answers use this explainer on Memory Papers.

Checking…