AuthorsChunyu Li, Jingyi Kang, Ding Chen et al.
TL;DR
MemReranker uses Elo/Bradley-Terry calibrated BCE→InfoNCE distillation to transfer LLM reasoning, reaching MAP 0.8043 on LongMemEval with only 4B parameters.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
MemReranker targets the failure where generic rerankers return memories that are “semantically highly relevant yet do not contain the key information needed to answer the question,” with left-skewed scores near 0.00.
In agent memory systems, this miscalibration breaks threshold-based filtering and causes memory hallucinations, so downstream LLM agents still answer incorrectly despite seemingly relevant retrieved dialogue fragments.
HOW IT WORKS
MemReranker combines Qwen3-Reranker, Teacher Label Generation, Pointwise BCE Distillation, Contrastive InfoNCE Fine Tuning, and Multi Turn Dialogue Data Construction to build calibrated, reasoning-aware relevance scoring.
You can think of MemReranker like a smart librarian: LLM teachers debate which book best answers a question, then MemReranker learns their judgment as a compact, well-calibrated index.
This design lets MemReranker reason over temporal constraints and coreference, using calibrated [0,1] scores and instructions, instead of relying on a plain context window and shallow semantic similarity.
DIAGRAM
This diagram shows how MemReranker uses staged distillation from LLM teachers to learn calibrated, reasoning-aware relevance scores.
DIAGRAM
This diagram shows how MemReranker reranks recalled dialogue fragments for a user query in agent memory systems.
PROCESS
Stage 0 General Capability Preservation
MemReranker initializes ranking using Rank-DistiLLM data to preserve Qwen3-Reranker capabilities before memory-specific adaptation.
Stage 1 Teacher Label Generation
MemReranker uses GPT and Qwen ensembles for pairwise document comparisons, then converts preferences into Elo Bradley Terry scores.
Stage 2 Pointwise Distillation
MemReranker regresses teacher soft labels with BCE loss on about 1M general pairs and 50K multi turn dialogue pairs for calibrated scores.
Stage 3 Contrastive Fine Tuning
MemReranker applies listwise InfoNCE on query positive negative tuples to sharpen discrimination in the 0.4–0.6 relevance band.
KEY CONTRIBUTIONS
MemReranker model family for agent memory retrieval
MemReranker introduces 0.6B and 4B rerankers that reach MAP 0.8043 on LongMemEval and 0.7366 on LOCOMO using Qwen3-Reranker foundations.
Two stage BCE and InfoNCE training paradigm
MemReranker combines BCE pointwise distillation with InfoNCE contrastive fine tuning plus Elo Bradley Terry calibration for well distributed scores.
Multi turn dialogue data engineering pipeline
MemReranker builds history distillation, hard negative generation, and instruction augmentation to teach coreference resolution and topic drift modeling.
RESULTS
MAP
0.8043
+0.0784 over Gemini-3-Flash on LongMemEval
MRR
0.8315
+0.0740 over BGE-v2-m3 on LongMemEval
NDCG@10
0.8354
higher than Qwen3-Reranker-8B (0.7551) on LongMemEval
Avg latency
247.2 ms
MemReranker-0.6B vs 1549.2 ms for GPT-4o-mini on 1k-token inputs
On the LongMemEval benchmark of 500 long term memory queries, MemReranker-4B achieves MAP 0.8043 and MRR 0.8315, surpassing Gemini-3-Flash and encoder baselines. These results show MemReranker can deliver LLM-level reasoning for memory retrieval with calibrated scores and much lower latency than GPT-4o-mini.
BENCHMARK
On the LongMemEval benchmark of 500 long term memory queries, MemReranker-4B achieves MAP 0.8043 and MRR 0.8315, surpassing Gemini-3-Flash and encoder baselines. These results show MemReranker can deliver LLM-level reasoning for memory retrieval with calibrated scores and much lower latency than GPT-4o-mini.
BENCHMARK
MAP on LongMemEval (500 queries) for MemReranker and key baselines.
BENCHMARK
MAP on LOCOMO for MemReranker and reranking baselines.
KEY INSIGHT
MemReranker-0.6B, with only 0.6B parameters, reaches MAP 0.7538 on LongMemEval, beating Gemini-3-Flash at MAP 0.7259 and GPT-4o-mini at 0.5684.
This is surprising because conventional wisdom says 7B plus LLM rerankers should dominate, yet MemReranker’s BCE distillation and Elo calibration let a tiny cross encoder match or exceed them.
WHY IT MATTERS
MemReranker unlocks calibrated, reasoning-aware memory retrieval that handles temporal constraints, causal logic, and multi turn coreference at production latencies.
Builders can now deploy long term companion agents with tens of thousands of memories, using MemReranker to reliably filter noise and set thresholds without paying GPT-4o-mini or Gemini-3-Flash inference costs.
Peilin Feng, Suorong Yang, Soujanya Poria
arXiv 2026 · 2026
Σ-Mem maintains historical competence evidence matrices M_p, a peer relationship matrix G, a memory readout r_{p,t}, and residual steering δ_{p,t} to guide multi-agent trust decisions. On the mixed counterfactual benchmark, Σ-Mem raises Qwen3-0.6B accuracy at CF@90 from 46.22% with the base model to 71.10% with Σ w/ G.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xiaohui Zhang, Zequn Sun et al.
· 2026
ActMem transforms dialogue history into atomic facts via Memory Fact Extraction, groups them with Fact Clustering, links them through a Memory KG Construction module, and uses Counterfactual-based Retrieval and Reasoning for action-aware answers. On ActMemEval, ActMem reaches 76.52% QA accuracy with DeepSeek-V3, beating LightMem’s 63.97% by 12.55 points and NaiveRAG’s 61.54%.
Answers use this explainer on Memory Papers.
Checking…