AuthorsAlex Mallen, Akari Asai, Victor Zhong et al.
TL;DR
Adaptive Retrieval uses entity popularity and relationship type to decide when to retrieve, boosting POPQA accuracy up to 46.5% while halving GPT‑3 retrieval costs.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
POPQA shows that GPT‑j 6B reaches only 16% accuracy and GPT‑3 davinci‑003 only 19% on the 4,000 least popular questions.
On open-domain QA, this means large LMs memorize popular entities but fail on long-tail factual knowledge, leading to hallucinations and unreliable answers for rare entities.
HOW IT WORKS
Adaptive Retrieval combines parametric knowledge, non-parametric memories, subject entity popularity, and relationship type thresholds to decide when to attach BM25 or Contriever passages.
You can think of Adaptive Retrieval like a cache plus disk: popular facts live in fast LM parameters, while rare facts trigger a slower retrieval system only when needed.
This popularity-aware gating lets Adaptive Retrieval answer long-tail questions that exceed a plain context window, without paying retrieval and latency costs for already-memorized popular knowledge.
DIAGRAM
This diagram shows how Adaptive Retrieval augments questions with BM25 or Contriever passages and feeds them into GPT-Neo or GPT-3 at inference time.
DIAGRAM
This diagram shows how POPQA samples Wikidata triples, converts them to questions, and attaches Wikipedia pageview popularity scores.
PROCESS
Sampling factual knowledge
Adaptive Retrieval is evaluated on POPQA where knowledge triples are sampled from Wikidata and paired with subject entity popularity and relationship type.
Converting triples to questions
Adaptive Retrieval uses natural language templates to turn each subject, relationship, object triple into a question that preserves the original relationship type.
Collect popularity
Adaptive Retrieval queries Wikipedia page views to assign a popularity score to each subject entity, serving as a proxy for web frequency.
Adaptive Retrieval
Adaptive Retrieval applies per-relationship popularity thresholds to decide whether to query BM25 or Contriever before prompting GPT-Neo or GPT-3.
KEY CONTRIBUTIONS
POPQA long-tail benchmark
Adaptive Retrieval is evaluated on POPQA, a 14k-question dataset built from Wikidata triples with 16 relationship types and explicit subject popularity scores.
Memorization analysis by popularity
Adaptive Retrieval reveals that GPT-j 6B reaches 16% and GPT-3 davinci-003 19% accuracy on the 4,000 least popular POPQA questions, showing limited long-tail memorization.
Adaptive Retrieval for parametric and non-parametric memories
Adaptive Retrieval introduces popularity-based thresholds that combine parametric knowledge with BM25 or Contriever, improving POPQA accuracy by up to 10% and halving GPT-3 API costs.
RESULTS
Accuracy POPQA
46.5%
+5.3 over retrieval-only GPT-3 davinci-003 with GenRead and Contriever
Accuracy least popular POPQA
19%
GPT-3 davinci-003 vs 16% GPT-j 6B on 4,000 least popular questions
Accuracy vanilla POPQA
35%
GPT-3 davinci-003 closed-book without retrieval
Accuracy GPT-Neo 20B POPQA
25%
GPT-Neo 20B closed-book vs GPT-3 davinci-003 35%
These numbers come from POPQA, a 14k-question open-domain QA benchmark targeting long-tail entities. The results show that Adaptive Retrieval raises GPT-3 davinci-003 from 35% closed-book to 46.5% with popularity-aware retrieval, while long-tail accuracy remains only 19% even for GPT-3 without retrieval.
BENCHMARK
These numbers come from POPQA, a 14k-question open-domain QA benchmark targeting long-tail entities. The results show that Adaptive Retrieval raises GPT-3 davinci-003 from 35% closed-book to 46.5% with popularity-aware retrieval, while long-tail accuracy remains only 19% even for GPT-3 without retrieval.
BENCHMARK
Accuracy on POPQA for GPT-3 davinci-003 with different parametric and non-parametric memory configurations.
KEY INSIGHT
Adaptive Retrieval shows that retrieval can hurt GPT-3 davinci-003 on about 10% of POPQA questions it would otherwise answer correctly.
This is surprising because non-parametric memories are often assumed to be universally helpful, but low recall@1 of 0.14 on these questions shows misleading passages can override correct parametric knowledge.
WHY IT MATTERS
Adaptive Retrieval gives practitioners a simple, popularity-based switch to combine parametric and non-parametric memories without fully trusting either.
Builders can now deploy retrieval-augmented QA systems that save latency and API cost on popular facts while still handling long-tail questions that closed-book LMs consistently miss.
Xingyu Lyu, Jianfeng He et al.
· 2026
ADAM combines Anchor extraction, Distribution estimation, Anchor selection, and Query generation to adaptively probe agent memory via an auxiliary generator and entropy based selection. On the EHRAgent benchmark with Llama2-7b-chat, ADAM reaches EQ=77 and ASR=1.00, compared to MEXTRA’s EQ=44 and ASR=0.89.
Okan Bursa
· 2026
Adaptive RAG Memory (ARM) augments a standard retriever–generator stack with a Dynamic Embedding Layer and Remembrance Engine that track usage statistics and apply selective remembrance and decay to embeddings. On a lightweight retrieval benchmark, ARM achieves NDCG@5 ≈ 0.9401 and Recall@5 = 1.000 with 22M parameters, matching larger baselines like gte-small while providing the best efficiency among ultra-efficient models.
Yijie Zhong, Yunfan Gao, Haofen Wang
· 2026
HingeMem combines Boundary Guided Long-Term Memory, Dialogue Boundary Extraction, Memory Construction, Query Adaptive Retrieval, Hyperedge Rerank, and Adaptive Stop to segment dialogues into element-indexed hyperedges and plan query-specific retrieval. On LOCOMO, HingeMem achieves 63.9 overall F1 and 75.1 LLM-as-a-Judge score, surpassing the best baseline Zep (56.9 F1) by 7.0 F1 without using category-specific QA formats.
Answers use this explainer on Memory Papers.
Checking…