AuthorsNilesh Prasad Pandey, Jason Kong, Lanxiang Hu et al.
TL;DR
AgentKVShift estimates a shared per-memory KV residual offset from probe tokens and corrects reused cache entries, reaching 0.319 F1 on LoCoMo at r=0.1 vs 0.178 for CacheBlend.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Existing training free KV reuse methods were designed for RAG style raw passages and degrade substantially on structured agentic memories.
When applied to metadata rich agentic memory systems, KV reuse leaves a large shared chunk level offset in unrefreshed tokens, hurting long horizon answer quality.
HOW IT WORKS
AgentKVShift combines Probe Selection, Mean Estimation, and AgentKVShift Correction to reuse KV cache per retrieved memory chunk while correcting stale entries.
You can think of AgentKVShift like adjusting all entries in a card catalog by measuring a few samples, then shifting the whole catalog instead of rewriting every card.
This chunk level residual correction lets AgentKVShift approximate fresh attention outputs using a small recompute budget, enabling long horizon agentic memory beyond a plain context window.
DIAGRAM
This diagram shows how AgentKVShift processes a retrieved memory chunk, selects probes, estimates residuals, and corrects reused KV states across layers.
DIAGRAM
This diagram shows how AgentKVShift is evaluated on AMem and LiCoMemory using LoCoMo and AMA Bench Recall with different LLMs and KV reuse baselines.
PROCESS
Probe Selection
AgentKVShift computes K and V divergence scores at the first transformer layer and selects top b probe tokens SKC and SVC for each chunk.
Mean Estimation
AgentKVShift recomputes fresh KV for probe tokens at every layer and estimates chunk level offsets μ̂KℓC and μ̂VℓC using weighted residual means.
AgentKVShift Correction
AgentKVShift applies a single weighted vector addition to each non probe token, producing corrected Kℓj,corr and Vℓj,corr while keeping probe tokens fresh.
Attention and Answer Generation
With corrected KV states, AgentKVShift computes attention outputs that approximate full recompute and generates long horizon answers using AMem or LiCoMemory.
KEY CONTRIBUTIONS
Failure of existing KV reuse on agentic memory
AgentKVShift demonstrates that training free KV reuse methods like CacheBlend and ProphetKV degrade substantially on agentic memory retrieval across 3B to 32B models.
Probe guided residual correction with spectral analysis
AgentKVShift uses per memory spectral analysis to show KV residuals are dominated by a shared offset and corrects them via probe guided mean shifts with tighter attention error bounds.
Near full recompute performance at low refresh
AgentKVShift reaches within 1.5–6% relative F1 of full recompute on LoCoMo while refreshing only 10% of the cache and achieves 2–3.5× prefill speedups on a single A100 GPU.
RESULTS
F1
0.319
+0.141 over CacheBlend on LoCoMo AMem Qwen2.5 3B at r=0.1
ROUGE-1
0.330
+0.142 over CacheBlend on LoCoMo AMem Qwen2.5 3B at r=0.1
BLEU-1
0.277
+0.133 over CacheBlend on LoCoMo AMem Qwen2.5 3B at r=0.1
Prefill speedup
3.33×
speedup vs full KV recompute on Qwen2.5 3B at 4096 tokens B=16 r=0.1
On LoCoMo, which tests long term dialogue memory, AgentKVShift nearly matches full recompute quality while using only 10% KV refresh. On AMA Bench Recall with Qwen3 32B, AgentKVShift attains 0.284 weighted F1 vs 0.270 for CacheBlend at r=0.3, showing effective KV reuse for agentic memory workloads.
BENCHMARK
On LoCoMo, which tests long term dialogue memory, AgentKVShift nearly matches full recompute quality while using only 10% KV refresh. On AMA Bench Recall with Qwen3 32B, AgentKVShift attains 0.284 weighted F1 vs 0.270 for CacheBlend at r=0.3, showing effective KV reuse for agentic memory workloads.
BENCHMARK
F1 on LoCoMo AMem with Qwen2.5 3B Instruct at r=0.1.
KEY INSIGHT
AgentKVShift matches near full recompute performance at r=0.1, while CacheBlend and ProphetKV only reach similar quality around r=0.45–0.55.
This is surprising because token selection methods were expected to scale smoothly with recompute ratio, but AgentKVShift shows that correcting unrefreshed tokens is far more impactful than recomputing more tokens.
WHY IT MATTERS
AgentKVShift unlocks efficient KV reuse for metadata rich agentic memory, allowing long horizon agents to maintain high quality answers with small refresh budgets.
Builders can now deploy AMem or LiCoMemory backed agents with 2–3.5× lower prefill latency and even combine KV reuse with 2–4 bit quantization while retaining over 2× the F1 of prior reuse methods.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xingyu Lyu, Jianfeng He et al.
· 2026
ADAM combines Anchor extraction, Distribution estimation, Anchor selection, and Query generation to adaptively probe agent memory via an auxiliary generator and entropy based selection. On the EHRAgent benchmark with Llama2-7b-chat, ADAM reaches EQ=77 and ASR=1.00, compared to MEXTRA’s EQ=44 and ASR=0.89.
Shannan Yan, Jingchen Ni et al.
· 2026
AdaMem organizes dialogue history into Working Memory, Episodic Memory, Persona Memory, and Graph Memory coordinated by a Memory Agent, Research Agent, and Working Agent. On LoCoMo with GPT-4.1-mini, AdaMem achieves 44.65 F1 overall, beating the best baseline LangMem at 41.76 F1 by +2.89.
Answers use this explainer on Memory Papers.
Checking…