AuthorsSiddharth Sharma, Nilesh Prasad Pandey, Onat Gungor, Tajana Rosing
TL;DR
LIMBO uses a two head LinUCB controller to treat replay memory as an inference time resource, matching strongest baselines while cutting cost by up to 83%.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Fixed experience replay on DBBench raises inference cost by more than 5× when scaling from no replay to k = 16, but accuracy improves by only 2 points.
This hurts lifelong LLM agents that must share a limited prompt budget with retrieval, tool use, and verification, wasting tokens on low value replay and degrading efficiency.
HOW IT WORKS
LIMBO’s core mechanism links a memory bank of successful interaction trajectories, feature extraction, and a two head LinUCB controller that selects memory actions and compute budgets per task.
You can think of LIMBO as a smart RAM controller that decides when to page in old trajectories and how much compute to allocate, instead of always filling the context window.
This two head LinUCB design lets LIMBO jointly predict correctness and cost, enabling replay and budget choices that a plain context window or fixed replay schedule cannot adaptively make.
DIAGRAM
This diagram shows how LIMBO runs its online loop over a task stream, updating the two head LinUCB controller and memory bank after each task.
DIAGRAM
This diagram shows how LIMBO is evaluated across models, environments, and baselines on LifelongAgentBench.
PROCESS
Feature extraction
LIMBO computes the 12 dimensional feature vector xi using prompt shape, runtime state, and previous outcome indicators before each task.
Two head LinUCB controller
Using xi, LIMBO’s two head LinUCB controller predicts expected correctness and normalized cost for each action in the retrieval gated budget primitives.
Memory actions
LIMBO selects a memory action ai specifying replay mode among none, full, compressed, or retrieved and pairs it with token, round, and tool budgets.
Prompt construction and agent execution
LIMBO constructs the prompt using the memory bank of successful interaction trajectories, executes the agent once, then updates bandit statistics and the memory bank with σi.
KEY CONTRIBUTIONS
LIMBO framework for inference allocation
LIMBO formulates inference time memory allocation as a contextual bandit, using feature extraction and a two head LinUCB controller to jointly choose replay strategy and compute budget.
Cost aware replay and budget primitives
LIMBO defines retrieval gated budget primitives that pair replay modes like full, compressed, and retrieved with token, round, and tool budgets under a unified action space.
Cost accuracy Pareto frontier results
LIMBO matches or nearly matches the strongest fixed replay baselines on LifelongAgentBench while reducing inference cost by up to ∼83% and ∼53% on average across six settings.
RESULTS
Accuracy Qwen DB
74.4 %
-0.45 over Fixed k=16 (74.85 %)
Cost Qwen DB
$0.050
-82.8% vs Fixed k=16 ($0.290)
Accuracy Llama DB
66.2 %
-3.8 over Fixed k=16 (70.0 %)
Cost GPT 4o DB
$0.292
-82.3% vs LongLLMLingua ($1.654)
On LifelongAgentBench DB and OS environments, LIMBO is evaluated with Qwen2.5 7B, Llama 3.1 8B, and GPT 4o mini, showing cost reductions up to ∼83% while staying within 3.8 accuracy points of the strongest fixed memory baselines.
BENCHMARK
On LifelongAgentBench DB and OS environments, LIMBO is evaluated with Qwen2.5 7B, Llama 3.1 8B, and GPT 4o mini, showing cost reductions up to ∼83% while staying within 3.8 accuracy points of the strongest fixed memory baselines.
BENCHMARK
Accuracy on LifelongAgentBench DB for Qwen 2.5 7B with different replay strategies.
KEY INSIGHT
On DBBench with Qwen2.5 7B, scaling fixed replay from no replay to k = 16 raises cost by more than 5× but improves accuracy by only 2 points.
LIMBO instead achieves a 1.4 point gain at only 1.10× cost, showing that more replay is not always better and breaking the assumption that larger buffers are uniformly beneficial.
WHY IT MATTERS
LIMBO unlocks online, cost aware control of inference time memory and compute, letting agents decide when replay is worth the tokens for each task.
Builders can now deploy lifelong LLM agents that adapt replay and budgets per environment and backbone without retraining, operating near the cost floor while staying close to the strongest memory augmented baselines.
Guilin Zhang, Wei Jiang et al.
· 2026
A-MAC scores candidate memories using Utility, Confidence, Novelty, Recency, and Type Prior combined by a learned linear admission policy with Algorithm 1 A-MAC Memory Admission. On the LoCoMo benchmark, A-MAC achieves F1 0.583 and 2644 ms latency, improving F1 by 0.042 and reducing latency by 1187 ms compared to A-mem.
Robbyant Team, Zelin Gao et al.
arXiv 2026 · 2026
LingBot-World combines a Data Engine, Fundamental World Model, Action-Conditioned World Model, and Post-Training causal adaptation to turn a 28B-parameter video generator into a real-time interactive world simulator. On the VBench benchmark, LingBot-World achieves a dynamic degree of 0.8857 versus 0.7612 for Yume-1.5, while also improving imaging quality to 0.6683.
Manoj Madushanka Perera, Adnan Mahmood et al.
· 2026
AgenticAI-DialogGen chains ChatPreprocessor, KnowledgeExtractor, TopicAnalyzer, KnowledgeGraphBuilder, PersonaGenerator, DuelingChat Agent, ConversationValidator, ConversationRefiner, QAGeneration, and PostProcessing to turn raw multi-session chats into topic-guided, persona-grounded conversations with explicit short- and long-term memories. On the TGC / KG memory QA benchmark, Mistral-7B fine-tuned within AgenticAI-DialogGen achieves 87.36 F1, compared to GPT-4’s 83.77 F1 in a zero-shot setting on the same task.
Answers use this explainer on Memory Papers.
Checking…