Beyond Heuristics: A Decision-Theoretic Framework for Agent Memory Management

AuthorsChangzhi Sun, Xiangyu Chen, Jixiang Luo et al.

2025

TL;DR

DAM uses hierarchical read and write policies with value and uncertainty estimators to reframe agent memory as a sequential decision problem under uncertainty, unifying heuristic memory operations under one decision-theoretic architecture.

SharePost on XLinkedIn

Read our summary here, or open the publisher PDF on the next tab.

THE PROBLEM

Heuristic Memory Ignores Delayed and Uncertain Utility

Existing LLM agents rely on hand-designed heuristics, offering little insight into the long-term and uncertain consequences of memory decisions.

In systems like TeleMem and other heuristic memory layers, brittle rules cause critical context to be lost or irrelevant noise to accumulate, degrading downstream retrieval quality and behavior.

HOW IT WORKS

DAM: Decision-Theoretic Agent Memory

DAM’s core mechanism is a hierarchical policy with Read Policy, Write Policy, Value Function, Uncertainty Estimator, and Aggregate Policy coordinating memory operations.

You can think of DAM like a computer with smart RAM for immediate reads and a risk-aware disk controller deciding what to persist or delete over time.

This decision-theoretic design lets DAM weigh delayed utility and uncertainty, enabling cautious, long-horizon memory management that a plain context window or static heuristic cannot provide.

DIAGRAM

Sequential Memory Decision Flow in DAM

This diagram shows how DAM processes interactions over time as a sequential decision problem with state, actions, exogenous information, and transitions.

DIAGRAM

Policy Components and Arbitration in DAM

This diagram shows how DAM’s read policy, write sub-policies, value functions, uncertainty estimators, and aggregate policy interact to produce memory updates.

PROCESS

How DAM Handles a Memory Management Decision

  1. 01

    State Variables

    DAM constructs State Variables St capturing the current input, summaries of past interactions, and memory metadata to ground subsequent decisions.

  2. 02

    Decision Variables

    DAM forms Decision Variables At as a tuple of aread t and awrite t, separating information access from storage maintenance actions.

  3. 03

    Read Policy

    DAM applies the Read Policy πread to select retrieval parameters, building an effective context that conditions the write decisions.

  4. 04

    Write Policy and Aggregate Policy

    DAM’s Write Policy uses Sub-policy, Value Function, and Uncertainty Estimator to propose operations, while the Aggregate Policy πagg arbitrates and executes coherent memory updates.

KEY CONTRIBUTIONS

Key Contributions

  • 01

    Decision-Theoretic Agent Memory Framework

    DAM formalizes memory management via State Variables, Decision Variables, and an Objective Function, framing memory as sequential optimization under uncertainty instead of ad-hoc heuristics.

  • 02

    Hierarchical Read and Write Policy Architecture

    DAM introduces a hierarchical policy with Read Policy, Write Policy, Sub-policy, and Aggregate Policy to separate immediate information access from long-term storage maintenance.

  • 03

    Value and Uncertainty Guided Memory Operations

    DAM pairs each write Sub-policy with a Value Function and Uncertainty Estimator, enabling risk-aware arbitration of add and delete operations via the Aggregate Policy.

RESULTS

By the Numbers

Cumulative Objective

∑ γt C(St, At, Wt+1)

formalizes long term memory utility vs heuristic baselines

Discount Factor γ

γ ∈ (0, 1)

controls weighting of delayed memory effects

Write Operations Set

{add, delete}

defines DAM’s storage action space

State Space

St ∈ S

captures context and accessible memory for decisions

DAM is a theoretical framework, so the key quantitative elements are its formal objective and action space rather than benchmark scores. These definitions show how DAM treats memory decisions as optimizing expected discounted utility over states and write operations under uncertainty.

BENCHMARK

Heuristic Memory vs Decision-Theoretic Memory Dimensions

Relative emphasis of five design dimensions contrasting heuristic memory systems and DAM’s decision-theoretic memory.

KEY INSIGHT

The Counterintuitive Finding

DAM highlights that memory operations with seemingly low immediate relevance can have high long-term utility, especially for user preferences and abandoned subgoals.

This is counterintuitive because heuristic systems assume current similarity or recency is enough, ignoring that deleting such items can irreversibly harm future responses many steps later.

WHY IT MATTERS

What this unlocks for the field

DAM unlocks the ability to treat agent memory as a controllable, uncertainty-aware decision process rather than a passive buffer or heuristic store.

Builders can now design memory modules with explicit value and risk estimates, enabling cautious deletion, strategic writing, and principled arbitration across long-lived LLM agents.

~12 min read← Back to papers

Related papers

BenchmarkAgent Memory

Active Context Compression: Autonomous Memory Management in LLM Agents

Nikhil Verma

· 2026

Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.

Agent Memory

ActMem: Bridging the Gap Between Memory Retrieval and Reasoning in LLM Agents

Xiaohui Zhang, Zequn Sun et al.

· 2026

ActMem transforms dialogue history into atomic facts via Memory Fact Extraction, groups them with Fact Clustering, links them through a Memory KG Construction module, and uses Counterfactual-based Retrieval and Reasoning for action-aware answers. On ActMemEval, ActMem reaches 76.52% QA accuracy with DeepSeek-V3, beating LightMem’s 63.97% by 12.55 points and NaiveRAG’s 61.54%.

Questions about this paper?

Paper: Beyond Heuristics: A Decision-Theoretic Framework for Agent Memory Management

Answers use this explainer on Memory Papers.

Checking…