APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay

THE PROBLEM

LLM agents stay stateless despite repeated, similar tasks (41.3% vs 89.6% on KGQAGen-10k)

HOW IT WORKS

APEX-EM — Procedural Knowledge Graphs plus PRGII experience replay

DIAGRAM

PRGII workflow: Plan–Retrieve–Generate–Iterate–Ingest loop

This diagram shows how APEX-EM runs the PRGII workflow to solve a task and commit a structured experience back into memory.

DIAGRAM

Evaluation setup across BigCodeBench, KGQAGen-10k, and HLE

This diagram shows how APEX-EM is evaluated on three benchmarks with frozen backbones and shared baselines like MemRL and RAG.

PROCESS

How APEX-EM Handles a Task via the PRGII Workflow

1. 01

   Plan Phase

   APEX-EM parses the task into Task Understanding, runs Entity Discovery and Schema Discovery, and uses the StructuralSignatureExtractor to hypothesize an abstract operation sequence.

2. 02

   Retrieve Phase

   APEX-EM queries the Experience Memory store using semantic search, structural signature matching, and PKG traversal to collect successful and failed experiences.

3. 03

   Generate Phase

   Conditioned on Goal Reflections, Procedure Reflections, and negative examples from the Error Registry, APEX-EM generates an executable artifact such as code or a SPARQL query.

4. 04

   Iterate and Ingest Phases

   Task Verifiers validate each artifact, drive refinement across iterations, then the Teacher and quality gate decide whether to store the run as a successful or failed Experience in the Procedural Knowledge Graph.

KEY CONTRIBUTIONS

Key Contributions

• 01

  Procedural Knowledge Graph

  APEX-EM introduces a Procedural Knowledge Graph that stores Experiences, Entities, Sub-Tasks, Operations, and TaskTopic nodes with structural signatures like [entity_resolution → temporal_filter → aggregation].

• 02

  PRGII workflow with Task Verifiers

  APEX-EM defines the Plan-Retrieve-Generate-Iterate-Ingest workflow where Task Verifiers and a Teacher provide multi-dimensional scores c, η, κ and an overall quality q with threshold θ.

• 03

  Dual-outcome Experience Memory

  APEX-EM builds a dual-outcome Experience Memory that treats successful experiences as positive in-context examples and failed ones as negative examples with structured Error Registry and Patch Reflections.

RESULTS

By the Numbers

On KGQAGen-10k, a structured query benchmark, APEX-EM reaches 89.6% accuracy and 95.3% CSR, beating the 41.3% no-memory baseline and the GPT-4o w/ SP oracle at 84.9%. On BigCodeBench and HLE, APEX-EM delivers +29.4pp and +22.8pp gains in success rate, showing that structured procedural replay scales across code and multi-domain reasoning.

BENCHMARK

By the Numbers

On KGQAGen-10k, a structured query benchmark, APEX-EM reaches 89.6% accuracy and 95.3% CSR, beating the 41.3% no-memory baseline and the GPT-4o w/ SP oracle at 84.9%. On BigCodeBench and HLE, APEX-EM delivers +29.4pp and +22.8pp gains in success rate, showing that structured procedural replay scales across code and multi-domain reasoning.

BENCHMARK

KGQAGen-10k: APEX-EM vs LLM and KG-RAG baselines

LASM Accuracy on KGQAGen-10k test split and training sample for APEX-EM and key baselines.

BENCHMARK

BigCodeBench: APEX-EM vs MemRL and no-memory baselines

Last Epoch Success Rate on BigCodeBench train split for APEX-EM and MemRL baselines.

KEY INSIGHT

WHY IT MATTERS

What this unlocks for the field