AuthorsShuhan Xue, Zixin Ding, Yichen Shen et al.
TL;DR
PAST-Bench uses matched persistence-on/off task families plus Hermes+’s five-stage agent-loop interventions to raise self-evolution gap Δ from +0.13 to +0.15 and mechanism evidence from 0.64 to 0.73.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
PAST-Bench shows that Hermes and nanobot can share the same Overall Δ = +0.13 while differing in mechanism evidence (0.64 vs. 0.57).
This means personal agents can appear to improve while bypassing intended memory, skill, or update pathways, making recursive self-improvement hard to trust and debug.
HOW IT WORKS
PAST-Bench structures evaluation into Cold, Learn, Evaluation, and Control episodes within task families, while Hermes+ adds Plan, Render, Route, Gate, and Close mechanisms to the Hermes loop.
Think of PAST-Bench as a controlled A/B lab for persistence and Hermes+ as a smarter operating system that manages memory, skills, and updates like typed records and executable procedures.
This combination lets PAST-Bench attribute later-task gains to specific save, retrieve, and update events, rather than to opaque long-context effects that a plain context window cannot separate.
DIAGRAM
This diagram shows how PAST-Bench runs ordered fresh-session episodes with persistence toggled on and off inside each task family.
DIAGRAM
This diagram shows how PAST-Bench evaluates single-mechanism and full Hermes+ configurations under fixed model, tasks, and grader.
PROCESS
Benchmark Construction
PAST-Bench defines Memory, Procedural Reuse, Information Gathering, and Update families, each with ordered episodes and matched control designs.
Evaluation Pipeline
PAST-Bench runs Cold, Learn, Evaluation, and Control episodes as fresh sessions while toggling evolve on and off for each family.
Mechanism Evidence and Generalization
PAST-Bench logs memory writes, skill lifecycle events, session search, and update events to compute the mechanism-evidence score Mech.
Diagnosis-Driven Design Hermes plus
PAST-Bench uses uneven gaps and Mech traces to design Hermes+ Plan, Render, Route, Gate, and Close interventions and test their ablations.
KEY CONTRIBUTIONS
PAST-Bench benchmark for self-evolving personal agents
PAST-Bench introduces 26 scenarios and 204 episodes across Memory, Procedural Reuse, Information Gathering, and Update, with matched persistence-on and persistence-off task-family runs.
Diagnosing self-evolution failures
PAST-Bench shows that Hermes and nanobot can both reach Overall Δ = +0.13 while differing in mechanism evidence (0.64 vs. 0.57), exposing hidden pathway failures.
Hermes plus agent framework baseline
Hermes+ extends Hermes with Plan, Render, Route, Gate, and Close mechanisms, raising Overall Δ from +0.13 to +0.15 and Mech from 0.64 to 0.73 on MiniMax-M2.7.
RESULTS
Overall Δ Hermes
+0.13
baseline for MiniMax-M2.7 before Hermes+
Overall Δ Hermes plus
+0.15
+0.02 over Hermes on MiniMax-M2.7
Mech Hermes
0.64
baseline mechanism evidence with MiniMax-M2.7
Mech Hermes plus
0.73
+0.09 mechanism evidence over Hermes on MiniMax-M2.7
These numbers come from PAST-Bench’s MiniMax-M2.7 experiments, which test Memory, Procedural Reuse, Information Gathering, and Update with matched persistence-on and persistence-off runs. The gains show that Hermes+ improves both outcome gaps and alignment with intended persistence pathways.
BENCHMARK
These numbers come from PAST-Bench’s MiniMax-M2.7 experiments, which test Memory, Procedural Reuse, Information Gathering, and Update with matched persistence-on and persistence-off runs. The gains show that Hermes+ improves both outcome gaps and alignment with intended persistence pathways.
BENCHMARK
Family-balanced Overall Δ on PAST-Bench for MiniMax-M2.7 across agent frameworks.
BENCHMARK
Mechanism-evidence score Mech on PAST-Bench for MiniMax-M2.7 across agent frameworks.
KEY INSIGHT
PAST-Bench reveals that Hermes and nanobot both achieve Overall Δ = +0.13 on MiniMax-M2.7, yet Hermes has Mech = 0.64 while nanobot has Mech = 0.57.
This is surprising because identical task-score gains usually suggest similar mechanisms, but PAST-Bench shows that self-evolution can ride on very different, sometimes unintended, persistence pathways.
WHY IT MATTERS
PAST-Bench lets researchers separate score gains from mechanism evidence, while Hermes+ offers a controllable agent loop for Plan, Render, Route, Gate, and Close.
Builders can now design personal agents that not only retain experience but also show traceable, diagnosable recursive self-improvement across memory, procedures, information gathering, and updates.
Jingbiao Mei, Jinghong Chen et al.
arXiv 2026 · 2026
ATM-Bench structurally evaluates long-term multimodal personal memory using Memory Ingestion, Retrieval, and Answer Generation with Schema-Guided Memory and Descriptive Memory variants. On ATM-Bench-Hard, Oracle with SGM reaches 47.3% QS while the best full system stays under 20% accuracy, revealing a large gap.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xingyu Lyu, Jianfeng He et al.
· 2026
ADAM combines Anchor extraction, Distribution estimation, Anchor selection, and Query generation to adaptively probe agent memory via an auxiliary generator and entropy based selection. On the EHRAgent benchmark with Llama2-7b-chat, ADAM reaches EQ=77 and ASR=1.00, compared to MEXTRA’s EQ=44 and ASR=0.89.
Answers use this explainer on Memory Papers.
Checking…