AuthorsJulian Acuna
TL;DR
EngramaBench shows that Engrama’s graph-structured memory activation and typed answers reach 0.6532 cross space score, beating GPT-4o full-context’s 0.6291 at much lower cost.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Large assistants are expected to retain weeks of history, yet Mem0 reaches only 0.5266 cross space accuracy and 0.2848 single space on EngramaBench.
This means long-term conversational systems mis-handle cross-space integration, temporal change, and adversarial abstention, limiting trustworthy multi-session collaboration.
HOW IT WORKS
EngramaBench probes Engrama’s Graph Memory, Memory activation, Neighborhood retrieval, Structured summary, and Typed answer layer over multi-session personas and semantic spaces.
Think of EngramaBench as a stress test where Engrama plays the hippocampus, organizing life events into an indexed graph instead of a raw transcript dump.
This structured surface lets Engrama recover configurations of related facts and abstain on fabricated queries in ways a plain context window cannot.
DIAGRAM
This diagram shows how EngramaBench feeds persona histories and queries into Engrama’s ingestion and retrieval pipeline for evaluation.
DIAGRAM
This diagram shows how EngramaBench constructs personas, conversations, queries, and scores Engrama against GPT-4o full-context and Mem0.
PROCESS
Ingestion and Consolidation
Engrama processes timestamped conversations, extracting entities, relations, and events, and attaches them into a consolidated graph memory surface.
Graph Memory Organization
Engrama organizes entities, semantic spaces, temporal traces, and associative links so work, family, and other regions remain distinct yet connected.
Query-conditioned Memory Activation
Engrama uses memory activation, including Lever 1’s entity-first activation and query planner, to select a relevant subregion of the graph.
Neighborhood Retrieval and Answer Composition
Engrama performs neighborhood retrieval, builds a structured summary, and applies the typed answer layer (Lever 3) to generate precise answers.
KEY CONTRIBUTIONS
EngramaBench benchmark for long-term conversational memory
EngramaBench defines five personas, 100 conversations, and 150 queries across single space, cross space, temporal cross space, adversarial, and emergent insight behaviors.
Controlled comparison of memory architectures
EngramaBench compares GPT-4o full-context, Engrama, and Mem0 using GPT-4o as the shared answering model, isolating memory architecture effects on composite and per-family scores.
Ablation analysis of Engrama’s design levers
EngramaBench studies Lever 1 and Lever 3, showing composite changes up to +0.0134 and cross space drops up to −0.0546, revealing a tradeoff between specialization and global optimization.
RESULTS
Composite score
0.5367
−0.0819 vs GPT-4o full-context’s 0.6186
Cross space
0.6532
+0.0241 over GPT-4o full-context’s 0.6291
Single space
0.5997
−0.1342 vs GPT-4o full-context’s 0.7339
Query-time cost
$0.67
about −$2.66 vs GPT-4o full-context’s $3.33 across 150 queries
On EngramaBench full v1, which tests long-term conversational memory across five task families, Engrama trades a lower composite score for a higher 0.6532 cross space score and much lower query-time cost than GPT-4o full-context.
BENCHMARK
On EngramaBench full v1, which tests long-term conversational memory across five task families, Engrama trades a lower composite score for a higher 0.6532 cross space score and much lower query-time cost than GPT-4o full-context.
BENCHMARK
Composite score on EngramaBench full v1.
BENCHMARK
Cross space score (n=30 queries).
KEY INSIGHT
Engrama’s Lever 3 removal raises composite to 0.5501 yet drops cross space from 0.6532 to 0.6033, a −0.0499 change on the most diagnostic slice.
This is surprising because you’d expect a stronger typed answer layer to help globally, but EngramaBench shows it mainly benefits structured cross-space reasoning rather than aggregate score.
WHY IT MATTERS
EngramaBench gives builders a controlled way to see how graph-structured memory like Engrama changes behavior on cross-space, temporal, adversarial, and synthesis queries.
With these measurements, system designers can tune memory activation, retrieval, and typed answer layers to target specific long-term behaviors instead of chasing a single undifferentiated long-context score.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xingyu Lyu, Jianfeng He et al.
· 2026
ADAM combines Anchor extraction, Distribution estimation, Anchor selection, and Query generation to adaptively probe agent memory via an auxiliary generator and entropy based selection. On the EHRAgent benchmark with Llama2-7b-chat, ADAM reaches EQ=77 and ASR=1.00, compared to MEXTRA’s EQ=44 and ASR=0.89.
Shannan Yan, Jingchen Ni et al.
· 2026
AdaMem organizes dialogue history into Working Memory, Episodic Memory, Persona Memory, and Graph Memory coordinated by a Memory Agent, Research Agent, and Working Agent. On LoCoMo with GPT-4.1-mini, AdaMem achieves 44.65 F1 overall, beating the best baseline LangMem at 41.76 F1 by +2.89.
Answers use this explainer on Memory Papers.
Checking…