AuthorsKexin Ma, Bojun Li, Yuhua Tang et al.
TL;DR
CAST uses character-centric 3D scenes as episodic memory, fusing them with a semantic graph to gain up to +46.59% J on episodic benchmarks.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Existing LLM agents mostly use key value, vector, or graph memories and struggle to recall coherent events grounded in who, when, and where.
On open and time sensitive conversational questions, these flat memories miss crucial episodic units like the quote Finding Freedom, causing incorrect or vague answers.
HOW IT WORKS
CAST’s core mechanism builds views, aggregates them into scenes, organizes them into character profiles, and pairs this episodic index with a semantic index and retrieval fusion.
You can think of CAST like a card catalog plus a timeline: the semantic graph is the catalog of facts, while character conditioned scenes act as event reels indexed by people, time, and place.
This design lets CAST answer questions about who did what when and where using explicit episodic anchors, rather than relying on a plain context window or raw vector similarity.
DIAGRAM
This diagram shows how CAST processes a user query by parallel semantic and episodic search and then fuses the evidence.
DIAGRAM
This diagram shows how CAST is evaluated on LOCOMO and epbench, including parameter sweeps and ablations.
PROCESS
Offline Index Construction
CAST first constructs views from the dialogue corpus, aggregates them into scenes, and builds character profiles plus a semantic index.
Role Assignment and Scene Aggregation
CAST assigns main and supporting roles within scenes, then performs greedy 3D scene aggregation over time, place, and topic to form episodic units.
Online Query Processing
For a query, CAST runs semantic search over the semantic graph and DPR, and episodic search over scene embeddings from the episodic index.
Retrieval Fusion and Answer Generation
CAST applies retrieval fusion to rerank overlapping memories, then passes the fused evidence to answer generation with gpt-4o-mini.
KEY CONTRIBUTIONS
Character and Scene based Memory Architecture
CAST introduces views, scene aggregation, and character profiles to represent episodic memory as coherent person conditioned scenes, improving average F1 by 8.11% and J by 10.21% over baselines.
Dual Semantic and Episodic Index
CAST builds a hybrid semantic index combining a heterogeneous graph with DPR and an episodic index of scene embeddings, enabling robust retrieval for both factual and event centric questions.
Retrieval Fusion Strategy
CAST proposes a lightweight retrieval fusion that reranks semantic memories using episodic matches, boosting J on locomo_open from 72.89 with Zep to 81.21.
RESULTS
F1 locomo_open
62.02
+13.24 over Zep
J locomo_open
81.21
+24.61 over RAG
F1 ep_scifi
52.04
+1.61 over HippoRAG
J ep_scifi
97.06
+2.57 over HippoRAG
On LOCOMO and epbench, which test long term conversational and episodic memory, CAST consistently achieves the best or near best F1 and J. These results show CAST’s character and scene episodic memory substantially improves correctness on who did what when and where questions compared to RAG, Zep, LangMem, Mem0, and HippoRAG.
BENCHMARK
On LOCOMO and epbench, which test long term conversational and episodic memory, CAST consistently achieves the best or near best F1 and J. These results show CAST’s character and scene episodic memory substantially improves correctness on who did what when and where questions compared to RAG, Zep, LangMem, Mem0, and HippoRAG.
BENCHMARK
F1 on locomo_open for CAST and key baselines.
KEY INSIGHT
Increasing the retrieval top k up to 10 can push CAST’s J on LOCOMO open questions as high as 87.61, far above smaller k settings.
This is surprising because more retrieved passages often introduce distractors, yet CAST’s episodic structure and fusion strategy turn extra memories into better episodic alignment instead of noise.
WHY IT MATTERS
CAST unlocks agent memories that can reliably answer fine grained episodic questions about people, times, places, and actions across long dialogues.
Builders can now design LLM agents whose memory behaves more like human autobiographical recall, using character profiles and scenes instead of flat key value or vector stores.
Peilin Feng, Suorong Yang, Soujanya Poria
arXiv 2026 · 2026
Σ-Mem maintains historical competence evidence matrices M_p, a peer relationship matrix G, a memory readout r_{p,t}, and residual steering δ_{p,t} to guide multi-agent trust decisions. On the mixed counterfactual benchmark, Σ-Mem raises Qwen3-0.6B accuracy at CF@90 from 46.22% with the base model to 71.10% with Σ w/ G.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xiaohui Zhang, Zequn Sun et al.
· 2026
ActMem transforms dialogue history into atomic facts via Memory Fact Extraction, groups them with Fact Clustering, links them through a Memory KG Construction module, and uses Counterfactual-based Retrieval and Reasoning for action-aware answers. On ActMemEval, ActMem reaches 76.52% QA accuracy with DeepSeek-V3, beating LightMem’s 63.97% by 12.55 points and NaiveRAG’s 61.54%.
Answers use this explainer on Memory Papers.
Checking…