AuthorsYasong Fan
TL;DR
Fan Duality Model (FDM) combines a phase-preserving Fan Operator with a local-global cache and Freeze-Scan training to reach 0.966 MQAR accuracy and fixed 867 MB O(1) decode memory.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
Transformers need 4,247 MB KV cache at N=8,192 for a 137M model, while Fan Duality Model (FDM) stays at 867 MB.
This KV cache growth makes long-context decode memory and decode speed impractical, degrading Transformer throughput by 83% and limiting associative recall tasks.
HOW IT WORKS
Fan Duality Model (FDM) combines a Fan Operator, Local-Global Cache, Freeze-Scan Training, and Holographic Reference Beam Decoding to decouple wave and particle behavior.
You can think of Fan Duality Model (FDM) like RAM plus an indexed card catalog: the wave state is compact RAM, while the cache is a small, addressable index.
This dual design lets Fan Duality Model (FDM) keep O(1) memory while still doing precise associative recall that a plain context window or pure scan cannot.
DIAGRAM
This diagram shows how Fan Duality Model (FDM) alternates between full training and cache-only optimization in the Freeze-Scan strategy.
DIAGRAM
This diagram shows how Fan Duality Model (FDM) is evaluated on WikiText-103, MQAR, and downstream benchmarks against Transformer baselines.
PROCESS
The Fan Operator
Fan Duality Model (FDM) first applies the Fan Operator recurrent scan, using phase-preserving Givens rotations to update the complex hidden state ht from embeddings.
Local Global Cache
Fan Duality Model (FDM) populates the Local Global Cache with W=256 local and K=16 global slots, selected by seff based associative addressing.
Freeze Scan Training
Fan Duality Model (FDM) runs Freeze-Scan Training, first training all parameters, then freezing Φwave and specializing Φcache and embeddings for induction style recall.
Holographic Reference Beam Decoding
Fan Duality Model (FDM) finally uses Holographic Reference Beam Decoding to modulate ht with xt, with 4-head orthogonal beams improving PPL by 2.13 points.
KEY CONTRIBUTIONS
FDM architecture
Fan Duality Model (FDM) introduces a Fan Operator wave scan plus Local Global Cache particle component, achieving O(1) decode memory of 867 MB with W=256 and K=16 slots.
Freeze Scan
Fan Duality Model (FDM) uses Freeze-Scan Training to avoid gradient sinks, improving WikiText-103 perplexity from 487 to 64.9 in 44K steps and crossing PPL 100 at 17K steps.
Holographic Reference Beam Decoding
Fan Duality Model (FDM) adds Holographic Reference Beam Decoding, where a 4-head orthogonal reference beam reduces PPL by 2.13 points to 62.79 with only 1.3M extra parameters.
RESULTS
Val PPL
64.9
-423.1 vs FDM — Full fine-tuning (487)
Val PPL
62.79
-2.13 vs FDM — Freeze-Scan (64.9)
MQAR accuracy
0.966
+0.360 over Transformer
Decode Memory (MB)
867
-3,380 MB vs Transformer at N=8,192
On WikiText-103 and MQAR, Fan Duality Model (FDM) trades some language modeling PPL for dramatically better associative recall and O(1) decode memory. These results show Fan Duality Model (FDM) can reach 0.966 MQAR accuracy while keeping decode memory fixed at 867 MB across 128–8,192 token prompts.
BENCHMARK
On WikiText-103 and MQAR, Fan Duality Model (FDM) trades some language modeling PPL for dramatically better associative recall and O(1) decode memory. These results show Fan Duality Model (FDM) can reach 0.966 MQAR accuracy while keeping decode memory fixed at 867 MB across 128–8,192 token prompts.
BENCHMARK
Accuracy on Multi Query Associative Recall Easy setting.
BENCHMARK
Validation perplexity on WikiText-103 for Fan Duality Model variants and Transformer.
KEY INSIGHT
Fan Duality Model (FDM) with pure scan (K=0) scores only 0.011 MQAR accuracy, while adding just K=16 cache slots jumps to 0.966.
This is surprising because linear recurrent models are often assumed to store long range information in their hidden state, yet Fan Duality Model (FDM) shows that without a tiny particle cache associative recall nearly collapses.
WHY IT MATTERS
Fan Duality Model (FDM) unlocks fixed 867 MB decode memory and stable decode speed across 128–8,192 tokens while still achieving strong associative recall.
Builders can now design long context systems where memory cost is O(1) in sequence length, yet Fan Duality Model (FDM) still retrieves specific tokens via a tiny learned cache and holographic decoding.
Pratyay Banerjee, Masud Moshtaghi et al.
· 2026
APEX-MEM constructs a semi-structured temporal property graph using Ontology, Entity and Property Resolution, Fact Extraction, and Graph Agents to store and query conversational memory. On LOCOMO, APEX-MEM with GPT5 achieves 88.88% overall accuracy, beating MIRIX at 85.38% by 3.50 percentage points.
Andy Nguyen, Danh Doan et al.
· 2026
ByteRover organizes memory through an LLM-driven Agent Layer, a sequential Execution Layer, and a file-based Context Tree with an Adaptive Knowledge Lifecycle. On LoCoMo, ByteRover reaches 96.1% overall accuracy versus 89.9% for HonCho, and on LongMemEval-S ByteRover scores 92.8% overall.
Martin Vogel, Falk Meyer-Eschenbach et al.
· 2026
Codebase-Memory parses repositories with a multi-pass pipeline using the Parse stage, Build stage, Serve stage, FunctionRegistry, Louvain communities, and MCP tool interface to build a persistent SQLite knowledge graph. On a 31-language benchmark, Codebase-Memory reaches 0.83 quality versus 0.92 for an Explorer Agent while using ten times fewer tokens and 2.1 times fewer tool calls.
Answers use this explainer on Memory Papers.
Checking…