All AI memory papers

Large Associative Memory Problem rewrites associative memory using coupled feature neurons, memory neurons, an energy function, and Lagrangian functions so that all interactions are pairwise yet recover Dense and modern Hopfield behavior. Large Associative Memory Problem shows that with appropriate choices of activation and Lagrangian functions, the effective dynamics match Dense Associative Memories and modern Hopfield networks, enabling storage of N_mem ∼ min(N_f^{n−1}, N_h) or even exponential in N_f memories without many‑body synapses.

Variational Semantic Memory combines variational prototype inference, variational semantic memory, latent memory m, and an attention-based memory update to build probabilistic class prototypes from long-term semantic knowledge. On miniImageNet 5-way 1-shot with a deep backbone, Variational Semantic Memory reaches 65.72% accuracy versus 64.82% for Tian et al. 2020.

MEMO combines common embeddings, multi head keys and values, recurrent attention, and a halting policy to flexibly chain episodic memories over multiple hops. On joint bAbI 10k, MEMO achieves 0.21% error versus 4.2% for Memory Networks, while also solving long-distance Paired Associative Inference and shortest path tasks.

RAG uses a Retriever pη(z|x), Generator pθ(yi|x,z,y1:i−1), RAG-Sequence, and RAG-Token to retrieve Wikipedia passages with DPR and generate answers with BART while marginalizing documents as latent variables. On open Natural Questions, RAG-Sequence reaches 44.5 Exact Match versus 41.5 for DPR, and on TriviaQA-Wiki RAG-Sequence reaches 68.0 versus 57.9 for DPR.

Self-Attentive Associative Memory (STM) combines Outer Product Attention (OPA), Self-attentive Associative Memory (SAM), Mi-Write, Mr-Read, and Mr-Transfer into a dual item–relational memory system. On the bAbI question answering benchmark, STM attains 0.39 ± 0.18 mean error versus 0.55 ± 0.74 for MNM-p, establishing a new state-of-the-art.

Adaptive Posterior Learning combines an Encoder, Memory store, Memory controller, and Decoder (relational self-attention, relational working memory, or LSTM) to approximate posteriors from a sparse external memory. On Omniglot, Adaptive Posterior Learning achieves 99.9% 5-way 5-shot accuracy, matching MAML and SNAIL, while using fewer than 2 stored examples per class.

Episodic Memory in Lifelong Language Learning combines an example encoder, task decoder, and episodic memory for sparse experience replay and Memory-based Parameter Adaptation (MBPA++). Episodic Memory in Lifelong Language Learning attains 70.6 averaged classification accuracy vs 66.9 for A-GEM and 62.4 QA F1 vs 57.9 for REPLAY on concatenated multi-dataset streams.

Episodic Memory Reader combines a Data Encoder, Memory Encoder (EMR-Independent, EMR-biGRU, EMR-Transformer), Value Network, external memory, and QA solver to learn which streaming items to keep. On TriviaQA, Episodic Memory Reader with EMR-biGRU achieves 57.57 F1 versus 50.10 for LIFO, and on TVQA Episodic Memory Reader reaches about 65% accuracy with 60 memory entries versus roughly 55% for LIFO.

Memory Recall Agent (MRA) integrates a pixel‑input convolutional residual network, an LSTM working memory, a slot‑based episodic memory, an auxiliary contrastive loss, and jumpy backpropagation into a single reinforcement learning agent. Across the Memory Tasks Suite with train/holdout splits, Memory Recall Agent (MRA) attains the highest average human‑normalized performance compared to LSTM‑only IMPALA and other ablations, especially on tasks requiring episodic recall.

On Tiny Episodic Memories in Continual Learning combines Experience Replay, Reservoir Sampling, Ring Buffer, k-Means, and Mean of Features memory writing to jointly train on current-task data and a tiny episodic memory. On Split CIFAR with only 1 example per class, On Tiny Episodic Memories in Continual Learning reaches about 0.56 average accuracy, a +15.6 percentage point gain over FINETUNE and +15 percentage points over EWC.

Working Memory Graphs combines a Core vector, Factor vectors, and persistent Memo vectors processed by a multi layer Transformer to implement shortcut recurrence over past observations. On the Pathfinding task, Working Memory Graphs with Memos nearly matches a full history non recurrent Working Memory Graphs baseline and exceeds a GRU agent by roughly 9.5 percentage points in zero shot quiz accuracy on 24 step episodes.

COG combines Visual processing, Semantic processing, Visual short-term memory, and a Controller so SYS_NAME can parse instructions, attend over images, and maintain working memory. SYS_NAME reaches 96.8% overall accuracy on CLEVR versus 95.5% for CNN+LSTM+RN, while achieving 93.7% on canonical COG and strong zero-shot task generalization.

Working Memory model combines a topic memory, history memory, local memory, genre embedding, and Topic Trace mechanism inside a GRU encoder–decoder to dynamically read and write salient poem context. On Chinese quatrains, Working Memory model achieves BLEU 1.315 and perplexity 86 versus iPoet’s BLEU 0.425 and perplexity 138.

Deep Episodic Memory uses an encoder network E, reconstruction-decoder Dr, prediction-decoder Dp, latent vector V, and a matching and retrieval mechanism to turn raw video into episodic encodings that can be reconstructed and predicted. On ActivityNet, Deep Episodic Memory with PCA achieves 45.55% first-match precision versus 32.31% for ResNet-50 Fisher Vectors, a +13.24 percentage point gain.

Episodic Memory Deep Q-Networks (EMDQN) augments Qθ(s, a) with an inference target S, an episodic memory target H, and a memory table built via random projection and kd-tree lookup. On 57 Atari games at 40M frames, EMDQN achieves a 528.4% mean human-normalized score versus 151.2% for DQN and 144.8% for NEC.

Progressive Memory Banks for Incremental Domain Adaptation augments a BiLSTM with a directly parameterized memory bank, key value memory, and an attention based memory mechanism that is progressively expanded during incremental domain adaptation. On MultiNLI Fic→Gov, Progressive Memory Banks for Incremental Domain Adaptation with memory and vocabulary expansion reaches 67.55% on Fic and 70.82% on Gov, compared to 65.62% and 69.90% for fine tuning with no memory expansion.

Emotional Chatting Machine (ECM) augments a GRU encoder decoder with Emotion Category Embedding, Internal Memory, and External Memory to control emotional content in generated replies. On the Emotional STC dataset, ECM reaches 0.773 emotion accuracy vs 0.724 for Emb and 0.179 for Seq2Seq, while keeping perplexity comparable.

Gradient Episodic Memory stores task examples in episodic memory Mt, constrains updates via inequality constraints on past-task losses, and solves a small quadratic program (GEM QP) to project gradients. Gradient Episodic Memory achieves 0.654 ACC on Incremental CIFAR100 with 5,120 memory slots, compared to 0.508 ACC for iCARL.

Neural SLAM combines an LSTM, Localization and Motion Prediction, Data Association, Measurement Update, and Mapping over an external memory map to guide exploration policies. On 16×16 grid worlds, Neural SLAM achieves 13.732 average reward and 46/50 success episodes, a +6.536 reward gain over A3C-Nav2.

On the Long-Term Memory of Deep Recurrent Networks analyzes Recurrent Arithmetic Circuits, Start-End separation rank, grid tensors, and Tensor Network constructions to quantify how depth affects temporal expressivity. The main result proves depth-2 RACs achieve Start-End separation rank on the order of the multiset coefficient (min{M,R} + T/2 − 1 choose T/2), while depth-1 RACs are limited to rank min{R, M^{T/2}}.

RNN-EM augments a simple recurrent neural network with an external memory Mt, forget gate ft, update gate ut, and a cosine-similarity weight wt to read and write contextual representations. On the ATIS language understanding benchmark, RNN-EM attains 95.25% F1, beating the previous best LSTM result of 94.85% by 0.40 points.