All AI memory papers

On the Long-Term Memory of Deep Recurrent Networks analyzes Recurrent Arithmetic Circuits, Start-End separation rank, grid tensors, and Tensor Network constructions to quantify how depth affects temporal expressivity. The main result proves depth-2 RACs achieve Start-End separation rank on the order of the multiset coefficient (min{M,R} + T/2 − 1 choose T/2), while depth-1 RACs are limited to rank min{R, M^{T/2}}.

RNN-EM augments a simple recurrent neural network with an external memory Mt, forget gate ft, update gate ut, and a cosine-similarity weight wt to read and write contextual representations. On the ATIS language understanding benchmark, RNN-EM attains 95.25% F1, beating the previous best LSTM result of 94.85% by 0.40 points.