AuthorsPeng Wang, Zexi Li, Ningyu Zhang et al.
TL;DR
WISE uses a dual parametric memory with side memory plus knowledge sharding and Ties-Merge routing to achieve 0.90 average reliability, generalization, and locality on ZsRE at T = 100 edits, beating MEMIT-MASS by 0.14.
Read our summary here, or open the publisher PDF on the next tab.
THE PROBLEM
WISE shows that editing either long term memory or working memory alone creates an impossible triangle where reliability, generalization, and locality cannot all be high.
On ZsRE with 100 continual edits, long term editors like ROME and FT-EWC lose locality, while working memory editors like GRACE lose generalization, degrading real world QA updates.
HOW IT WORKS
WISE builds a dual parametric memory using a Side Memory copied from the FFN value matrix, plus Main Memory, Memory Routing, and Knowledge Sharding and Merging modules.
You can think of WISE like a brain with a stable cortex for pretrained knowledge and a separate mid term region that accumulates edits, connected by a learned router instead of a single shared weight matrix.
This design lets WISE keep pretrained behavior intact while routing only edited queries through dense sharded subspaces, something a plain context window or single parameter space cannot provide.
DIAGRAM
This diagram shows how WISE routes each query between main and side memory using the activation indicator Δact(x) and threshold ϵ during inference.
DIAGRAM
This diagram shows how WISE trains side memory with sharded edits, merges subspaces, and evaluates reliability, generalization, and locality on ZsRE.
PROCESS
Side Memory Design
WISE initializes Side Memory by copying the FFN value matrix Wv into Wv′ at a mid to late layer, separating pretrained and edited knowledge.
Routing Between Side Memories and Main Memory
WISE uses Memory Routing with the activation indicator Δact(x) and margin based loss La to learn when to send queries through side memory versus main memory.
Knowledge in Random Memory Subspaces
WISE applies Knowledge Sharding by generating random gradient masks Mi with ratio ρ and training k side memory copies on different edit shards.
Knowledge Merging
WISE performs Knowledge Merging using Ties-Merge on edit vectors τe to combine subspace memories into one side memory while preserving previous edits.
KEY CONTRIBUTIONS
Impossible Triangle of Lifelong Model Editing
WISE identifies that editing only long term or working memory yields an impossible triangle where reliability, generalization, and locality cannot all be high in continual settings.
WISE Dual Parametric Memory Scheme
WISE introduces a Side Memory plus Memory Routing design that stores edits in a separate FFN value matrix and routes queries by Δact(x), achieving mid term memory behavior.
Knowledge Sharding and Merging Mechanism
WISE proposes Knowledge Sharding and Merging with random subspace masks and Ties-Merge, showing that 20% FFN parameters can store at least 500 edits without conflicts.
RESULTS
Avg. (ZsRE T=100 LLaMA-2-7B)
0.90
+0.14 over MEMIT-MASS
Rel. (ZsRE T=100 LLaMA-2-7B)
0.90
+0.14 over MEMIT-MASS
Gen. (ZsRE T=100 LLaMA-2-7B)
0.81
+0.13 over MEMIT-MASS
Loc. (ZsRE T=100 LLaMA-2-7B)
1.00
+0.15 over MEMIT-MASS
These numbers come from the ZsRE QA benchmark under T = 100 continual edits on LLaMA-2-7B, which tests reliability, paraphrase generalization, and locality. The MAIN_RESULT shows that WISE simultaneously maintains 0.90 average score and 1.00 locality, while MEMIT-MASS reaches only 0.76 average and 0.85 locality.
BENCHMARK
These numbers come from the ZsRE QA benchmark under T = 100 continual edits on LLaMA-2-7B, which tests reliability, paraphrase generalization, and locality. The MAIN_RESULT shows that WISE simultaneously maintains 0.90 average score and 1.00 locality, while MEMIT-MASS reaches only 0.76 average and 0.85 locality.
BENCHMARK
Average score (Rel., Gen., Loc. mean) on ZsRE with 100 continual edits for LLaMA-2-7B.
KEY INSIGHT
WISE shows that a side memory using only 20% of FFN parameters can handle at least 500 edits while keeping locality at 1.00.
This is surprising because many assume more edited parameters always help, but WISE finds that higher knowledge density in small subspaces can improve generalization instead of causing overfitting.
WHY IT MATTERS
WISE unlocks a practical way to perform thousands of edits on deployed LLMs while preserving pretrained behavior and paraphrase generalization.
Builders can now maintain a stable base model and plug in discardable side memories for different domains or time periods, enabling safer, reversible, and lifelong model editing without full retraining.
Nikhil Verma
· 2026
Focus Agent adds start_focus, complete_focus, a persistent Knowledge block, and an optimized Persistent Bash plus String-Replace Editor scaffold to actively compress context during long software-engineering tasks. On five hard SWE-bench Lite instances against a Baseline ReAct agent, Focus Agent achieves 22.7% token reduction (14.9M → 11.5M) while matching 3/5 = 60% task success.
Xingyu Lyu, Jianfeng He et al.
· 2026
ADAM combines Anchor extraction, Distribution estimation, Anchor selection, and Query generation to adaptively probe agent memory via an auxiliary generator and entropy based selection. On the EHRAgent benchmark with Llama2-7b-chat, ADAM reaches EQ=77 and ASR=1.00, compared to MEXTRA’s EQ=44 and ASR=0.89.
Shannan Yan, Jingchen Ni et al.
· 2026
AdaMem organizes dialogue history into Working Memory, Episodic Memory, Persona Memory, and Graph Memory coordinated by a Memory Agent, Research Agent, and Working Agent. On LoCoMo with GPT-4.1-mini, AdaMem achieves 44.65 F1 overall, beating the best baseline LangMem at 41.76 F1 by +2.89.
Answers use this explainer on Memory Papers.
Checking…