---
translation_of: parametric-memory
lang: en
title: Writing every interaction into the model
summary: The personal AI I want has one requirement. Every interaction affects the next one, and memory lives in the model itself and not in a limited context. This post explains why models remember world knowledge but not you, how context, memory tables and weights relate, what existing work has achieved, and a two-layer design for parametric memory.
---

The personal AI I want has one requirement: every interaction between a user and the model affects the next one.

Today's large models cannot do this. After training, their weights are read-only. A model "remembers" you because an outside program finds past content and puts it back into the current context. Context has a limit, its contents must be read again every time, and reading them leaves no change behind.

So my idea is this: **memory should stop living in the context and move into the model's parameters.** This post continues "[Dynamic intelligence](/blog/dynamic-intelligence/)". That post defined the form in which running also writes to the model. This one covers its most concrete goal: how to make a model remember you.

I did not come up with this first. In 2024, AI-native Memory proposed training a personal model for each person. In 2026, User as Engram wrote user facts into parameters in under a millisecond, and Google's Sleep framework had models consolidate short-term memory into long-term parameters offline. This post looks at that work together, explains what I think the problem really is, and says what is still missing.

## Why models remember world knowledge

A model will not tell you that 1+1=3, which suggests it can remember knowledge reliably. The direct idea follows: write every interaction in as world knowledge.

The idea holds, but first we need to see where reliability comes from.

Training minimizes the average prediction error over all text. Something that appears a million times carries a million times the weight of something that appears once. A model's memory is therefore allocated by frequency, not by whether a fact is certain. That is why rare facts often come out wrong: on the 4,000 least popular questions in PopQA, GPT-3 was right 19% of the time.

This comes from the training objective. It is not a limit on what neural networks can do. Neural networks can fully fit random labels, which have no pattern to learn. How LoRA Remembers fine-tuned Qwen3-8B with a method designed for memorization and reached 100% exact recall on a phone-book task. Remembering rare facts reliably is possible when the training objective asks for it.

## How context, memory tables and weights relate

In 2020, Domingos pointed out that models trained by gradient descent approximate kernel machines. A kernel machine keeps its training examples and predicts by weighting them by similarity. Seen this way, the weights of a neural network are effectively a superposition of the training examples.

This gave me a more basic way to see the problem: **context, memory tables and weights all hold past examples, and they differ mainly in how compressed those examples are.**

| Form | Stored as | Writing | Reading | Transfer |
| --- | --- | --- | --- | --- |
| Context | Original text | Just add it | All reread each time, with a limit | Reasoned on the spot |
| Memory table | Raw examples, by address | One write | Only relevant entries | Weak |
| Weights | All examples, compressed together | Slow, needs repeated training | No lookup | Strong |

![Four forms of memory](/blog/assets/parametric-memory/memory-forms.en.svg)

Internalizing means memory is gradually compressed into shared representations. Memory table entries are parameters of the model too: the model reads them directly during its forward pass, without using context. That differs from RAG, which retrieves text that must be read again as input.

## Why weights alone are not enough

Fast writing and generalization conflict. Generalization needs similar content to share parameters, so that unseen questions land near existing knowledge. Fast writing needs content not to overlap, or a change in one place affects everything.

In 1995, McClelland and colleagues used this conflict to explain why the brain has two learning systems: the hippocampus records specific experiences quickly, and the neocortex extracts structure through slow, interleaved learning.

User as Engram gives a direct measurement. When user facts were written as a LoRA, prediction on unrelated text got clearly worse, with val bpb up 1.784. When written as a few rows of an Engram hash table, it rose 0.00005, a difference of about 33,000 times. A LoRA changes shared weights. A hash table changes an address.

Different fields have reached similar structures:

| Field | Fast layer | Slow layer |
| --- | --- | --- |
| Brain | Hippocampus | Neocortex |
| Databases | LSM-tree memtable | Merged files on disk |
| Recommender systems | User and item vectors, synced in minutes | Main model, synced daily |
| Large models (Sleep) | High-frequency memory modules | Low-frequency memory modules |
| Personal models | Memory table | Individual weights |

When you need both fast writes and good generalization, one layer is hard to make work. So I no longer see the problem as making fine-tuning fast enough. I split it in two: **write in real time, consolidate continuously.**

## What writing into weights takes

The second step is essentially automatic fine-tuning. The most complete work so far is Google's Sleep framework. A model takes in new information while awake. During sleep it pauses input and does two things:

- **Consolidation**: it distills short-term memory into newly added low-rank experts without overwriting old parameters, then clears the fast memory.
- **Dreaming**: it uses reinforcement learning to generate synthetic practice, filters it, and keeps training.

Its knowledge injection experiment is telling. Without context, the base model scores 31.9% on SQuAD, plain fine-tuning 33.4%, Sleep without dreaming 35.7%, and full Sleep 48.9%. Injecting 200 paragraphs in sequence, full Sleep reaches 46.2%.

Most of the gain comes from synthetic practice, and the result is still under 50%. This is far harder than the phone-book task above: a phone book only asks the model to reproduce what it has seen, while SQuAD asks new questions about a paragraph. Remembering something and being able to use it are different things.

Synthetic Document Finetuning (SDF) uses much more exposure: it generates 40,000 synthetic documents for each fact and trains a LoRA on them. Plausible facts often behave like real knowledge, carrying into related reasoning and holding up under challenge. Facts that contradict basic knowledge stay brittle.

Both results show that internalizing depends on enough synthetic exposure. Forty thousand documents is far too expensive, but I have two reasons to think personal memory needs fewer:

- **Consistent information integrates faster.** Kumaran, Hassabis and McClelland note that the speed of integration depends on how consistent new information is with existing knowledge. A fact like "the user lives in Shanghai" does not contradict world knowledge, so little existing structure has to change.
- **Replay can be selective.** Similarity-weighted interleaved learning replays only old examples similar to the new content, and keeps comparable results with far fewer of them. A 2026 preprint found that at the same replay share, spaced-repetition scheduling retains more old knowledge than uniform replay.

Nobody yet knows how much exposure one personal memory needs. Existing experiments run from 5 dreams per paragraph in Sleep and 30 rewrites in the Reversal Curse to 40,000 documents in SDF, several orders of magnitude apart. This is the most important unknown in this direction.

## My design: two layers of parametric memory

In the levels from the previous post, the memory table is level 1 and the individual weights are level 2.

| Layer | When it is written | What it holds | Purpose |
| --- | --- | --- | --- |
| Shared base | Every few months | Knowledge of the world | Shared by all users |
| Individual weights | Consolidated continuously in the background | Common facts, preferences, habits | Known without lookup |
| Memory table | Right after every interaction | All interactions, with time and source | The next interaction is affected at once |

![Two layers of parametric memory](/blog/assets/parametric-memory/two-layers.en.svg)

Three design points matter:

- **The memory table is the source of truth, and the individual weights are a rebuildable cache.** Base upgrades, deletion requests and wrong memories can all be handled by consolidating the weights again from the table. This is the biggest difference from Sleep, which clears fast memory after consolidation. Personal memory has to be editable and deletable, so the original records must stay.
- **What gets written is an event, not the truth.** The system records "the user said X on this day" and does not treat X as fact. Mistakes then stay out of world knowledge, and changes such as moving house can be told apart by time.
- **Not every memory has to be internalized.** All interactions stay retrievable in the memory table. Only frequent, patterned content moves into the weights.

Consolidation does not need to be reinvented. It can borrow directly from Sleep's knowledge distillation and dreaming.

## How this differs from existing work

| Work | What it achieved | What is missing |
| --- | --- | --- |
| Memory, RAG | Most mature; several systems self-report over 90 on LoCoMo | Memory sits outside the model and uses context |
| AI-native Memory, Second Me | One model per person, with memory in parameters | Offline training; results from a single pilot user |
| User as Engram | Writes in under 1ms; 100% direct recall; about 41% indirect reasoning | Bases up to 1.22B; 35% top-1 at 1,000 facts; 12.9% multi-hop recall when facts must really be combined |
| Sleep framework | Short-term memory is periodically consolidated into added parameters without overwriting old ones | Pauses input during sleep; under 50% on knowledge injection; does not discuss personal memory, fact updates or deletion |
| SDF | Implanted facts come close to real knowledge | 40,000 documents per fact; a one-off experiment |
| Knowledge editing | Changes facts directly in the weights | Related facts do not always update |
| Titans, TTT | Update internal state while running | Aimed at long context, not personal memory across sessions |

Each of these solves a part: some define the personal model, some achieve real-time writing, and Sleep closes the loop from short-term memory to long-term parameters. Sleep targets general continual learning. I care about personal memory: every interaction enters the model at once, memory has a durable source of truth, it can be updated and deleted, and results are measured on personal memory. This part is more about system design, and it is what I want to build.

## What I am unsure about

- **Retrieval may stay good enough.** RAG is more accurate at small scale and easier to audit. If the memory table recalls no better than RAG, the first layer's only advantage is not using context, and the real difference has to come from consolidation.
- **Interconnected memory is unsolved.** Multi-hop recall in Engram is low, and SDF only tested fairly simple facts. A person's long-term memory is interconnected, and its scale and structure go beyond existing experiments.
- **Literally unlimited is out of reach.** The memory table can keep growing, but I do not expect every memory to become as reliable as world knowledge. A realistic goal is that every interaction enters the model and affects later conversations, all memories can be retrieved, and frequent memories are gradually internalized.
- **The design in this post has not been tested.**

## Next

The next step is a personal memory benchmark that runs every interaction through one long continuous conversation and measures recall, reasoning, updates to old facts, and cost as history grows. It will first measure RAG, automatic fine-tuning, Engram-style writing and Sleep-style consolidation, then answer how much exposure one memory needs.

---

Note: updated on November 2, 2026. The first version said I had found no complete system connecting real-time writing and continuous consolidation. That missed Google's Sleep framework and has been corrected.

References:

- Personal models: [AI-native Memory](https://arxiv.org/abs/2406.18312), [Second Me](https://arxiv.org/abs/2503.08102), [User as Engram](https://arxiv.org/abs/2606.19172)
- Parametric memory: [Language Models Need Sleep](https://arxiv.org/abs/2606.03979), [Sleep framework post](https://research.google/blog/the-sleep-framework-a-new-way-for-llms-to-retain-knowledge-in-continual-learning/), [How LoRA Remembers?](https://arxiv.org/abs/2605.30260), [Believe It or Not (SDF)](https://arxiv.org/abs/2510.17941), [The Reversal Curse](https://arxiv.org/abs/2309.12288), [MEMIT](https://arxiv.org/abs/2210.07229)
- Memory theory: [Every Model Learned by Gradient Descent Is Approximately a Kernel Machine](https://arxiv.org/abs/2012.00152), [Complementary learning systems (1995)](https://stanford.edu/~jlmcc/papers/McCMcNaughtonOReilly95.pdf), [What Learning Systems Do Intelligent Agents Need?](https://web.stanford.edu/~jlmcc/papers/KumaranHassabisMcClelland16FinalMS.pdf)
- Replay: [Similarity-weighted interleaved learning](https://escholarship.org/uc/item/1rw1b1m4), [When to Review](https://arxiv.org/abs/2608.17530)
- Others: [PopQA](https://arxiv.org/abs/2212.10511), [Understanding Deep Learning Requires Rethinking Generalization](https://arxiv.org/abs/1611.03530), [Monolith](https://arxiv.org/abs/2209.07663), [LoCoMo](https://arxiv.org/abs/2402.17753), [Titans + MIRAS](https://research.google/blog/titans-miras-helping-ai-have-long-term-memory/), [TTT](https://arxiv.org/abs/2407.04620)