Brilliant paper from LinkedIn.
(bookmark it)
I have been saying that memory is one of the most challenging things to get right when building agents. It's an are you want to invest time in optimizing.
This paper focuses on whether memory is portable, which is crucial given that more users are using agents across models, agents, and providers.
If you have ever upgraded the model behind an agent and watched it start forgetting things it used to know, this one is for you.
In this work, the same agent history is stored four ways, kept verbatim for long context reading, chunked for retrieval, compressed into model-written notes, and normalized into a fixed-schema knowledge graph. Then the model reading it gets swapped.
Fixed-schema knowledge graphs barely move, 0.0004 accuracy change across a writer swap. Model-written notes move by +9.91 or -13.28 points depending on which direction you migrate, so the same pair of models gives you opposite results based on which one wrote the memory.
Partial embedding migration is also expensive. A 50/50 mixed index captures only 4.96 of the 11.90 points that full re-embedding gives you.
The two failure modes have different causes. 80% of the notes deficit comes from information lost when the note was written. 81% of the retrieval deficit comes from retrieval missing the right chunk.
Repair needs the raw history. Store-only repair of notes failed to reach 90% recovery in all 48 test cases, while keeping the source history recovered 34 of 48.