MemTrace:大语言模型记忆系统的错误追踪与归因
阅读原文· arxiv.org本文研究了大语言模型记忆系统中错误追踪与归因的新问题。我们提出了MemTrace框架,它将记忆管线转化为可执行的记忆演化图,以实现操作信息流的细粒度追踪。我们构建了MemTraceBench基准,数据来自Long-Context、RAG、Mem0和EverMemOS等代表性系统,用于系统研究记忆失效模式。此外,我们引入了一种自动归因方法,通过迭代追踪操作子图来定位失败根因。分析表明,记忆失败具有系统性,源于信息丢失与检索错位等操作级问题。利用这些细粒度归因信号指导下游提示词优化,可构建一个自动纠错的闭环系统,将端到端任务性能最高提升7.62%。
Memory is essential for enabling large language models to support long-horizon reasoning, yet existing memory systems remain unreliable and difficult to debug. Tracing memory's dynamic evolution is crucial to understand how information is synthesized, propagated, or corrupted over time. In this work, we study the new problem of error tracing and attribution in LLM memory systems. We propose a novel framework that transforms memory pipelines into executable memory evolution graphs, enabling fine-grained tracing of operational information flow. We then construct MemTraceBench, a benchmark collected from representative memory systems such as Long-Context, RAG, Mem0, and EverMemOS, to systematically study memory failure modes. We further introduce an automatic attribution method that iteratively traces operation subgraphs to pinpoint the root cause of any failed case. Our analysis reveals that memory failures are systematic, stemming from operation-level issues like information loss and retrieval misalignment. Crucially, we leverage these fine-grained attribution signals to guide downstream prompt optimization, establishing a closed-loop system that automatically corrects faults and boosts end-task performance by up to 7.62%. Code will be released at https://github.com/zjunlp/MemTrace.