我们准备好迎接智能体原生记忆系统了吗?
阅读原文· arxiv.org从数据管理视角对LLM智能体记忆系统进行系统性实验研究,将其分解为表示与存储、提取、检索与路由、维护四个核心模块。评估了12个代表性记忆系统和两个参考基线,覆盖5个基准工作负载共11个数据集。端到端结果显示无单一架构占优,效果取决于记忆结构与工作负载瓶颈的对齐。细粒度消融实验量化了各模块对表示保真度、检索精度、更新正确性和长期稳定性的影响。实际工作负载下局部维护比全局重组更具成本效益。代码已公开。
Memory for large language model (LLM) agents has rapidly evolved from simple retrieval-augmented mechanisms into a data management system that supports persistent information storage, retrieval, update, consolidation, and dynamic lifecycle governance throughout agent execution. Despite this evolution, existing evaluations still benchmark agent memory mainly through end-to-end task success metrics (e.g., F1, BLEU), while treating the underlying system as a monolithic black box. As a result, critical system-level concerns, including operational costs, architectural trade-offs across memory modules, and robustness under dynamic knowledge updates, remain insufficiently explored. In this paper, we present a systematic experimental study of agent memory from a data management perspective. We propose an analytical framework that decomposes agent memory into four core modules: memory representation and storage, extraction, retrieval and routing, and maintenance. Under this framework, we evaluate 12 representative memory systems and two reference baselines across five benchmark workloads spanning 11 datasets. Our extensive end-to-end evaluation shows that no single architecture dominates across all scenarios; instead, effectiveness depends heavily on how well the memory structure aligns with the workload bottleneck. Furthermore, through fine-grained ablation studies, we quantify their individual effects on representation fidelity, retrieval precision, update correctness, and long-horizon stability. Finally, we reveal cost-performance trade-offs under realistic workloads, showing localized maintenance is more cost-efficient than global reorganization. Based on these findings, we identify promising directions towards building truly agent-native memory systems. The code is publicly available at https://github.com/OpenDataBox/MemoryData.