EvoEmbedding:用于长上下文检索和智能体记忆的可演化表征
阅读原文· arxiv.org现有嵌入模型是静态的,孤立编码文本片段,忽略上下文与时间顺序。EvoEmbedding 生成可演化表征,专为信息动态、序列化的长上下文场景设计。模型顺序处理输入时维护连续更新的隐记忆,并与原始内容共同生成演化嵌入,使同一查询能根据上下文检索不同目标。为联合优化隐记忆与检索,构建了训练数据集 EvoTrain‑180K,并引入记忆队列防止表征崩塌,结合分段批处理加速训练 3.8 倍。实验表明,该模型在长上下文检索基准上超越更大规模专用模型,并可泛化至上下文长 10 倍的下游任务;集成至简单 RAG 管线即可超越专用智能体记忆系统。
Existing embedding models are inherently static: they encode text segments in isolation, ignoring their surrounding context and temporal order. This paper introduces EvoEmbedding, a novel embedding model that generates evolvable representations for retrieval. It is tailored for long-context scenarios, where information is dynamic, sequential, and requires continuous state tracking. Our design is simple: EvoEmbedding maintains a continuously updated latent memory as it sequentially processes inputs, and uses it alongside the raw content to jointly generate evolvable embeddings. Consequently, for the same query, our model adapts its representation to retrieve distinct targets based on the evolving context, going beyond static semantic search. To equip the model with this capability, we construct EvoTrain-180K, a diverse dataset for the joint optimization of latent memory and retrieval. Furthermore, we introduce a memory queue to prevent representation collapse during recurrent encoding, alongside segment-batching techniques that tackle significant length variance and accelerate training by 3.8times. Extensive experiments show that our model not only outperforms larger-scale specialists (e.g., Qwen3-Embedding-8B and KaLM-Embedding-Gemma3-12B) across a range of long-context retrieval benchmarks, but also generalizes well to downstream tasks (e.g., personalization) with contexts 10times longer than its training window. Notably, EvoEmbedding seamlessly integrates into agentic workflows to boost performance. For instance, a naive RAG pipeline equipped with our model surpasses dedicated agentic memory systems. Project Page: https://clare-nie.github.io/EvoEmbedding.