MLLM-based embedding models remain limited in compositional retrieval, often failing to distinguish scenes containing the same concepts but different attribute-object bindings. Yet the same backbone can resolve such distinctions when used as a cross-attentive reranker, motivating us to distill its compositional judgments into the embedding model. We propose CORE, which synthesizes candidate lists spanning five compositional matching levels and introduces a Rank-KL objective that trains the embedding model to reproduce the reranker's fine-grained ranking. We further introduce a graded evaluation protocol and compare contrastive learning, pairwise CoSENT, and listwise Rank-KL under the same data and tuning budget. Our comparison shows that both CoSENT and Rank-KL use the multi-level supervision more effectively than contrastive learning, with Rank-KL achieving the strongest overall performance. Across three compositional reasoning benchmarks (COLA, SUGARCREPE++, NEGBENCH), CORE-RERANKER-8B achieves an 82.7% total average, outperforming Jina-Reranker by 10.7 points, while CORE-EMBED-8B achieves the best total average (0.666) among all evaluated embedding models. The improvements transfer to the MCMR benchmark without sacrificing retrieval performance on COCO and Flickr30K.
CORE 论文提出用重排器蒸馏改进多模态嵌入的组合推理
AI 导读
论文提出 CORE,通过合成五个组合匹配级别的候选列表和 Rank-KL 目标,把多模态模型作为重排器时的组合判断蒸馏进嵌入模型。
HuggingFace Daily Papers(社区热门论文)
37
AI 编辑部评分,满分 100CORE 论文提出用重排器蒸馏改进多模态嵌入的组合推理
论文提出 CORE,通过合成五个组合匹配级别的候选列表和 Rank-KL 目标,把多模态模型作为重排器时的组合判断蒸馏进嵌入模型。
原文 · 保持原样,未翻译原文 · 未翻译
来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org