CORE 论文提出用重排器蒸馏改进多模态嵌入的组合推理

HuggingFace Daily Papers(社区热门论文)·2026-09-03 08:00·1天前
AI 导读

论文提出 CORE,通过合成五个组合匹配级别的候选列表和 Rank-KL 目标,把多模态模型作为重排器时的组合判断蒸馏进嵌入模型。

HuggingFace Daily Papers(社区热门论文)
37AI 编辑部评分,满分 100

CORE 论文提出用重排器蒸馏改进多模态嵌入的组合推理

2026-09-03 08:00· 1天前
AI 导读

论文提出 CORE,通过合成五个组合匹配级别的候选列表和 Rank-KL 目标,把多模态模型作为重排器时的组合判断蒸馏进嵌入模型。

MLLM-based embedding models remain limited in compositional retrieval, often failing to distinguish scenes containing the same concepts but different attribute-object bindings. Yet the same backbone can resolve such distinctions when used as a cross-attentive reranker, motivating us to distill its compositional judgments into the embedding model. We propose CORE, which synthesizes candidate lists spanning five compositional matching levels and introduces a Rank-KL objective that trains the embedding model to reproduce the reranker's fine-grained ranking. We further introduce a graded evaluation protocol and compare contrastive learning, pairwise CoSENT, and listwise Rank-KL under the same data and tuning budget. Our comparison shows that both CoSENT and Rank-KL use the multi-level supervision more effectively than contrastive learning, with Rank-KL achieving the strongest overall performance. Across three compositional reasoning benchmarks (COLA, SUGARCREPE++, NEGBENCH), CORE-RERANKER-8B achieves an 82.7% total average, outperforming Jina-Reranker by 10.7 points, while CORE-EMBED-8B achieves the best total average (0.666) among all evaluated embedding models. The improvements transfer to the MCMR benchmark without sacrificing retrieval performance on COCO and Flickr30K.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org