# ELDR： 面向PD分离式MoE服务的专家局部性感知解码路由

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-07-01 08:00
- AIHOT 分数：39
- AIHOT 链接：https://aihot.virxact.com/items/cmr36nkpd012nsly0kunt9vmv
- 原文链接：https://arxiv.org/abs/2607.00466

## AI 摘要

ELDR是为预填充-解码分离式MoE服务设计的解码路由算法。它从请求预填充阶段提取专家激活特征，构建预测生成阶段所需专家的签名，通过离线平衡K-means聚类将签名空间分配到解码节点，在线路由时优先将请求发往签名匹配且负载最轻的节点。签名缓存与KV缓存以KV-block粒度协同索引，保证前缀缓存下签名准确。在vLLM实现，最高40块GPU部署测试，相比四种负载均衡基线中最强的一种，在三个MoE模型和两种负载上降低中位TPOT 5.9–13.9%，模型输出不变。

## 正文

In prefill-decode (PD) disaggregated LLM serving, each request is assigned to a decode worker after prefill. Existing decode routers balance only load; for mixture-of-experts (MoE) models this is incomplete: equally loaded workers can differ in latency, since each decode step loads the weights of every distinct expert its batch activates. We present ELDR, an expert-locality-aware decode router for PD-disaggregated MoE serving. From a request's prefill expert activations, ELDR builds an expert signature predicting the experts it will activate during generation. Offline, balanced K-means partitions signature space across decode workers; online, locality-band routing sends each request to the least-loaded worker among those best matching its signature. A signature cache, co-indexed with the KV cache at KV-block granularity, keeps signatures exact under prefix caching. Implemented in vLLM and evaluated on deployments of up to 40 GPUs, ELDR reduces median TPOT by 5.9-13.9% over the strongest of four load-balancing baselines across three MoE models and two workloads, with model outputs unchanged.