FlashPrefill V2:面向长上下文 LLM 服务的块稀疏预填充注意力

HuggingFace Daily Papers(社区热门论文)·2026-08-20 16:02·5天前
AI 导读

FlashPrefill V2 将稀疏注意力从算法原型推进到实用化长上下文服务,通过均值校正项抑制近似误差,并采用 PackGQA 内存访问、warp 特化和 pingpong 流水线,对齐 FlashAttention-3/4 并支持 FP8 推理。

HuggingFace Daily Papers(社区热门论文)
56AI 编辑部评分,满分 100

FlashPrefill V2:面向长上下文 LLM 服务的块稀疏预填充注意力

2026-08-20 16:02· 5天前
AI 导读

FlashPrefill V2 将稀疏注意力从算法原型推进到实用化长上下文服务,通过均值校正项抑制近似误差,并采用 PackGQA 内存访问、warp 特化和 pingpong 流水线,对齐 FlashAttention-3/4 并支持 FP8 推理。

Long-context modeling is a pivotal capability for Large Language Models, yet the quadratic complexity of attention remains a critical bottleneck, particularly during the compute-intensive prefilling phase. Our previous work, FlashPrefill, mitigates this cost through instantaneous pattern discovery and max-based dynamic thresholding; however, it remains an algorithmic prototype that is still distant from production deployment. In this paper, we present FlashPrefill V2, which evolves FlashPrefill from a prototype toward practical long-context serving along three dimensions. First, we introduce a mean correction term that effectively suppresses the approximation error, keeping performance degradation manageable even at extreme sparsity levels. Second, we redesign the sparse attention operator with PackGQA memory access, warp specialization, and pingpong pipelining, fully aligning with the latest FlashAttention-3/4 implementations and supporting FP8 inference to meet practical quantization requirements. Third, FlashPrefill V2 natively supports paged KV cache and continuous batching, allowing integration as an attention backend in modern inference frameworks such as SGLang. Extensive evaluations on NVIDIA H20 GPUs---among the most widely deployed inference accelerators---demonstrate that FlashPrefill V2 delivers up to 47.26x and 27.19x speedups over FlashAttention-2 at 128K context length under FP8 and BF16 precision, respectively, and, in FP8, still achieves a 30.49x speedup against an FA3/4-aligned dense baseline.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org