SKIM:面向大语言模型的自适应多分辨率程序性技能压缩框架
阅读原文· arxiv.org可复用的自然语言技能被频繁调用时,将完整文本放入每个上下文会显著增加预填充成本和延迟。SKIM(SKIll coMpression)是一种自适应多分辨率软token压缩框架,专为程序性技能设计。它根据每项技能的复杂度生成不同数量的软token,在提升LLM推理效率的同时保留技能使用效果。实验表明,SKIM将技能压缩至原始token长度的30%到60%,且在任务性能上优于现有压缩方法。代码已开源。
Large language models (LLMs) are widely used to tackle complex tasks with autonomous workflows. Recently, reusable natural language skills have emerged as a popular paradigm to inject procedural knowledge into LLM applications. Since popular skills are often invoked repeatedly, placing their full text in every context significantly increases prefill cost and latency. While text compression techniques have the potential to solve this problem, most existing methods are designed to compress factual knowledge in documents instead of procedural knowledge, making them insufficient for skill compression. In this paper, we argue that an effective skill compression method should: 1) preserve logical dependencies among workflows and tool protocols, 2) enable lightweight, offline compression for frequently updated community skills, and 3) be adaptable to varying complexities across skills. To address this, we present SKIM (SKIll coMpression), an adaptive multi-resolution soft token compression framework for procedural skills. Depending on the complexity of each skill, SKIM creates different numbers of soft tokens that not only improve the efficiency of LLM inference, but also preserve the effectiveness of skill usage. Experiments indicate that SKIM compresses skills to 30 to 60 percent of their original token length while preserving task performance better than existing compression methods.We have released our code at https://github.com/bebr2/SKIM .