Decoding-Level Taboo:面向 LLM 鲁棒性的诊断压力测试

HuggingFace Daily Papers(社区热门论文)·2026-08-10 08:00·15天前
AI 导读

论文提出 Decoding-Level Taboo,一种零提示诊断压力测试,在运行时直接干预 logit 空间,通过动态屏蔽词边界处的主要候选 token,迫使模型偏离常规生成路径。对多个开源模型家族的评估显示,偏离路径的鲁棒性受参数规模与后训练指令对齐影响显著,且通常随模型规模与对齐程度提升而增强。该方法还可用于生成多样化合成数据集、压力测试运行时安全护栏及部署前审计模型可靠性。

HuggingFace Daily Papers(社区热门论文)
45AI 编辑部评分,满分 100

Decoding-Level Taboo:面向 LLM 鲁棒性的诊断压力测试

2026-08-10 08:00· 15天前
AI 导读

论文提出 Decoding-Level Taboo,一种零提示诊断压力测试,在运行时直接干预 logit 空间,通过动态屏蔽词边界处的主要候选 token,迫使模型偏离常规生成路径。对多个开源模型家族的评估显示,偏离路径的鲁棒性受参数规模与后训练指令对齐影响显著,且通常随模型规模与对齐程度提升而增强。该方法还可用于生成多样化合成数据集、压力测试运行时安全护栏及部署前审计模型可靠性。

Large language model evaluations typically focus on performance under nominal conditions, creating an illusion of capability where models comfortably walk a narrow, highly optimized generation corridor. In real-world deployments, however, complex system prompts, safety guardrails, and structural constraints continuously force models off this nominal path, driving a divergence between benchmark scores and deployment performance. To address this issue, we introduce Decoding-Level Taboo, a zero-prompt diagnostic stress test that intervenes directly in logit space at runtime, forcing models out of their nominal paths. By dynamically masking primary candidate tokens at word boundaries, Taboo forces machine circumlocution. Evaluating Taboo across several open-weight model families reveals that off-path robustness is heavily influenced by both parameter scale and post-training instruction alignment, with robustness generally improving with model size and alignment. Beyond the results presented in this paper, Taboo provides a novel primitive for generating diverse synthetic datasets, stress-testing runtime safety guardrails, and auditing model reliability prior to real-world deployment.

来源:HuggingFace Daily Papers(社区热门论文)· arxiv.org