中间层知晓:从熵动力学检测越狱攻击
阅读原文· arxiv.org通过分析冻结大语言模型各层的 token 级预测熵轨迹,发现越狱攻击相关信息主要编码在中间层而非输出头。静态聚合熵统计量(如均值、方差)区分力弱,而反映熵随 token 位置演化的特征(如单调排名趋势分数)更具判别力。该信号在 Llama、Qwen、Gemma 等多个模型和对抗性基准上表现一致,无需额外训练。
Jailbreak attacks reveal a persistent weakness in aligned Large Language Models: carefully crafted prompts can elicit policy-violating responses despite safety training. While most defenses operate at the prompt or output level, it remains unclear how harmful intent is encoded within the model's internal representations. We investigate this question by analyzing token-level predictive entropy trajectories across layers of a frozen LLM using the logit lens. We find that static aggregate statistics of prompt-level entropy (e.g., mean, variance) carry little discriminative signal, whereas features capturing how entropy evolves across token positions, such as monotonic rank-based trend scores, are substantially more informative. Importantly, this signal is not uniform across model depth: it is concentrated in intermediate layers and degrades at the final layer, indicating that jailbreak-relevant structure is most pronounced in mid-network representations rather than at the output head. Across multiple models (Llama, Qwen, Gemma) and adversarial benchmarks, these entropy dynamics provide architecture-consistent separation without additional training. Together, our findings show that jailbreak behavior is reflected in structured intermediate uncertainty dynamics, clarifying both which entropy-derived features encode harmful intent and where in the network that signal is most pronounced.