# 揭示逻辑推理的算法演绎电路

- 来源：HuggingFace Daily Papers（社区热门论文）
- 发布时间：2026-05-27 08:00
- AIHOT 分数：68
- AIHOT 链接：https://aihot.virxact.com/items/cmppaj22e0db0slv401tok1im
- 原文链接：https://arxiv.org/abs/2605.27824

## AI 摘要

研究表明，大语言模型（LLMs）在符号辅助的 Chain-of-Thought（CoT）提示下，能够通过类似图遍历的算法实现逻辑推理。本研究旨在定位负责具体推理步骤的注意力头，并分析它们之间传递的信息类型。研究发现，在CoT提示框架下，引导推理进程的token位置常伴有低置信度分数。通过因果中介分析，识别出了负责特定推理模式的注意力头（约占总头数的3%）。进一步分析表明，LLMs通过专用注意力头获取单个子任务的事实与规则信息，而更高层的注意力头则主要负责信息整合与全局推理策略（如图遍历算法）的涌现，以协调多个中间步骤来解决整体任务。

## 正文

Recent studies have shown that Large Language Models (LLMs) can achieve strong reasoning performance by incorporating functional symbolic representations that abstractly describe graph traversal algorithms and step-by-step reasoning in few-shot learning settings. However, it remains unclear how LLMs genuinely understand the abstract meaning of each reasoning step and the overall algorithm from only a limited number of demonstrations. This work aims to localize the attention heads responsible for individual reasoning steps and characterize the types of information transferred among them. We first align constituent reasoning steps with their corresponding token logits under a symbolic-aided Chain-of-Thought (CoT) prompting framework. Our analysis shows that token positions that steer the reasoning process are associated with low confidence scores caused by constraints on satisfying reasoning behavior patterns in demonstrations. We then adopt causal mediation analysis techniques to identify the attention heads responsible for these patterns. In addition, our findings indicate that LLMs retrieve factual and rule-based information for individual sub-reasoning tasks through specialized attention heads (approximately 3% total heads), whereas higher layers predominantly facilitate information integration and the emergence of global reasoning strategies (e.g., graph traversal algorithms) that coordinate multiple intermediate reasoning steps to solve the overall task.