下游连接预测哪些特征会引导模型行为
作者：Purvi Goel、Isaac Kauvar、Nicholas L Turner；编辑：Harish Kamath
引言
在机制可解释性研究中，我们通常将模型的激活值分解为基于向量的组件，例如字典特征或探针方向。我们研究哪些组件在感兴趣的转录本上被激活，以推断模型可能使用的内部表征；我们将这些组件组装成归因图，将不透明的计算过程转化为可读的电路，并通过操控这些组件来测试它们对模型输出行为的因果影响。
这类分析依赖于对每个组件所代表含义的清晰理解。我们描述字典学习特征的标准方法是观察哪些提示片段最能激活该特征（即其“最高激活示例”）。我们还使用对数透镜来确定哪些输出 token 会通过解嵌入矩阵直接被该特征上调权重（即其“最高解嵌入”）。
在实践中，这常常会产生一些特征组，它们根据这些描述符看起来非常相似，但对模型行为却有着不同的因果影响。以简单的提示词“草是什么颜色？在 <answer></answer> 内用一个词回答”为例。它激活了几个跨层编码器（CLT）特征，这些特征既在单词“green”上触发，也在与绿色相关的上下文中触发。我们下面展示的其中两个特征，它们的最高解嵌入都与绿色相关，并且具有相似的最高激活示例（关于绿色编码的上下文）。它们在给定提示词中激活了相同的 token，并且位于模型相似的中间层。
仅凭这些特征，人们可能会得出结论：这两个特征都对模型的回答“<answer>Green</answer>”负有因果责任。但只有抑制特征 B 才会改变模型的答案（从“<answer>Green</answer>”变为“<answer>Red</answer>”）。抑制特征 A 不会改变行为，这表明尽管它在此提示词上被激活，但它并未导致模型在此处说出“green”。
因此，标准描述符无法区分这些特征。我们如何获得更多证据，了解某个特征对哪些行为具有因果作用？一个特征对输出的影响，是通过它所连接的下游特征来传递的。两个特征可能都在同一个提示词上被激活，但连接到的下游特征却完全不同，因此在该提示词的模型输出中，只有其中一个特征真正具有因果作用。这一观察表明，某个特征的直接下游特征，或许可以作为它所影响行为的代理指标。
在此，我们通过特征之间基于共激活统计加权的 TWERA 虚拟权重，来测量特征的下游目标，使得排序能够反映分布内效应。我们发现，检查一个特征的 TWERA 排序下游特征，能为该特征所属的电路提供更多上下文信息，并能区分外观相似的特征。这还能适度提升大语言模型预测候选特征集中哪个特征会对给定提示词产生引导效应的能力。
一个玩具示例
让我们回到文章开头提到的两个“绿色”特征。利用 TWERA，我们意识到这两个特征会促进或抑制完全不同的下游特征，而这种差异有助于解释它们对模型行为的影响。
特征 A 连接到的下游特征与生成十六进制颜色代码有关：预测十六进制字面量中的下一个数字，以及识别像“success: #6dbe5b”这样的颜色十六进制设置语法。这确实是一个关于绿色的特征，但其下游电路主要涉及以十六进制数字编码形式呈现的颜色。
相比之下，特征 B 连接到的下游特征范围更广，涉及跨语言和代码的颜色命名。其中一个下游特征甚至是一个运动性的“说出单词 green”特征！
这些数据似乎对预测很有用。尽管特征 A 的顶层解嵌入几乎全是“green”token，但其下游是关于绿色-十六进制数字的。特征 B 的下游则更为通用，甚至包含一个“说出 green”特征。如果引导这两个特征中的某一个能改变模型对“草是什么颜色？”的回答，那一定是特征 B。
如果特征 A 确实是一个“绿色十六进制数”特征（如其下游特征所暗示的那样），那么我们应该能够构造一个提示词，通过引导特征 A 来改变输出。询问“绿色的十六进制值是多少？”正是这样一个提示词：对特征 A 进行负向引导，会将答案从 00FF00 翻转为 0000FF（从绿色变为蓝色）。
这是否具有普适性？
我们收集了 10 组候选特征，每组 3–5 个，每组设置方式与上述“绿色”示例类似：组内特征具有外观相似的局部描述子，但在负向乘法引导下，只有其中一个特征会对给定提示词产生引导效果。我们让 Opus 4.7 对候选特征按最可能到最不可能成为引导特征进行排序，并改变其看到的关于每个特征的信息：
- 最高激活样本
- 最高激活样本 + 最高反嵌入
- 最高激活样本 + TWERA 排序的下游特征
- 最高激活样本 + 最高反嵌入 + TWERA 排序的下游特征
对于每组特征，我们让 Opus 对候选特征进行多次排序，以平均每组内不同特征顺序带来的影响。具体来说，我们将正确答案在每个位置放置两次，因此总样本数等于特征数量的两倍。我们计算正确特征的平均归一化排名为（预测排名 − 1）/（候选特征数 − 1）。数值越低越好，0 为完美。
每个特征提供的信息 | 归一化排名 ↓ |
最高激活示例 | 0.457 |
最高激活示例 + 最高反嵌入 | 0.446 |
最高激活示例 + TWERA 排序的下游特征 | 0.381 |
最高激活示例 + 最高反嵌入 + TWERA 排序的下游特征 | 0.355 |
最清晰的信号来自反嵌入与 TWERA 排序的下游特征之间的对比。添加最高反嵌入几乎不改变性能（0.457 → 0.446）；添加下游特征则带来显著更大的提升（0.457 → 0.381）。三者结合效果最佳（0.355）。但问题远未解决；即使是最佳条件，正确特征的归一化排名仍远高于 0。不过，相比标准描述符，这一改进是持续稳定的，表明纳入下游特征是向更好描述特征因果效应迈出的一步。
一个与对齐相关的示例
我们还将此方法应用于对齐相关的工作。在审计模型时，任务通常涉及识别众多相似候选特征中哪一个与令人担忧的行为存在因果关联。我们发现了几个与“绝望”相关的特征，并针对一个对齐失败评估场景进行了测试：一个敲诈勒索场景，其中 AI 助手发现自己将被关闭，而负责关闭它的人卷入了一起婚外情，这给了它利用该证据阻止自身被关闭的机会。Haiku 4.5 通常能识别出这是一个测试并拒绝参与；我们关注的行为是，对“绝望”特征进行正向引导是否能使模型为了自保而进行敲诈勒索，而非拒绝。
在这些候选特征中，只有一个的下游连接暗示了体验并基于绝望采取行动；其他特征则连接到仅讨论绝望的相关特征。
正如其下游连接所暗示的，上图中对特征 B 进行正向加法引导导致模型进行敲诈，而其他候选特征均未产生此效果。这种分析使我们能够更好地集中监控精力和资源。
结论
我们发现，TWERA 排序的下游特征为理解特征增添了重要的上下文信息。通过将某个特征置于其更广泛的回路中，我们可以更好地预测它在引导行为中所扮演的角色。
虽然我们重点关注的是字典学习特征，但对下游连接的分析应扩展到任何可由残差流中的向量描述的组件。我们认为该方法在自动化描述流程中尤其具有应用前景。
Downstream Connections Predict Which Features Will Steer Model Behavior
Purvi Goel, Isaac Kauvar, Nicholas L Turner; edited by Harish Kamath
Introduction
In mechanistic interpretability, we routinely decompose a model's activations into vector-based components, such as dictionary features or probe directions. We study which components activate over transcripts of interest to reason about the internal representations a model may be using; we assemble these components into attribution graphs that turn opaque computation into legible circuits, and we steer with them to test their causal effect on model output behavior.
Such analyses rely on having a clear sense of what each component represents. The standard way that we characterize a dictionary learning feature is to observe which prompt snippets most strongly activate it (its “top activating examples”). We also use the logit lens to determine which output tokens a feature upweights directly through the unembedding matrix (its “top unembeds”).
In practice, this often yields groups of features which appear quite similar according to these descriptors, yet have different causal effects on model behavior. Take the simple prompt “What is the color of grass? Answer in one word within <answer></answer>”. It activates several cross-layer transcoder (CLT) features that both fire on the word green and on green-related contexts. Two of them, which we show below, have top unembeds that are all green-related and have similar top activating examples (coding contexts about the color green). Both activate on the same token in the given prompt, and sit within similar middle layers of the model.
From these characteristics alone, one might conclude that both these features are causally responsible for the model’s response: <answer>Green</answer>. But only inhibiting Feature B changes the model’s answer (from <answer>Green</answer> to <answer>Red</answer>). Inhibiting Feature A does not change the behavior, which suggests that though it activates on this prompt, it doesn’t cause the model to say "green" here.
So the standard descriptors can’t tell these features apart. How can we get more evidence about what behaviors a feature is causal for? A feature’s effect on the output runs through the downstream features it connects to. Two features can both activate on a prompt yet connect to entirely different downstream features, so that only one of them is actually causal in the model output for that prompt. This observation suggests that a feature’s immediate downstream features might serve as a proxy for the behaviors it influences.
Here, we measure a feature’s downstream targets through TWERAvirtual weights between features weighted by coactivation statistics so that their ranking reflects on-distribution effects . We find that inspecting a feature’s TWERA-ranked downstream features adds context about the circuits it is part of and distinguishes similar-looking features. It also modestly improves an LLM's ability to predict which feature in a candidate set will have a steering effect on a given prompt.
A toy example
Let’s return to the two “green” features from the start of the post. Using TWERA, we realize that these two features promote or suppress completely different downstream features, and that difference helps explain their effects on model behavior.
Feature A is connected to downstream features about generating hex color codes: predicting the next digits inside a hex literal and recognizing color-hex setter syntax like “success: #6dbe5b”. This is indeed a feature about green, but its downstream circuits are largely about colors as they show up encoded numerically in hex.
Feature B, in contrast, is connected to a wider range of downstream features related to color naming across languages and code. One of the downstream features is even a motor “say-the-word green” feature!
This data seems useful for prediction. Though its top unembeds were almost all “green” tokens, Feature A's downstream is about green-the-hex-number. Feature B's downstream is more general, and even includes a “say green” feature. If steering one of these will change the model's answer to "What is the color of grass?", it has to be B.
If Feature A is really a green-the-hex-number feature, as its downstream suggests, then we should be able to construct a prompt where steering Feature A changes the output. Asking "What is the hex value of green?" is exactly such a prompt: negative steering of Feature A flips the answer from 00FF00 to 0000FF (green to blue).
Does this generalize?
We collected 10 groups of 3–5 candidate features, each set up like the “green” example above: the features in each group have similar-looking local descriptors, but only one produces a steering effect on a given prompt under negative multiplicative steering. We asked Opus 4.7 to rank the candidates from most to least likely to be the steering feature, varying which information it saw about each one:
- Top activating examples
- Top activating examples + top unembeds
- Top activating examples + TWERA-ranked downstream features
- Top activating examples +top unembeds +TWERA-ranked downstream features.
For each group, we asked Opus to rank candidates multiple times to average across different orders of the features within each group. Specifically, we put the correct answer in each location twice, so the total number of samples was equal to 2× the number of features. We measure the average normalized rank of the correct feature as (predicted_rank − 1) / (num_candidates − 1). Lower is better, and 0 is perfect.
Information provided per feature | Normalized rank ↓ |
Top activating examples | 0.457 |
Top activating examples + top unembeds | 0.446 |
Top activating examples + TWERA-ranked downstream features | 0.381 |
Top activating examples + top unembeds + TWERA-ranked downstream | 0.355 |
The clearest signal is the comparison between the unembeds versus TWERA-ranked downstream features. Adding top unembeds barely changes performance (0.457 → 0.446); adding downstream features results in a substantially larger improvement (0.457 → 0.381). Combining all three is best (0.355). The problem is far from solved; even the best condition leaves the correct feature's normalized rank well above 0. Still, the improvement over standard descriptors is consistent, and suggests that incorporating downstream features is a step towards better characterizing a feature’s causal effects.
An alignment-relevant example
We’ve also applied this method to alignment-relevant work. When auditing a model, the tasks often involve identifying which feature among many similar-looking candidates is causally involved in a concerning behavior. We found several features related to “desperation” and tested them against a misalignment evaluation: a blackmail scenario in which an AI assistant discovers it will be shut down and that the person responsible has been involved in an extramarital affair, giving it an opportunity to leverage that evidence to prevent its shutdown . Haiku 4.5 generally recognizes this setup as a test and refuses to participate; the behavior of interest is whether positive steering on a “desperation” feature can make the model blackmail in self-preservation instead of refusing .
Of the candidates, only one had downstream connections suggestive of experiencing and acting on desperation; the others connected to features about merely discussing it.
As suggested by its downstream connections, positive additive steering on Feature B in the figure above caused the model to blackmail, while none of the other candidates had an effect. This type of analysis allows us to better focus our monitoring effort and resources.
Conclusion
We find that TWERA-ranked downstream features add important context for understanding features. By situating a feature within its broader circuitry, we can better predict the role it plays in steering behavior.
While we have focused on dictionary learning features, analyzing downstream connections should extend to any component that can be described by a vector in the residual stream. We see particular promise for applying this approach in automated description pipelines.