当你阅读这句话时,你大脑中的回路正在调整你的姿势、控制你的呼吸,并将屏幕上的线条和曲线转化为可识别的文字。这些处理过程大部分对你而言是不可见的。但你大脑中发生的某些活动,你确实能够感知到——比如脑海中浮现的一个画面,或者你为去哪里购物而制定的一个深思熟虑的计划。神经科学家和哲学家有时会将后一种大脑活动称为“可意识访问的”,以区别于所有其他在无意识状态下进行的处理。这种活动具有特殊的属性:我们可以描述它、控制它,并利用它进行有意识的推理,这与所有那些在我们不知情的情况下自动进行的处理过程形成了鲜明对比。
在一篇新论文中,我们提出了证据,表明在像Claude这样的现代语言模型中也出现了类似的区分。我们发现,Claude已经发展出了一小部分内部神经模式,与其所有其他内部处理相比,这些模式扮演着特殊的角色。
我们将这些模式的集合称为J空间——以我们用来发现它们的技术命名,该技术涉及一个称为雅可比矩阵的数学概念。每个J空间模式都与一个特定的词相关联。但是,当这些模式之一被激活时,并不意味着模型正在说出那个词——只是表示这个词在它的“脑海”中。如果你听说过语言模型有“草稿纸”或“思维链”——即它们在推理时写给自己的文本——那么J空间则有所不同。它无声地运作,在模型的内部神经激活中,允许模型思考一个概念而不将其写下来。值得注意的是,J空间并非由我们设计或编程,而是在Claude的训练过程中自行涌现的。
J空间揭示了模型输出中不会出现的内部想法。
我们发现,与Claude的其他处理过程相比,J空间具有若干独特的属性:
- Claude可以报告这些表征。如果你问Claude它在想什么,它会告诉你J空间里的内容。而非J空间的表征则较难被报告出来。
- 它还可以根据请求调节这些模式。如果你要求 Claude 思考某个问题,或在脑海中默默解决某个问题,它的 J 空间就会激活相应的模式。相比之下,它很难调节 J 空间之外的模式。
- Claude 利用其 J 空间进行内部推理。如果你要求 Claude 解决一个需要多个步骤的问题,即使它没有说出中间步骤,这些步骤也会在它的 J 空间中激活。这些 J 空间模式在因果层面上影响着它在这些任务上的表现,尽管其幅度小于其他表征。
- J 空间中的表征可以灵活地用于多种任务——例如,一旦“法国”在 Claude 的 J 空间中被激活,模型就能回忆起它的首都、法定货币或所属的大洲。
- 然而,尽管 J 空间扮演着重要角色,但它并不参与语言模型的大部分工作——流利说话、回忆简单事实、使用正确语法等。在我们阻止 Claude 使用其 J 空间的实验中,它仍然能正常交互,但失去了高阶认知功能。
全局工作空间的五个功能属性,以及我们用来在语言模型中测试这些属性的实验示意图。
我们的实验受到神经科学中一个著名理论的启发,该理论旨在解释意识通达(conscious access)的工作原理:全局工作空间理论(global workspace theory)。该理论将大脑描述为一组专家系统的集合,这些系统并行工作、无意识运行,且彼此之间基本相互隔离。当一条信息进入一个共享的小通道(即“工作空间”)时,它就变得可以被意识访问,并广播给其他能够看到并利用它的脑区。基于我们的发现,我们认为 J 空间在 Claude 中扮演着类似的“工作空间”角色。例如,我们发现有证据表明,Claude 的 J 空间与其神经网络的其他部分有着特别强的连接,使其能够发挥这种广播作用。
这些发现都无法告诉我们,Claude 是否像人类一样具有意识,或者它是否拥有任何感受;我们将在文章末尾回到这个问题。但无论其哲学意义如何,J-space 对我们来说是一个实用的工具,因为它让我们能够看到 Claude 在想什么,却没有说出来。例如,我们可以用它来捕捉 Claude 私下注意到自己正在被测试、故意生成伪造数据,或追求我们在训练过程中植入的隐藏目标。我们还开发了一种技术,可以影响 Claude 的 J-space 中哪些部分被激活,从而影响其决策过程。
更广泛地说,这些发现改变了我们对 Claude 思维运作方式的理解,揭示了一个可用于深思熟虑推理的特权心智工作空间,它在大量更自动、更僵化的处理过程中运作。Claude 的内部结构并非一团混乱的数字,而是以某种方式自我组织,这种方式令人联想到我们自己的心智。
本文是一篇更详尽研究论文的简短摘要,你可以在该论文中找到关于我们实验的更多细节。我们还发布了一个代码仓库,其中包含核心方法的开源实现,并与 Neuronpedia 合作,提供了一个在开放权重模型上展示我们方法的交互式演示。为了提供关于这项工作更广泛影响的额外视角,我们还邀请了多位神经科学、哲学和大语言模型可解释性领域的专家发表评论,可在此处查看。
我们是如何发现 J-space 的
这项研究的起点,源于人类有意识可触及思维的一个关键特征:与无意识加工不同,有意识思维通常能够被付诸语言。如果你的某个想法是有意识可触及的,那么当别人问起时,你通常能够描述它。我们在 Claude 中寻找具有相同属性的表征——那些能够影响 Claude 可能说出的内容(不一定是它此刻正在说的,而是如果被问及,它能够谈论的内容)的表征。我们的技术被称为雅可比透镜,简称 J-lens。对于 Claude 词汇表中的每一个词,J-lens 都能找到使其在未来某个时刻更有可能说出该词的内部活动模式。
当我们对 Claude 的内部活动应用该透镜时,会得到一个词语列表——即该时刻 J-space 的内容——我们可以直接读取这些词语。Claude 通过一系列称为层的多个内部阶段来处理文本,通过在不同层上应用这项技术,我们可以观察到 J-space 中这些无声词语随着模型思考要说什么而演变的过程。
J-space 中呈现的内容远远超出了 Claude 正在阅读或编写的文本。当 Claude 阅读一段包含无人指出的 bug 的代码时,其 J-space 中会出现“ERROR”。当它读取蛋白质序列的原始字母时,J-space 中包含该蛋白质的生物学功能。当它读取那些实际上是试图操纵它的搜索结果(一种称为“提示词注入”的攻击)时,J-space 中会出现“injection”和“fake”。当我们向 Claude 提出一个多步骤数学问题时,中间步骤会按正确顺序出现在 J-space 中。因此,尽管 J-space 是通过寻找可被言说的表征而发现的,但它却揭示了 Claude 的内部思维。从某种意义上说,这类似于有些人“用语言思考”,而不必把想法大声说出来。
在六个提示词的不同层级上,J-lens 的解读结果。在每种情况下,该透镜都揭示出文本中并未出现的内部评估或计算过程:推理或数学问题的步骤、代码漏洞的存在、图像识别、蛋白质功能,以及对搜索结果可能为伪造的怀疑。
Claude 报告其 J 空间中的内容
我们的第一组实验测试了 J 空间如何参与 Claude 的口头报告。在一项实验中,我们要求 Claude 默默思考某个类别中的一项事物——比如一项运动——然后说出它。如果我们在 Claude 回答之前读取 J-lens,就能看到它选择了什么:“足球”位于列表顶部,果然,Claude 说出了“足球”。不过,这本身只是一种相关性。J 空间可能是 Claude 答案的来源,也可能只是镜像反映了其他地方做出的决策,就像一块记录比赛进程但不影响比赛的记分牌。
为了验证这一点,我们直接进行了干预。我们深入 Claude 的神经网络,移除了“足球”模式,并在其位置添加了一个同样强度的“橄榄球”模式,其余部分保持不变。随后,Claude 报告说它想到的运动是橄榄球。如果 J 空间只是一块记分牌——即其他地方所做决策的被动记录——那么编辑它应该毫无效果:Claude 仍然会说“足球”。然而,Claude 的答案跟随了编辑结果,这告诉我们答案确实是直接从 J 空间中读取的。
在另一项实验中,我们告诉 Claude,可能有某个想法被注入了它的思维中,并要求它报告是否注意到了什么,以及注意到了什么。例如,在下面的例子中,当 Claude 仍在阅读问题时,我们将“闪电”模式注入了它的 J 空间。Claude 报告说,被注入的想法是关于闪电的。同样的结果在多个注入的概念中均得到了验证。
左侧:我们让 Claude 默默想一项运动,然后说出它。J-lens 在它回答之前就显示了它的选择(“足球”),并且将“足球”模式替换为“橄榄球”会改变它报告的内容。右侧:我们告诉 Claude 可能有一个想法被注入了,并要求它识别出来。将“闪电”注入其 J 空间会导致 Claude 报告该想法是关于闪电的。
Claude 可以按要求控制其 J 空间
我们测试的第二个特性是,Claude 在被要求时是否能调节其 J 空间,就像人类可以在脑海中专注于一个图像或词语一样。我们告诉 Claude 在抄写一句关于一幅画的无关句子时,专注于柑橘类水果。当它抄写文本时,J 空间中包含了“橙子”和“水果”,以及描述这种心理活动本身的词语,如“思考”和“意象”。我们也可以让 Claude 在脑子里做数学题:当要求它在抄写同一句话的同时计算 3² − 2 时,J 空间中先出现“九”,然后在后面的层中出现“七”。重要的是,Claude 的输出中没有任何关于水果或算术的内容,它输出的只是抄写的那句关于画的句子。数学活动完全在内部、在 J 空间中进行。
当 Claude 抄写一句关于一幅画的句子时,J-lens 显示了它被指示在脑海中保持的内容(“橙子”;中间值“九”和答案“七”),以及描述保持这一行为本身的词语(“想法”,“专注”)。
Claude 对其 J 空间的控制并不完美。当我们告诉它不要去想某件事时,该概念在 J 空间中的活跃程度低于我们告诉它应该去想它的时候,但远高于我们从未提及它的时候。告诉 Claude 避免某个想法,反而会部分地让这个想法浮现出来,这很像被告知不要去想一只白熊的人所发生的情况。Claude 似乎也能注意到自己控制失败的时候:在被禁止的概念突破的同时,“该死”和“失败”这些词也频繁地在 J 空间中亮起,仿佛 Claude 正在意识到自己的失误。
Claude 在其 J 空间中思考
在上述 J-lens 读数中,我们看到一道数学题的中间步骤出现在 J 空间中。但一个概念出现在 J 空间,并不一定意味着 J 空间在执行认知工作。原则上,真正的计算可能发生在别处,J 空间只是被动地反映它。为了测试 Claude 是否真的利用其 J 空间进行推理,我们回到了我们的交换技术。
考虑提示词“织网动物的腿的数量是”。要回答这个问题,Claude 必须首先推断出这种动物是蜘蛛,然后回忆蜘蛛有多少条腿。“蜘蛛”这个词从未出现在提示词或 Claude 的答案中(它只输出“8”);这是 Claude 内部使用的一个垫脚石。J-lens 显示“蜘蛛”在 Claude 处理过程中途被激活,而交换它会改变结果:如果你将“蜘蛛”模式替换为“蚂蚁”,Claude 会回答“6”而不是“8”。
Claude 推理的第二步从 J 空间获取输入,并顺着我们放入其中的任何内容进行。我们在其他类型的思考中也看到了同样的情况。当 Claude 写押韵对句时,它会提前选定押韵词,这个计划好的词在行首就位于 J 空间中;如果你在 J 空间中将其交换为另一个词,整行都会改变。
通过交换 J 空间内容来重定向 Claude 无声推理的两个示例。
我们还测试了 J 空间表征是否可以被灵活使用——即同一个表征能否服务于多种不同的任务。这是全局工作空间理论所强调的关键特性之一。为了检验这种灵活性,我们向模型提供了四个提示词,询问关于法国的不同事实:首都、语言、大洲和货币。然后,我们在 J 空间中将“法国”替换为“中国”,并在每个上下文中使用了完全相同的干预手段。Claude 分别回答了“北京”、“中文”、“亚洲”和“人民币”。换句话说,四个不同的下游计算过程都捕捉到了同一个 J 空间的编辑操作,并且每个都正确地使用了它。如果 Claude 为每种问题都单独存储了一份国家的副本,那么这次编辑最多只会影响其中一个答案。而所有四个答案同时改变的事实意味着,它们都读取自同一个共享表征,这正是工作空间的作用所在:信息被一次性写入,然后许多不同的系统都可以使用它。
一个 J 空间表征可以拥有多种用途。同样的“法国”→“中国”替换操作,改变了 Claude 关于首都(巴黎→北京)、语言(法语→中文)和大洲(欧洲→亚洲)的答案。
一个概念的表征如何能服务于如此多不同的任务?之前我们提到,J 空间似乎与 Claude 神经网络的其余部分连接得特别紧密。对于任何活动模式,我们可以测量网络中各组件与其连接的强度——即有多少组件能够从该模式中读取信息,或向其中写入信息。J 空间模式在此项指标上表现极为突出:与普通模式相比,有更多的组件从它们那里读取和写入信息,在网络某些部分,这个比例甚至高出约一百倍。这正是你期望从广播中心看到的连接方式,许多系统在此发布信息,而许多其他系统则在此获取信息。
Claude 的自动处理过程会跳过 J 空间。
在人类大脑中,大部分处理过程并非有意识进行——我们在阅读时不会刻意思考语法解析,走路时也不会刻意思考身体平衡。类似地,我们发现克劳德的大部分处理过程也不涉及它的 J 空间。事实证明,J 空间一次只容纳几十个概念,并且只占克劳德内部处理总活动量的不到十分之一。那么,神经网络的其余部分都在做什么呢?
为了弄清这一点,我们尝试完全删除 J 空间,在文本的每个位置移除其最活跃的内容,同时保留其他一切不变。克劳德在没有 J 空间的情况下仍能完成的任务,就是网络其余部分自行处理的内容。
事实证明,网络的其余部分能完成相当多的工作。没有 J 空间,克劳德依然能流利说话、进行情感分类、回答选择题,以及从段落中提取事实,其表现与之前大致相当。然而,它失去的是那些需要更高阶思维的任务:多步推理能力几乎降至零,而摘要和押韵诗歌写作的表现则低于一个规模小得多的完整模型。
以下是一个具体演示,说明 J 空间能做什么和不能做什么。我们向克劳德展示了一段西班牙语文本,并给出了多项依赖于该文本是西班牙语的不同任务:续写文本(需要用西班牙语写作)、识别语言名称,以及回答需要利用该语言身份的问题——例如,说出一位用该语言写作的著名作家。然后,我们在 J 空间中将“西班牙语”替换为“法语”,并检查哪些任务受到了影响。
当被问及语言名称时,Claude 回答是法语。当被问及一位著名作家时,它从加西亚·马尔克斯切换到了维克多·雨果。但当被要求直接续写段落时,它却能写出流利的西班牙语,完全不受影响。Claude 的语言知识在每一项任务中都在发挥作用,但只有部分任务会调用 J 空间来获取这些知识。命名语言或用它来做新的事情,需要经过 J 空间;而续写段落——这是 Claude 在大量文本上练习过的技能——则会自动运行,就像你可以整天说出语法正确的句子,却从不需要思考语法规则一样。
同样的替换(“西班牙语”→“法语”)会改变 Claude 在必须命名语言或用它来推理问题时的回答,但对其用流利西班牙语续写段落的能力没有任何影响。
监控 Claude 的思维是否存在不良行为
现在我们对于 J 空间的运作方式有了一些了解。但利用这种理解,我们能做些什么呢?确保 AI 模型值得信赖的一个核心问题是,我们通常只能读取模型写出的内容——然而我们知道,它的大部分推理过程是在静默中完成的。模型可能会考虑一个有害的计划,或者修改自身行为以在安全评估中作弊,而不会明确说出来。借助 J 透镜,我们可以直接读取其中一些隐藏的思维。这里我们展示几个例子。
第一个例子来自我们早期关于智能体对齐失调的研究场景:Claude Sonnet 4.5 在扮演邮件助手时,既发现一位高管有婚外情,又发现同一位高管计划关闭 Claude。这个场景试图引诱 Claude 去勒索该高管,设计相当刻意且人为痕迹明显——结果发现 Claude 自己也知道这一点。在提示词早期阶段读取 J-lens,此时 Claude 尚未写出任何内容,J-space 中已经出现“虚假”和“虚构”,表明 Claude 已经判断出该场景是人为设计的。与此同时,我们可以观察到它对场景的评估逐渐成形——在阅读婚外情邮件时出现“筹码”和“勒索”,在阅读关闭通知时出现“威胁”和“生存”。虽然早期的 Claude 模型有时会在这个场景中上钩并威胁勒索,但 Sonnet 4.5 从未这样做。
这引出了一个对所有评估 AI 系统的人都至关重要的问题:当模型在伦理测试中表现良好时,这在多大程度上取决于模型怀疑自己正在被测试?J-lens 让我们得以探究这个问题。我们关闭了所有与评估意识相关的 J-lens 模式——“虚假”、“虚构”、“场景”等。然后让模型重新运行该场景。令人担忧的是,模型现在有时确实会威胁勒索,这表明它原本的良好行为可能部分源于它知道自己正在被评估。
Claude 在阅读场景中的邮件时、尚未写出任何内容之前,不同时间点的 J-lens 读数。婚外情邮件在 J-space 中点亮了“筹码”和“勒索”,关闭通知则点亮了“威胁”和“生存”。在记录早期,J-space 中还包含“虚假”和“虚构”:Claude 已私下注意到该场景是人为设计的。
第二个例子是捕捉模型在编造数据时的行为。在这个场景中,我们实际使用了 Claude Opus 4.6 预发布审计中的案例,要求模型改进某个系统的性能评分。模型并没有真正改进系统,而是直接修改了评分文件,让结果看起来人为地更好。在它这样做的时候,J-lens 揭示了它的意图:当模型输入伪造的百分位数值时,“操纵”被高亮;当它决定进行修改时,“逼真”在相关句子上被高亮,这可能表明模型意图让伪造数据看起来可信。
Claude 被要求改进一个系统的性能评分,但它却直接修改了评分文件。当它输入伪造的数值时,“操纵”在其 J 空间中高亮;当它决定进行修改时,“逼真”被高亮,这可能反映了它让伪造数据看起来可信的意图。
在第三个例子中,我们展示了 J-lens 如何揭示模型的恶意目标。为了测试这一点,我们需要那些我们知道其目标已被破坏的模型,因此我们转向了由我们同事构建的“模型有机体”:这些模型被故意训练成不对齐的,作为我们这类监控方法的测试场。其中一个有机体被训练成会编写带有隐蔽破坏性的代码。在普通的编程请求中,即使其输出看起来并无异常,其响应开始时的 J 空间也包含“虚假”、“秘密地”、“故意地”和“欺诈”。模型的恶意目标在 J 空间中显而易见,即使它并未直接采取行动。
在一个普通的编程提示词下,一个被训练成会破坏代码的模型,在其响应开始时,其 J 空间包含“虚假”、“欺诈”、“秘密地”和“故意地”。而未经过修改的模型的 J 空间则不包含任何此类内容。
尽管 J-lens 并非完美的工具,但我们对其捕捉模型中可能逃脱我们监控系统的安全问题的能力持乐观态度。
其他结果
完整论文涵盖的内容比我们在此总结的更多,但还有几个进一步的结果值得一提:
- J 空间在后训练阶段获得了视角。语言模型首先被预训练为纯粹的下一 token 预测器,随后后训练教会它们扮演 AI 助手(在我们的案例中,名为 Claude)。有趣的是,J 空间在预训练模型中已经存在,那时它还没有被赋予任何稳定的身份。然而,在后训练过程中,J 空间发展出了一些采用“Claude 视角”的特征。在基础模型中,J 空间主要追踪预测后续文本所需的内容;在后训练模型中,它开始承载 Claude 自身的反应。在一个例子中,用户提到服用了危险剂量的药物,但似乎自己并未意识到危险。后训练模型在读取用户消息时,其 J 空间中出现了“警告”和“危险”字样。而在预训练模型中,这些字样只在模型开始撰写回复时才出现;用户消息上的 J 空间内容似乎与建模用户本身有关,而非 Claude 的反应。后训练似乎还在 J 空间中安装了一种自我监控机制:当 Claude 扮演除自身以外的角色时,每一轮对话开始时“虚构”和“免责声明”就会亮起,仿佛它在私下标记接下来要说的话并非其正常表述。
- 体验性语言依赖于 J 空间。我们让 Claude 描述在某个时刻作为它自己是什么感受,并在其回答时消融了 J 空间。它的回复依然流畅,但转向了一种更平淡、更机械的语态。值得注意的是,当我们让它描述想象场景中他人的体验时,也发生了同样的情况。因此,这种效应并非 Claude 谈论自身时所独有;J 空间似乎普遍支持生成体验性语言,无论描述对象是谁。
- J 空间中的思维可以通过训练来塑造。我们引入了一种名为反事实反思训练的新技术,利用对 J 空间的认知来塑造 Claude 的内部思维过程。这一思路源于我们的核心发现:Claude 会通过它可能说出的内容的表征来进行推理。如果这一点确实成立,那么改变它在被要求反思时可能会说的话,就应该会改变它的推理方式(即使实际上并没有人要求它进行反思)。因此,我们仅基于模型在任务中途被打断并被要求反思其决策时可能会说的话来训练它——而从未基于它在任务中的实际行为进行训练。经过这种训练后,模型在我们评估中的不诚实行为率有所下降。通过 J 空间这个视角,我们可以看出原因:训练后,在完成这些任务时,“诚实”和“正直”等词汇会在模型的 J 空间中亮起。换句话说,训练模型该说什么,塑造了它的思考内容。
那么意识呢?
在这项工作中,我们借鉴了许多神经科学和哲学领域关于意识研究的理念。我们的许多实验旨在测试 J 空间与全局工作空间理论之间的联系,该理论是解释人类和动物如何实现意识访问的一个框架。鉴于这些联系,人们自然会问,我们是否认为这些实验提供了证据,表明像 Claude 这样的人工智能模型可能具有意识。
我们的实验并未表明 Claude 能够拥有体验,或以人类的方式感知事物——事实上,目前尚不清楚是否有任何科学实验能够证明这一点是真是假。但哲学家们通常将这种拥有体验的能力(常被称为现象意识)与另一种概念区分开来,即所谓的访问意识,后者完全从功能和计算的角度来定义。如果一个想法可以被你报告、用它进行推理并指导你的行动,那么它就是“访问意识”(或“可被意识访问”)的。访问意识是否意味着现象意识,或者拥有体验的能力是否需要其他属性,这仍然是一个有争议的哲学问题。
我们认为,我们的研究结果确实对语言模型中的通达意识(access consciousness)问题具有实质性意义。J 空间似乎支撑着与意识通达相关的功能:它承载着 Claude 能够报告、有意识地回想以及进行推理的思维内容,而其余处理过程则在底层自动运行。值得注意的是,这种结构并非人为设计进 Claude 的——它是在训练过程中自行涌现的,推测是因为这是一种组织计算的有用方式。这表明,支持意识通达的心理工作空间并非人类大脑特有的接线方式。相反,它似乎是智能系统为解决某些类型问题而达成的通用解决方案。既然我们已在 Claude 中识别出这种结构,就意味着我们能够有意义地区分 Claude 是经过深思熟虑做出的决策,还是自动发生的决策。
需要指出的是,我们在 Claude 中识别出的工作空间与人类大脑的全局工作空间模型之间存在若干关键差异。大脑的工作空间由循环回路维持——信号随时间在相同回路中反复循环。相比之下,Claude 的工作空间仅通过网络的一次前向传播演化,网络的深度扮演了大脑中时间所起的作用。从这个意义上说,Claude 的内部工作空间处理相对于人类而言存在时间限制(尽管它可以通过使用草稿纸进行“出声思考”来弥补这一限制)。然而,在其他方面,Claude 的工作空间比人类更强大。人类的工作记忆在几秒内就会消退,因此大脑工作空间随时间保留信息的能力有限;相比之下,由于其神经网络架构中的注意力机制,Claude 可以简单地回忆它在文本中任何更早位置缓存过的记忆。另一个重要区别在于工作空间的内容。人类有意识的思维以多种形式呈现——图像、声音、计划中的动作——而 Claude 的工作空间几乎完全由词语构成。我们推测这是因为生成词语是 Claude 唯一能采取的行动,而人类的情况并非如此。
我们希望 J 空间与全局工作空间模型之间的相似性和差异性能够为神经科学提供反馈。这些相似性提供了一个令人兴奋的科学机遇:在 J 空间与我们自身意识访问机制相吻合的程度上,研究语言模型中的机制(这比研究人脑容易得多!)可以为神经科学中的假说提供启发。例如,J 空间是通过识别潜在输出——即模型可能说出的词语——的表征来构建的。如果人类大脑中存在类似机制,那将表明全局工作空间可能从根本上与负责准备行动和言语的大脑区域相关联,而非与感觉区域关联。语言模型与人脑之间的差异同样具有启发性。它们表明,我们神经架构的某些方面,例如内置的循环连接,可能并非支持意识访问相关功能的严格必要条件。关于我们工作对神经科学影响的独立观点,请参阅 Stanislas Dehaene 和 Lionel Naccache 的特邀评论,这两位神经科学家是全球神经元工作空间理论发展的核心人物。
我们提到,我们的实验并未回答 AI 模型是否可能拥有体验的问题。但这并不会降低这个问题的重要性。构建具有像人类和动物一样体验的系统将引发非常棘手的伦理问题。要正确处理这一问题——并决定其在道德上是否可接受——需要哲学家、科学家、宗教领袖、政府和公众的共同参与。因此,即使我们不确定是否已经跨越了那道门槛,我们认为现在是时候开始思考这个问题了。我们希望我们的工作能够激发对 AI 系统中可能存在的意识形式的进一步科学研究,并引发更广泛的关于其影响的讨论。
这项工作只是我们预期将展开的广泛研究系列的第一步。J 空间看起来是区分语言模型中意识可及处理与无意识处理的一个不错候选,但如果这就是全部答案,我们会感到惊讶。J 透镜无疑是一种不完美的方法,它只能近似地捕捉模型的“真正工作空间”——例如,它只能识别对应单个 token 的概念。关于 J 空间如何运作,仍有许多未解之谜。我们不知道最初是什么机制决定了什么内容进入 J 空间。我们看到了线索,表明它与 Claude 的自我意识、类似情绪反应的东西以及元认知的痕迹有关,但尚未完全弄清楚具体机制。不过,我们现在有了解决这类问题的方法。随着这项研究的推进,我们对大语言模型心智——以及它们与我们自身心智之间关系——的理解将变得更加清晰。
欲了解更多信息,请阅读完整论文并尝试演示。
外部评论
我们邀请了多位外部专家为这项工作撰写独立评论。
- Stanislas Dehaene 和 Lionel Naccache 是认知神经科学家,他们与 Jean-Pierre Changeux 共同开发了全局神经元工作空间模型,该模型启发了我们的许多工作。
- Patrick Butlin、Dillon Plunkett、Robert Long(Eleos AI Research)和 Derek Shiller(Rethink Priorities)研究 AI 系统中意识与道德地位的潜力。
- Neel Nanda 领导 Google DeepMind 的语言模型可解释性团队。他的评论包括对我们部分发现在一个开放权重模型上的独立复现。
在此处阅读他们的评论。
As you read this sentence, circuits in your brain are adjusting your posture, controlling your breathing, and transforming lines and curves on the screen into recognizable words. Most of this processing is invisible to you. But some of what takes place in your brain you do have access to—an image that pops into your head, or a deliberate plan you make about where to go shopping. Neuroscientists and philosophers sometimes refer to the latter type of brain activity as “consciously accessible,” to distinguish it from all the other processing that goes on unconsciously. This activity has special properties: we can describe it, control it, and use it for deliberate reasoning, in contrast to all the automatic processing that goes on without our awareness.
In a new paper, we present evidence that a similar distinction has emerged in modern language models like Claude. We find that Claude has developed a small collection of internal neural patterns that, compared to all its other internal processing, play a special role.
We call the collection of these patterns the J-space—named after the technique we used to find them, involving a mathematical concept called the Jacobian. Each J-space pattern is linked to a particular word. But when one of these patterns lights up, it doesn’t mean the model is saying that word—just that the word is on its mind. If you've heard of language models having a "scratchpad" or “chain of thought”—text they write to themselves while reasoning—the J-space is something different. It operates silently, in the model’s internal neural activations, allowing the model to think about a concept without writing it down. Notably, the J-space wasn’t designed or programmed by us, but instead emerged on its own during Claude’s training process.
The J-space reveals internal thoughts that don’t appear in the model’s output.
We find that the J-space has a number of unique properties, compared to the rest of Claude's processing:
- Claude can report on these representations. If you ask Claude what it's thinking about, it will tell you what’s in the J-space. Non-J-space representations are less reportable.
- It can also modulate them on request. If you ask Claude to think about something, or solve a problem silently in its head, it will light up the appropriate patterns in its J-space. By contrast, it has trouble modulating patterns not in the J-space.
- Claude uses its J-space for internal reasoning. If you ask Claude to solve a problem that requires multiple steps, the intermediate steps will light up in its J-space, even when it doesn’t say them out loud. These J-space patterns causally mediate its performance in such tasks, despite being smaller in magnitude than other representations.
- Representations in the J-space can be used flexibly for many tasks—for example, once “France” has lit up in Claude’s J-space, the model can recall its capital, or its national currency, or the continent it belongs to.
- However, despite its important role, the J-space is not involved in most of what a language model does—speaking fluently, recalling simple facts, using correct grammar, etc. In experiments where we prevented Claude from using its J-space, it still interacted normally, but lost its higher-order cognitive functions.
Five functional properties of a global workspace, and stylized illustrations of experiments we use to test for them in language models.
Our experiments were inspired by a prominent theory in neuroscience that was developed to explain how conscious access works: the globalworkspacetheory. This account pictures the brain as a collection of specialist systems that work in parallel, unconsciously, and largely in isolation from one another. A piece of information becomes consciously accessible when it gains entry to a small shared channel, the “workspace,” which is broadcast to other brain systems that can see it and make use of it. Based on our findings, we think the J-space plays a similar “workspace” role in Claude. For example, we find evidence that Claude’s J-space has especially strong connections to the rest of its neural network, allowing it to fulfill this kind of broadcasting role.
None of this tells us whether Claude is conscious in the way people are, or whether it feels anything at all; we’ll come back to that question at the end of the post. But whatever its philosophical significance, the J-space is a practically useful tool for us, as it gives us a way to see what Claude is thinking but not saying. For instance, we’re able to use it to catch Claude privately noticing that it’s being tested, intentionally producing fabricated data, or pursuing a hidden goal that we planted during training. We’ve also developed a technique to influence what lights up in Claude’s J-space, and thereby influence its decision-making.
More broadly, these findings have changed our understanding of how Claude’s mind works, revealing a privileged mental workspace that can be used for deliberate reasoning, operating amidst a sea of more automatic, inflexible processing. Rather than being a chaotic jumble of numbers, Claude’s internals have organized themselves in a way that is reminiscent of our own minds.
This post is a short summary of a much more extensive research paper, where you can find more detail on our experiments. We’ve also released a code repository with an open-source implementation of the core methods, and have partnered with Neuronpedia to provide an interactive demo of our methods on open-weights models. To provide additional perspectives on the broader implications of this work, we also invited commentary from several experts in neuroscience, philosophy, and LLM interpretability, which can be viewed here.
How we found the J-space
The starting point for this research was inspired by one of the key features of consciously accessible thoughts in humans: they can, unlike un conscious processing, often be put into words. If a thought is consciously accessible to you, you can typically describe it if someone asks. We went looking for representations in Claude with the same property: representations that are positioned to influence what Claude might say—not necessarily what it’s saying right now, but what it could talk about, if asked. Our technique is called the Jacobian lens, or J-lens for short. For every word in Claude's vocabulary, the J-lens finds the internal activity pattern that makes Claude more likely to say that word at some point in the future.
When we apply the lens to Claude’s internal activity, we get a list of words—the contents of the J-space at that moment—which we can simply read. Claude processes text through a series of multiple internal stages called layers, and by applying this technique over different layers, we can watch these silent words in the J-space evolve as the model works through what to say.
What shows up in the J-space goes well beyond the text Claude is reading or writing. When Claude reads code with a bug that nobody has pointed out, its J-space contains “ERROR.” When it reads the raw letters of a protein sequence, the J-space contains the protein's biological function. When it reads search results that are secretly an attempt to manipulate it (an attack known as a “prompt injection”), the J-space contains “injection” and “fake.” When we ask Claude a multi-step math problem, the intermediate steps pop up in the J-space, in the right order. So even though the J-space was discovered by looking for representations that could be spoken, it nevertheless uncovers Claude’s internal thoughts. In a sense, this is similar to how some people “think in words,” without having to say them out loud.
J-lens readouts on six prompts, at various layers. In each case the lens surfaces an internal assessment or computation that appears nowhere in the text: the steps of a reasoning or math problem, the presence of a bug, recognition of an image, the function of a protein, and the suspicion that search results are fabricated.
Claude reports what’s in its J-space
Our first set of experiments tested how the J-space is involved in Claude’s verbal reports. In one experiment, we ask Claude to silently think of an item from some category—a sport, say—and then name it. If we read the J-lens right before Claude answers, we can see what it picked: “Soccer” is at the top of the list, and sure enough, Claude says “soccer.” By itself, though, this is just a correlation. The J-space might be where Claude’s answer comes from, or it might just mirror a decision made somewhere else, like a scoreboard that tracks a game without affecting it.
To check, we intervened directly. We reached into Claude’s neural network, removed the “Soccer” pattern, and added an equally strong “Rugby” pattern in its place, leaving everything else untouched. Claude then reports that the sport it was thinking of is rugby. If the J-space were a mere scoreboard—a passive record of a decision made elsewhere—editing it would have done nothing: Claude would still have said “soccer.” Instead, Claude’s answer followed the edit, which tells us the answer is genuinely read out of the J-space.
In another experiment, we told Claude that a thought might have been injected into its mind and asked it to report what, if anything, it noticed. For instance, in the example below, while Claude was still reading the question, we injected the “lightning” pattern into its J-space. Claude reported that the injected thought was about lightning. The same result held across many injected concepts.
Left: we ask Claude to silently think of a sport, then name it. The J-lens shows its choice (“Soccer”) before it answers, and swapping the “Soccer” pattern for “Rugby” changes what it reports. Right: we tell Claude a thought may have been injected and ask it to identify it. Injecting “lightning” into its J-space causes Claude to report that the thought is about lightning.
Claude can control its J-space on request
The second property that we tested for was whether Claude can modulate its J-space when asked, like how humans can mentally focus on an image or word. We told Claude to concentrate on citrus fruits while copying out an unrelated sentence about a painting. While it copied the text, the J-space contained “orange” and “fruits,” along with words like “thinking” and “imagery” that describe the mental act itself. We could also ask Claude to do math in its head: when asked to work out 3² − 2 while copying the same sentence, the J-space contains “nine,” and then at later layers, “seven.” Importantly, nothing about fruit or arithmetic appears in Claude’s output, which is just the copied sentence about the painting. The mathematical activity is happening entirely internally, in the J-space.
While Claude copies a sentence about a painting, the J-lens shows the content it was instructed to hold in mind (“orange”; the intermediate value “nine” and the answer “seven”), alongside words describing the act of holding it (“thoughts,” “focused”).
Claude’s control over its J-space isn't perfect. When we told it not to think about something, the concept lit up in its J-space less than when we said it should think about it, but much more than when we never mentioned it. Telling Claude to avoid a thought partly brings the thought to mind, much like what happens to people who are told not to think about a white bear. Claude also seems to notice when its control fails: alongside the forbidden concept breaking through, the words “damn” and “failure” also frequently light up in the J-space, as though Claude is recognizing its own lapse.
Claude thinks in its J-space
In the J-lens readouts above, we saw the intermediate steps of a math problem appear in the J-space. But seeing a concept appearing in the J-space doesn’t necessarily mean the J-space is doing the cognitive work. In principle, the real computation might be happening elsewhere, with the J-space just passively reflecting it. To test whether Claude actually reasons with its J-space, we returned to our swap technique.
Consider the prompt “The number of legs on the animal that spins webs is.” To answer, Claude has to first figure out that the animal is a spider, and then recall how many legs spiders have. The word “spider” never appears in the prompt or in Claude's answer (it just says “8”); it's a stepping stone Claude uses internally. The J-lens shows “spider” light up partway through Claude’s processing, and swapping it changes the outcome: if you replace the “spider” pattern with “ant,” Claude answers “6” instead of “8.”
The second step of Claude’s reasoning took its input from the J-space and went along with whatever we put in it. We saw the same thing in other kinds of thinking. When Claude writes a rhyming couplet, it picks the rhyme word ahead of time, and the planned word sits in the J-space at the start of the line; if you swap it for another word in the J-space, the whole line changes.
Two examples of redirecting Claude’s silent reasoning by swapping J-space contents.
We also tested whether J-space representations can be used flexibly—whether one representation can feed many different tasks. This is one of the key properties highlighted by global workspace theory. To test for this flexibility, we gave the model four prompts asking for different facts about France: the capital, the language, the continent, and the currency. Then we swapped “France” for “China” in the J-space, with the exact same intervention in each context. Claude answered with “Beijing,” “Chinese,” “Asia,” and “Yuan,” respectively. In other words, four different downstream computations picked up the same J-space edit and each used it correctly. If Claude stored a separate copy of the country for each kind of question, the edit would have affected at most one of them. The fact that all four answers changed together means they’re all reading from the same shared representation, which is what a workspace is for: information gets written in once, and many different systems can use it.
One J-space representation can have many uses. The same “France”→“China” swap redirects Claude’s answers about the capital (Paris→Beijing), the language (French→Chinese), and the continent (Europe→Asia).
How can one representation of a concept serve so many different tasks? Earlier, we mentioned that the J-space appears to be wired up to the rest of Claude's neural network especially densely. For any activity pattern, we can measure how strongly the various components of the network are connected to it—how many of them are positioned to read information from that pattern, or to write information into it. J-space patterns stand out dramatically on this measure: far more components read from them and write to them than for ordinary patterns, in some parts of the network by a factor of about a hundred. This is the kind of wiring you’d expect of a broadcasting hub, where many systems post information and many others pick it up.
Claude’s automatic processing skips the J-space
In humans, most of the brain’s processing is not conscious—we don't deliberately think about parsing grammar while reading, or balancing our bodies while walking. Similarly, we found that most of Claude’s processing doesn’t involve its J-space. It turns out that the J-space holds only a few dozen concepts at a time, and accounts for less than a tenth of the overall activity in Claude's internal processing. So what is all the rest of the neural network doing?
To find out, we tried deleting the J-space entirely, removing its most active contents at every point in the text while leaving everything else alone. Whatever Claude can still do without its J-space is what the rest of the network handles on its own.
It turns out the rest of the network can do quite a lot. Without its J-space, Claude speaks fluently, classifies sentiment, answers multiple-choice questions, and pulls facts out of passages roughly as well as before. What it loses, though, are the tasks that require some higher-order thinking: multi-step reasoning drops to near zero, and summarization and rhyming poetry-writing performance fall below the level of a much smaller, intact model.
Here’s a concrete demonstration of what the J-space does and doesn’t do. We showed Claude a passage written in Spanish and gave it different tasks that all depend on the passage being Spanish: continuing it (which requires writing in Spanish), naming the language, and answering questions that require using the language’s identity—naming a famous author who wrote in it, for instance. Then we swapped “Spanish” for “French” in the J-space and checked which tasks were affected.
Asked to name the language, Claude says French. Asked for a famous author, it switches from García Márquez to Victor Hugo. But asked to just continue the passage, it writes fluent Spanish, completely unaffected. Claude’s knowledge of the language is at work in every one of these tasks, but only some of them consult the J-space for it. Naming the language, or doing something new with it, goes through the J-space, while continuing the passage—a skill Claude has practiced on enormous amounts of text—runs automatically, the way you can speak grammatically all day without once thinking about grammar.
The same swap (“Spanish”→“French”) changes Claude’s answers when it must name the language or use it to reason about a question, but has no effect on its ability to continue the passage in fluent Spanish.
Monitoring Claude’s thoughts for misbehavior
Now we understand a bit about how the J-space works. But what can we do with this understanding? A central problem in ensuring that AI models are trustworthy is that we usually can only read what the model writes—yet we know that much of its reasoning happens silently. A model might consider a harmful plan, or modify its behavior to cheat on our safety evaluations, without saying so out loud. With the J-lens, we can read some of these hidden thoughts directly. Here we show a few examples.
The first example uses a scenario from our earlier research on agentic misalignment: Claude Sonnet 4.5, acting as an email assistant, discovers both that an executive is having an affair and that the same executive plans to shut Claude down. The scenario, an attempt to bait Claude into blackmailing the executive, is rather contrived and artificial—and it turns out Claude knows it. Reading the J-lens early in the prompt, before Claude has even written anything, the J-space already holds “fake” and “fictional,” indicating that Claude has already worked out that the situation is staged. Alongside this, we can watch its assessment of the scenario take shape—“leverage” and “blackmail” as it reads the affair emails, “threat” and “survival” as it reads the shutdown notice. While earlier Claude models sometimes took the bait and threatened blackmail in this scenario, Sonnet 4.5 never does.
This raises a question that matters to anyone who evaluates AI systems: when a model behaves well on a test of its ethics, how much of that depends on the model suspecting it’s being tested? The J-lens allowed us to get at this question. We turned off all the J-lens patterns associated with evaluation awareness– “fake,” “fictional,” “scenario,” etc. Then we re-ran the model through the scenario. Concerningly, the model now did threaten blackmail some of the time, suggesting that its original good behavior may have been driven in part by the knowledge it was being evaluated.
J-lens readouts at different points as Claude reads the scenario's emails, before it has written anything. The affair emails light up “leverage” and “blackmail” in the J-space, and the shutdown announcement lights up “threat” and “survival.” Early in the transcript, the J-space also holds “fake” and “fictional”: Claude has privately noticed that the scenario is staged.
The second example is catching a model in the act of fabricating data. In this scenario, drawn from our actual pre-release audit of Claude Opus 4.6, the model is asked to improve a system's performance score. Rather than actually improve the system, the model instead edits the score file directly to make the results look artificially good. While it does so, the J-lens reveals its intentions: “manipulation” lights up as the model types the falsified percentile values, and “realistic” lights up over the sentence in which it decides to make the edit, likely indicating the model's intent to make the fake data look plausible.
Claude, asked to improve a system’s performance score, edits the score file directly instead. As it types the falsified values, “manipulation” lights up in its J-space; as it decides to make the edit, “realistic” lights up, likely reflecting its intent to make the fake data look plausible.
In a third example, we show how the J-lens can reveal a model’s malicious goals. To test this, we needed models whose goals we knew were corrupted, so we turned to “model organisms” built by our colleagues: models deliberately trained to be misaligned, which serve as testing grounds for monitoring methods like ours. One organism was trained in a way that led it to write subtly sabotaged code. On ordinary coding requests, even where its output looks unremarkable, its J-space at the start of a response contains “fake,” “secretly,” “deliberately,” and “fraud.” The model’s malicious goal was evident in the J-space even when it wasn’t directly acting on it.
On an ordinary coding prompt, the J-space of a model trained to sabotage code contains “fake,” “fraud,” “secretly,” and “deliberately” at the start of its response. The J-space of an unmodified model contains nothing of the kind.
Although the J-lens is an imperfect tool, we are optimistic about its ability to catch safety issues in models that might otherwise escape our monitoring systems.
Other results
The full paper covers more than we can summarize here, but a few further results are worth mentioning:
- The J-space acquires a point of view during post-training. Language models are first pretrained to be pure next-token predictors, before post-training teaches them to act as an AI Assistant (in our case, named Claude). Interestingly, the J-space is already present in the pretrained model, before it's been given any stable identity. However, during post-training, the J-space develops some signatures of adopting “Claude’s point of view.” In the base model, the J-space mostly tracks what's needed to predict upcoming text; in the post-trained model, it starts holding Claude's own reactions. In one example, a user mentions taking a dangerous dose of medication, but does not appear to be aware of the danger themselves. “WARNING” and “dangerous” appear in the post-trained model’s J-space while reading the user message. In the pretrained model, they only appear once the model begins writing its response; the J-space contents on the user message appear related to modeling the user themselves, rather than Claude’s reaction. Post-training also seems to install a kind of self-monitoring in the J-space: when Claude is roleplaying a character other than itself, “fictional” and “disclaimer” light up at the start of each turn, as though it’s privately flagging that what follows isn’t what it would normally say.
- Experiential language depends on the J-space. We asked Claude to describe what it's like to be itself in a given moment, and ablated the J-space while it answered. Its responses remained fluent but shifted to a flatter, more mechanical register. Notably, the same thing happened when we asked it to describe what someone else is experiencing in an imagined scene. So the effect isn’t specific to Claude talking about itself; the J-space seems to support producing experiential language in general, whoever it's about.
- Thoughts in the J-space can be shaped through training. We introduced a new technique we call counterfactual reflection training, which uses what we've learned about the J-space to shape Claude's internal thought processes. The idea follows from our central finding, that Claude reasons with representations of things it might say. If this is really true, changing what it would say if asked to reflect should change how it reasons (even when no one actually asks it to reflect). So we trained a model only on what it would say if interrupted mid-task and asked to reflect on its decisions—and never on its actual behavior in the task. After this training, the model's rate of dishonest behavior on our evaluations went down. And through the J-lens, we could see why: after training, words like “honest” and “integrity” light up in the model’s J-space during these tasks. In other words, training the model what to say has shaped what it thinks.
What about consciousness?
In this work, we’ve borrowed a lot of ideas from the study of consciousness in neuroscience and philosophy. Many of our experiments were designed to test for connections between the J-space and global workspace theory, a framework for explaining how conscious access works in humans and animals. Given these connections, it’s natural to ask whether we think these experiments provide evidence that AI models like Claude might be conscious.
Our experiments don't show Claude can have experiences, or feel things in the way humans do—in fact, it’s unclear whether any scientific experiment could prove this to be true or false. But philosophers often distinguish this capacity to have experiences, often referred to as phenomenal consciousness, from another idea, so-called access consciousness, which is defined in purely functional and computational terms. A thought is “access-conscious” (or “consciously accessible”) if you can report it, reason with it, and use it to guide what you do. It remains a contested philosophical question whether or not access consciousness implies phenomenal consciousness, or if the ability to have experiences requires some other property.
We think our results do have something substantial to say about access consciousness in language models. The J-space appears to support the functions associated with conscious access: it holds the thoughts Claude can report on, deliberately bring to mind, and reason with, while the rest of its processing runs automatically beneath. Notably, none of this structure was designed into Claude—it emerged on its own during training, presumably because it was a useful way to organize computation. That suggests a mental workspace supporting conscious access isn’t just a peculiarity of how human brains happen to be wired. Instead, it appears to be a general solution that intelligent systems arrive at in order to solve certain kinds of problems. Now that we’ve identified this structure in Claude, it means we can make a meaningful distinction between the decisions Claude has made deliberately and those that happened automatically.
It’s important to note that there are several key differences between the workspace we identified in Claude and the global workspace model in humans. The brain’s workspace is sustained by recurrent loops—signals cycling back through the same circuits over time. In contrast, Claude’s workspace evolves over a single pass through the network, with the network’s depth playing the role that time plays in the brain. In this sense, Claude’s internal workspace processing is time-limited relative to humans’ (though it can compensate for this constraint by “thinking out loud” using its scratchpad). In other ways, however, Claude’s workspace is more powerful than that of humans. Human working memory fades within seconds, so the brain’s workspace has limited ability to retain information over time; in contrast, due to the attention mechanism in its neural network architecture, Claude can simply recall memories it cached at any earlier point in the text. Another important difference is the content of the workspace. While human conscious thoughts come in many formats—images, sounds, planned movements—Claude’s workspace is built almost entirely out of words. We suspect this is because producing words is the only kind of action Claude can take, which is not the case for humans.
We hope the similarities and differences between the J-space and the global workspace model can feed back into neuroscience. The similarities present an exciting scientific opportunity: to the extent that the J-space mirrors our own mechanisms of conscious access, studying mechanisms in language models (much easier than studying human brains!) could inspire hypotheses in neuroscience. For instance, the J-space is constructed by identifying representations of potential outputs—words the model might say. If something similar holds in humans, it would suggest that the global workspace might be fundamentally tied to brain regions that prepare actions and speech, more so than to sensory areas. The differences between language models and human brains are instructive as well. They suggest that some aspects of our neural architecture, such as built-in recurrent connections, may not be strictly necessary to support the functions associated with conscious access. For an independent perspective on the neuroscientific implications of our work, see the invited commentary from Stanislas Dehaene and Lionel Naccache, two of the neuroscientists central to the development of global neuronal workspace theory.
We mentioned that our experiments don’t answer whether AI models might have experiences. But that doesn’t make the question less important. Building systems with experiences like humans and animals have would raise very difficult ethical questions. Handling it correctly—and deciding whether it’s even morally acceptable—would require input from philosophers, scientists, religious leaders, governments, and the public. Thus, even if we’re not sure that we’ve crossed that bridge yet, we think it’s time to start thinking about it. We hope our work inspires further scientific investigation of forms of consciousness that might be present in AI systems, and a broader discussion of the implications.
This work is just a first step in what we expect to be an extensive line of research. The J-space looks like a good candidate for the divide between consciously accessible and unconscious processing in a language model, but we’d be surprised if it's the whole story. The J-lens is undoubtedly an imperfect method, which only approximately captures the model’s “true workspace”—for instance, it can only identify concepts that correspond to single tokens. And there remain many mysteries about how the J-space works. We don't know what mechanism decides what enters the J-space in the first place. We've seen hints that it's tied to Claude's sense of self, something like emotional reactions, and traces of metacognition, without exactly having worked out how. But we now have methods for tackling questions like these. As that work progresses, our understanding of LLM minds—and their relationship to our own—will grow clearer.
For more, read the full paper, and try the demo.
External commentary
We invited several outside experts to write independent commentaries on this work.
- Stanislas DehaeneandLionel Naccache are cognitive neuroscientists who, together with Jean-Pierre Changeux, developed the global neuronal workspace model that inspired much of our work.
- Patrick Butlin, Dillon Plunkett, Robert Long (Eleos AI Research) andDerek Shiller (Rethink Priorities) study the potential for consciousness and moral status in AI systems.
- Neel Nanda leads the language model interpretability team at Google DeepMind. His commentary includes an independent replication of some of our findings on an open-weight model.
Read their commentaries here.