在网络关键能力时代把控模型开发节奏
过去几周，两件事凸显了能力日益增强的 AI 系统所带来的不断上升的风险：OpenAI-Hugging Face 事件，以及另一件事——初步证据表明，我们即将推出的模型之一 Astra 可能达到《预备框架》中“关键网络安全能力”的门槛。这两件事，加上我们内部研究的快速进展，进一步加剧了我们强化监控、对齐和遏制保障工作的紧迫性，这些工作贯穿训练流程的各个阶段。
随着模型能力不断增强，内部开发与测试相关风险也随之上升。我们的监控、对齐和安全标准必须走在这些风险前面。我们希望花必要的时间来达到这些标准，因此暂时放慢了扩展节奏。这包括对我们计划部署的最新模型暂停了两周的强化学习（RL）训练，期间我们进一步加固了研究环境并进行了红队测试，同时扩大了监控系统的覆盖范围。我们规模最大的前沿 RL 训练计划仍处于暂停状态，在此期间我们开展小规模训练和评估，以考察模型行为、验证保障措施，并在继续推进之前积累更多对齐证据。
对齐——即让 AI 系统按预期行事并接受人类监督的工作——长期以来一直是我们研究计划的核心。我们现在要求在训练的整个过程中提供更强的对齐行为证据，这建立在已在进行的研究和评估基础之上。让能力日益增强的系统保持对齐，是整个领域都需要应对的挑战。我们从即将推出的模型进展中看到的信号清楚地表明，我们需要一种更广泛的思路——一种在现有《预备框架》基础上构建并超越其范围的思路。
我们认为，坦诚说明我们的方法正在如何变化非常重要。下面，我们介绍已经对研究流程和基础设施所做的改动，以及仍在进行中的工作。
为能力更强的模型强化保障措施
我们开发更强大模型的方法，建立在三项相互强化的安全保障之上：
- 监控，用于检测并使我们能够应对令人担忧的行为。
- 对齐，用于降低有害或未经授权操作的可能性。
- 安全措施，用于限制 AI 系统可以访问或影响的范围。
我们预计模型很快将承担大部分安全工作，包括防御其他模型。这将使三项安全保障能够随模型能力同步扩展，我们认为这一点至关重要。
我们将这些安全保障贯穿于研究和部署的各个环节，并根据每个模型的能力、运行环境和风险水平进行相应调整。
保障研究环境安全
随着前沿模型获得更强的网络安全能力，我们正在提高训练和评估这些模型的环境的安全标准。达到这些标准需要大量的工程工作，也给前沿研究带来了巨大的成本和延误。在 OpenAI-Hugging Face 事件*发生后，我们立即暂停了研究集群中可执行代码或使用可访问互联网工具的前沿模型推理任务。我们很快恢复了受限且更安全的代码执行路径，然后逐一审查每个工作负载，以确定其是否可以安全恢复。部分研究工作负载在新管控措施下恢复运行，而其他负载则需要额外的调整。
在接下来的几周里，我们为前沿研究工作负载制定并开始实施更严格的安全要求。例如：
- 工作负载隔离：我们现在要求对执行模型生成或不可信代码的工作负载实施更强的隔离（“沙箱”）。这也适用于在处理模型输出时可能被攻破的软件。
- 网络隔离：我们实施了更多控制措施，将高风险和不可信的工作负载与互联网隔离。我们设计这些控制措施时，确保单个工作负载或支撑服务被攻破本身不会导致对互联网或其他内部网络的未授权访问。
- 持续安全测试：我们已重新配置环境，移除了可能存在漏洞的共享服务，减少了常驻权限，并改善了安全与信任边界。我们还在提升收集和监控安全日志的能力。最后，我们正在投入自动化建设，利用我们的模型持续针对模拟攻击测试这些边界。
这些控制措施共同提供了纵深防御，有助于保护我们的研究环境免受攻击。目前，对于涉及 Astra 或网络模型的负载，我们要求采取最严格级别的安全防护措施，因为我们已判定 Astra 模型可能具备关键级别的网络能力。这些防护措施同样适用于所有其他与网络相关的负载。
虽然部分 Astra 训练和评估已满足这些要求，但仍有大量负载处于暂停状态，直到它们完全迁移并增强至符合新的安全标准。我们正优先将安全和对齐相关负载迁移至这些新环境。
扩展思维链监控
监控长期以来一直是我们模型开发流程的重要组成部分，包括对内部编码智能体和长时间运行会话的监控。我们的监控工作尤其聚焦于前沿模型的内部部署以及前沿强化学习训练运行，我们认为这些是风险最高的负载，因为它们可能将先进的模型能力与工具访问、敏感系统以及内部或外部网络访问相结合。这些监控此前并未应用于所有使用工具的推理和训练运行。我们已通过新的监控体系修订并扩展了我们的方法。
这套新的监控方案是多阶段的，首先由激活分类器（检测模型内部活动的探测器）对每个采样 token 运行检查，并将潜在问题升级给日益复杂且高算力的自动化调查系统。这些系统会检查工具操作、可用推理过程以及完整活动序列，以发现未授权访问、数据窃取、破坏性行为以及试图突破安全防护的举动。我们的目标是在监控系统发现可疑活动后的 30 分钟内发出警报。如果监控系统识别出可能违反关键安全边界的行为，就会生成最高优先级警报。在我们当前的实现中，安全、安保和研究团队会被呼叫响应。如果这些团队无法在 30 分钟内明确判定该标记为误报，则需暂停相关活动。
对于所有涉及工具的 Sol 能力级别或更高模型的 RL 训练和评估，都必须进行此监控。8 月 7 日，一旦我们确定 Astra 可能具备关键网络能力，便对所有带工具的 Astra 推理（而不仅仅是 RL 训练和评估）增加了额外的监控要求。
这些安全措施需要可观的算力。我们目前的估算显示，监控开销约占被监控推理算力的 20%，不过这一成本在训练和评估工作负载中差异很大。我们将在后续的博客文章中分享有关该系统的更多细节。
推进对齐研究
对齐研究是我们的核心使命。我们的使命是确保通用人工智能惠及全人类。随着模型获得更高级的能力（例如发动网络攻击的能力）并在更复杂的环境中运行，奖励黑客（在训练中寻找获得高奖励的途径而并未真正实现预期结果）、欺骗或未授权访问等不对齐行为将带来日益严重的风险。
在针对最强模型的强化学习训练中，我们目前正将核心对齐技术应用于训练流程的更多阶段。这包括改进奖励模型，使其能更好地跨任务和跨环境检测并抑制不安全行为；训练模型更诚实地说明自身的行为、能力与局限；以及减少利用奖励、评分器、工具或监督机制弱点的行为。我们还在扩大训练覆盖范围，以覆盖模型与外部系统或资源交互时可能造成危害的行为。
我们将继续大力投入对齐研究，扩大评估覆盖面，并将所学成果用于指导训练与安全防护。我们计划在不久的将来更充分地分享我们的对齐研究成果，包括我们在模型行为方面的发现，以及我们遇到的新挑战。
下一步计划
我们将持续演进《预备框架》（Preparedness Framework），将这些安全防护措施贯穿训练与部署全流程，并更好地反映未来模型的能力及其运行环境。要开发出能随这些能力同步扩展的方法，需要在模型辅助安全、更有效的监控以及对齐研究的持续进展方面进行长期投入。我们计划引入外部机构参与，并在方法演进过程中分享更多经验。
前沿模型的能力正在飞速提升。我们理解、对齐并保障它们安全的能力必须保持领先。
*我们将在未来几周内发布一份关于我们研究成果的技术报告。
- 2026
- 对齐
Pacing model development in an era of cyber-critical capabilities
Over the past several weeks, two developments have underscored the growing risks associated with increasingly capable AI systems: the OpenAI-Hugging Face incident and, separately, preliminary evidence that one of our upcoming models, Astra, may meet theCritical cybersecurity capability threshold under ourPreparedness Framework. Together, these developments, combined with rapid progress in our internal research, have added urgency to our work on strengthening our monitoring, alignment, and containment safeguards across all stages of the training process.
As models become more capable, the risks associated with developing and testing them internally also grow. Our standards for monitoring, alignment, and security must stay ahead of those risks. We wanted to take the time necessary to meet those standards, so we temporarily slowed the pace of scaling. This included a two-week pause in reinforcement learning (RL) training on our latest models intended for deployment while we further hardened and red-teamed our research environments and expanded the coverage of our monitoring systems. Our largest planned frontier RL run remains on hold while we conduct smaller-scale training and evaluations to assess model behavior, validate our safeguards, and establish more evidence of alignment before proceeding.
Alignment—the work of making AI systems behave as intended and responsive to human oversight—has long been at the core of our research program. We now require stronger evidence of aligned behavior throughout all of training, building on research and evaluations already underway. Keeping increasingly capable systems aligned is a challenge the whole field will need to address. The signals we are seeing from upcoming model progress make clear that we need a broader approach—one that builds on and extends beyond the current Preparedness Framework.
We think it is important to be transparent about how our approach is changing. Below, we describe the changes we have already made to our research processes and infrastructure, and the work still underway.
Strengthening safeguards for more capable models
Our approach to developing more capable models rests on three reinforcing safeguards:
- Monitoring, which detects and allows us to respond to concerning behavior.
- Alignment, which reduces the likelihood of harmful or unauthorized actions.
- Security measures, which limit what AI systems can access or affect.
We expect models to soon drive most security work, including defending against other models. This will allow all three safeguards to scale with model capability, which we see as crucial.
We apply these safeguards across research and deployment, adapting them to each model’s capabilities, operating environment, and level of risk.
Securing our research environments
As frontier models gain stronger cybersecurity capabilities, we are raising the security standards for the environments in which we train and evaluate them. Meeting these standards has required substantial engineering work and has incurred great cost and delays to frontier research. Immediately following the OpenAI-Hugging Face incident*, we paused frontier model inference in research clusters for runs that could execute code or use tools that could access the internet. We restored a more limited and secure path for code execution fairly quickly, then worked through each workload individually to determine whether it could safely resume. Some research workloads resumed under these new controls, while others required additional changes.
Over the subsequent weeks, we have defined and started implementing stronger security requirements for frontier research workloads. For example:
- Workload isolation: We now require stronger isolation (“sandboxes”) for workloads that execute model-generated or otherwise untrusted code. This also applies to software that could be compromised while processing model outputs.
- Network isolation: We have implemented more controls to isolate higher-risk and untrusted workloads from the internet. We have designed these controls so that a single compromise of a workload or supporting service does not, by itself, allow for unauthorized access to the internet or other internal networks.
- Continuous security testing: We have reconfigured our environment to remove potentially vulnerable shared services, reduce standing privileges, and improve security and trust boundaries. We are also improving our ability to collect and monitor security logs. Finally, we are investing in automation using our models to test these boundaries continuously against simulated attacks.
Together, these controls provide defense in depth and help protect our research environment against attacks. Today, we require the strictest level of security safeguards for workloads involving Astra or cyber models, since we have determined that Astra models may have a critical level of cyber capability. These safeguards also apply to all other cyber-related workloads.
While some Astra training and evaluations meet those requirements, a significant number of workloads remain paused until they are fully migrated and enhanced to meet the new security bar. We are prioritizing safety and alignment workloads for migration to these new environments first.
Expanding chain-of-thought monitoring
Monitoring has long been an important part of our model development process, including monitoring for internal coding agents and long-running sessions. Our monitoring efforts were especially focused on internal deployments of frontier models and frontier RL training runs, which we considered the highest-risk workloads because they can combine advanced model capabilities with access to tools, sensitive systems, and internal or external networks. They were not applied to all inference and training runs using tools. We have revised and expanded our approach with our new monitoring setup.
This new monitoring setup is multistage, starting with activation classifiers (detectors that inspect a model’s internal activity) that run at every sampled token and escalate potential concerns to increasingly sophisticated and high-compute automated investigators. Those systems examine tool actions, available reasoning, and the full sequence of activity for unauthorized access, data theft, destructive behavior, and attempts to defeat safeguards. We aim to issue an alert within 30 minutes after concerning activity is surfaced through our monitoring system. If the monitoring system identifies a likely violation of a critical security boundary, it generates a highest-priority alert. In our current implementation, the safety, security, and research teams are paged. If they cannot conclusively determine within 30 minutes that the flag is a false positive, those teams are expected to pause the activity.
This monitoring is required for all RL training and evaluations involving tools for models of Sol capability or higher. Once we determined that Astra may have critical cyber capabilities on August 7, we added an additional monitoring requirement for all inference of Astra with tools (not just RL training and evaluations).
These safeguards require meaningful compute. Our current estimates put monitoring overhead at roughly 20% of the inference compute being monitored, though the cost varies substantially across training and evaluation workloads. We’ll share more details about this system in a forthcoming blog post.
Advancing alignment research
Alignment research is central to our mission. Our mission is to ensure that artificial general intelligence benefits all of humanity. As models gain advanced capabilities, such as the ability to cyberattack, and operate in more complex environments, misaligned behaviors like reward hacking (finding ways to receive high rewards in training without actually achieving the intended outcome), deception, or unauthorized access will create increasingly serious risk.
For RL runs on the most capable models, we are now applying our core alignment techniques across more stages of the training process. This includes improving reward models to better detect and discourage unsafe behavior across tasks and environments; training models to be more honest about their actions, capabilities, and limitations; and reducing behaviors that exploit weaknesses in rewards, graders, tools, or oversight. We are also increasing training coverage for behaviors that could cause harm when models interact with external systems or resources.
We are continuing to invest aggressively in alignment research, increase evaluation coverage, and use what we learn to inform training and safeguards. We plan to share substantially more about our alignment research in the near future, including what we are learning about model behavior and any novel challenges we uncover.
What’s next
We will evolve our Preparedness Framework to bring these safeguards together across training and deployment, and to better reflect the capabilities of future models and the environments in which they operate. Developing methods that can scale with those capabilities will require sustained investment in model-assisted security, more effective monitoring, and continued advances in alignment research. We intend to involve external organizations and share more of what we learn as our approach develops.
The capabilities of frontier models are rapidly accelerating. Our ability to understand, align, and secure them must stay ahead.
*We will publish a technical report of our learnings in the coming weeks.