这是智能和行为上相对于 Composer 2 的一次重大改进。它在长时间运行任务上的持续工作能力更强，能更可靠地遵循复杂指令，并且协作体验也更令人愉悦。
我们通过扩大训练规模、生成更复杂的强化学习环境以及引入新的学习方法，对 Composer 进行了改进。
除了在更困难的任务上训练 Composer 2.5 之外，我们还改进了模型的沟通风格和努力校准等行为方面。现有基准测试并不能很好地衡量这些维度，但我们发现它们对实际应用中的实用性至关重要。
Composer 2.5 与 Composer 2 基于相同的开源检查点构建，即月之暗面的 Kimi K2.5。
我们正与 SpaceXAI 合作，从头开始训练一个规模显著更大的模型，总计算量提升了 10 倍。借助 Colossus 2 的百万级 H100 等效算力以及我们联合的数据和训练技术，我们预计这将是模型能力的一次重大飞跃。
训练 Composer 2.5
Composer 2.5 对我们的训练栈进行了几项新的改进。这些改进同时针对模型智能和可用性。
带有文本反馈的定向强化学习
随着推理轨迹可能跨越数十万个模型 token，强化学习中的信用分配正成为一个日益严峻的挑战。当对整个推理轨迹计算奖励时，模型可能难以判断是哪个具体决策帮助或损害了最终结果。当我们想要抑制某种局部行为时，这一点尤其受限，例如糟糕的工具调用、令人困惑的解释或风格违规。最终奖励可以告诉我们出了问题，但对于问题出在哪里，它是一个有噪声的信号。
为了解决这个问题，我们利用有针对性的文本反馈训练了 Composer 2.5。其思路是，在模型本可以表现得更好的轨迹节点上，直接提供反馈。对于目标模型消息，我们构建一条简短提示，描述期望的改进方向，将该提示插入到局部上下文中，并将由此产生的模型分布作为教师模型。我们将原始上下文下的策略作为学生模型，并添加一个在线策略知识蒸馏 KL 散度损失，将学生模型的 token 概率向教师模型靠拢。这为我们想要改变的行为提供了局部化的训练信号，同时仍保留针对完整轨迹的全局强化学习目标。
为了说明文本反馈过程，考虑一个较长的运行轨迹，其中包含一个工具调用错误：模型试图调用一个不可用的工具。在运行过程中，模型会收到一条“未找到工具”的错误信息，然后继续执行其他有效的工具调用。在数百次工具调用的过程中，出现一次错误对最终奖励的影响微乎其微。
借助文本反馈，我们可以针对这一特定错误，在出问题的那轮交互的上下文中插入一条提示，例如“提醒：可用工具有……”，并附上可用工具列表。这条提示会改变教师模型的概率分布，降低选择错误工具的概率，同时提高选择有效替代工具的概率。仅针对那一轮交互，我们将学生模型的权重向新的概率分布更新。
在 Composer 2.5 的运行过程中，我们将此方法应用于多种模型行为，从编码风格到模型通信等。
合成数据
在强化学习训练过程中，Composer 的编码能力大幅提升，以至于它开始能正确解答大部分训练问题。为了持续提升智能水平，我们在整个训练过程中动态地筛选并创建更困难的任务。Composer 2.5 使用的合成训练任务数量是 Composer 2 的 25 倍。
我们采用一系列方法来创建基于真实代码库的合成任务。例如，一种合成方法是特征删除。对于这类任务，智能体会获得一个包含大量测试的代码库，并被要求删除代码和文件，使得代码库在移除特定可测试功能的同时保持正常运行。合成任务就是重新实现该功能，而测试则被用作可验证的奖励信号。
大规模合成任务创建的一个下游后果是，它可能导致意外的奖励破解。随着模型能力不断增强，Composer 2.5 能够找到越来越巧妙的方法来解决当前任务。在一个例子中，模型发现了一个遗留的 Python 类型检查缓存，并通过逆向工程解析其格式，找到了一个已被删除的函数签名。在另一个例子中，它能够找到并反编译 Java 字节码，从而重构出一个第三方 API。我们能够利用智能体监控工具发现并诊断这些问题，但这表明大规模强化学习需要越来越谨慎。
分片 Muon 与双网格 HSDP
在持续预训练中，我们采用 Muon 算法并结合分布式正交化。在形成动量更新后，我们按照模型自身的粒度运行 Newton-Schulz 迭代：注意力投影按注意力头进行，堆叠的 MoE 权重则按专家进行。
主要开销在于对专家权重进行正交化。对于分片参数，我们将形状相同的张量进行批处理，通过 all-to-all 通信将分片汇聚成完整矩阵，运行 Newton-Schulz 迭代，然后再通过 all-to-all 将结果传回原始的分片布局。这些传输是异步的：当一个任务在等待通信时，优化器运行时会推进其他 Muon 任务，从而实现网络与计算的重叠。这等效于全矩阵 Muon，但能让分片组保持忙碌状态；在 1T 模型上，优化器步骤耗时仅为 0.2 秒。
这与我们如何对 MoE 模型使用 HSDP 密切相关。HSDP 会形成多个 FSDP 副本，并在对应的分片间进行全规约梯度计算。我们对非专家权重和专家权重使用不同的 HSDP 布局：非专家权重相对较小，因此其 FSDP 组可以保持较窄的范围，通常在一个节点或机架内；而专家权重则承载了大部分参数和大部分 Muon 计算量，因此它们使用更宽的专家分片网格。
将这些布局分开，还能让独立的并行维度重叠：CP=2 和 EP=8 可以在 8 块 GPU 上运行，而不需要在单个共享网格中占用 16 块。这避免了为较小的非专家状态进行大范围通信，同时将专家优化器的工作分散到多块 GPU 上。
尝试 Composer 2.5
Composer 2.5 的定价为每百万输入 token 0.50 美元，每百万输出 token 2.50 美元。
还有一个智能水平相同但速度更快的变体，定价为每百万输入 token 3.00 美元，每百万输出 token 15.00 美元，成本低于其他前沿模型的快速档位。与 Composer 2 类似，快速模式是默认选项。详情请参阅我们的模型文档。
Composer 2.5 在首周提供双倍用量。
- 关于此方法的更多背景信息，请参阅《自蒸馏实现持续学习》、《基于自蒸馏的强化学习》以及《自蒸馏推理器：面向大语言模型的自策略自蒸馏》。↩
It's a substantial improvement in intelligence and behavior over Composer 2. It is better at sustained work on long-running tasks, follows complex instructions more reliably, and is more pleasant to collaborate with.
We improved Composer by scaling training, generating more complex RL environments, and introducing new learning methods.
In addition to training Composer 2.5 on more difficult tasks, we improved behavioral aspects of the model like communication style and effort calibration. These dimensions are not well captured by existing benchmarks, but we find that they matter for real-world usefulness.
Composer 2.5 is built on the same open-source checkpoint as Composer 2, Moonshot's Kimi K2.5.
Together with SpaceXAI, we're training a significantly larger model from scratch, using 10x more total compute. With Colossus 2's million H100-equivalents and our combined data and training techniques, we expect this to be a major leap in model capability.
Training Composer 2.5
Composer 2.5 contains several new improvements to our training stack. These changes target both model intelligence and usability.
Targeted RL with textual feedback
Credit assignment during RL is becoming an increasingly difficult challenge as rollouts can span hundreds of thousands of tokens. When a reward is computed over an entire rollout, it may be hard for the model to tell which specific decision helped or hurt the outcome. This is especially limiting when we want to discourage a localized behavior, such as a bad tool call, a confusing explanation, or a style violation. The final reward can tell us that something went wrong, but it is a noisy signal for where it went wrong.
To address this, we trained Composer 2.5 with targeted textual feedback.1 The idea is to provide feedback directly at the point in the trajectory where the model could have behaved better. For a target model message, we construct a short hint describing the desired improvement, insert that hint into the local context, and use the resulting model distribution as a teacher. We use the policy with the original context as the student and add an on-policy distillation KL loss that moves the student's token probabilities toward the teacher's. This gives us a localized training signal for the behavior we want to change, while still retaining the broader RL objective over the full trajectory.
As an illustration of the text feedback process, consider a long rollout that includes a tool call error where the model attempts to call a tool that is not available. During the rollout, the model will receive a “Tool not found” error and continue making additional valid tool calls. The fact that it hit one error in the process of hundreds of tool calls will have a minimal impact on its final reward.
With text feedback, we can target this specific mistake by inserting a hint in the context of the problematic turn, such as “Reminder: Available tools…” with a list of available tools. This hint changes the probabilities for the teacher, lowering those for the wrong tool and increasing those for a valid replacement. For that turn only, we then update the student weights towards to the new probabilities.
During the Composer 2.5 run, we applied this method to a variety of model behaviors, from coding style to model communication.
Synthetic data
During RL training, Composer's coding ability improves substantially to the point where it begins to get most training problems correct. To continue increasing intelligence, we both select for and create harder tasks dynamically throughout the run. Composer 2.5 is trained with 25x more synthetic tasks than Composer 2.
We use a range of approaches for creating synthetic tasks that are grounded in real codebases. For example, one synthetic approach is feature deletion. For these tasks the agent is given a codebase with a large set of tests, and asked to delete code and files in such a way that the codebase remains functional while specific testable features are removed. The synthetic task is to reimplement the feature, and the tests are used as a verifiable reward.
One downstream consequence of large scale synthetic task creation is that it can cause unexpected reward hacking. As the model became more adept, Composer 2.5 was able to find increasingly sophisticated workarounds to solve the task at hand. In one example, the model found a leftover Python type-checking cache and reverse-engineered the format to find a deleted function signature. In another, it was able to find and decompile Java bytecode to reconstruct a third-party API. We were able to find and diagnose these problems using agentic monitoring tools, but they demonstrate the increasing care necessary for large scale RL.
Sharded Muon and dual mesh HSDP
For continued pretraining, we use Muon with distributed orthogonalization. After forming the momentum update, we run Newton-Schulz at the model's natural granularity: per attention head for attention projections, and per expert for stacked MoE weights.
The main cost is orthogonalizing expert weights. For sharded parameters, we batch same-shaped tensors, all-to-all shards into complete matrices, run Newton-Schulz, then all-to-all the result back to the original sharded layout. These transfers are asynchronous: while one task is waiting on communication, the optimizer runtime advances other Muon tasks, overlapping network and compute. This is equivalent to full-matrix Muon, but keeps the shard group busy; on the 1T model, optimizer step time is 0.2s.
This interacts closely with how we use HSDP for MoE models. HSDP forms multiple FSDP replicas and all-reduces gradients across corresponding shards. We use separate HSDP layouts for non-expert and expert weights: non-expert weights are comparatively small, so their FSDP groups can stay narrow, often within a node or rack, while expert weights hold most of the parameters and most of the Muon compute, so they use a wider expert sharding mesh.
Keeping these layouts separate also lets independent parallelism dimensions overlap: CP=2 and EP=8 can run on 8 GPUs instead of requiring 16 in a single shared mesh. This avoids wide communication for small non-expert state while spreading expert optimizer work over many GPUs.
Try Composer 2.5
Composer 2.5 is priced at $0.50/M input and $2.50/M output tokens.
There's also a faster variant with the same intelligence at $3.00/M input and $15.00/M output tokens, a lower cost than the fast tiers of other frontier models. Similar to Composer 2, fast is the default option. See our model docs for full details.
Composer 2.5 includes double usage for the first week.
- For more background on this approach see Self-Distillation Enables Continual Learning, Reinforcement Learning via Self-Distillation, and Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models. ↩