关键要点
- 一个德国研究联盟发布了开源大语言模型 Soofi S，该模型完全在德国电信的 AI 云基础设施上完成训练。
- 该模型采用资源高效的混合架构，每生成一个模型 token 仅激活其 316 亿参数中的 32 亿，即使输入极长也能保持处理速度恒定。
- 由于训练数据高度侧重德语，Soofi S 在德语、英语和编程任务的基准测试中，表现优于其他完全开源模型，如 Olmo 3 32B 和 Apertus 70B。
Soofi S 是首批完全在慕尼黑德国电信工业 AI 云上训练的大语言模型之一。这款开源的 30B 模型采用精简的混合架构，训练数据组合有意向德语倾斜。
由德国人工智能协会（KI Bundesverband）协调的德国研究联盟发布了 Soofi S 30B-A3B，这是一款开源大语言模型。根据其预训练报告，该模型在英语和德语基准测试中，于完全开源模型中取得了最高分数，超越了此前领先的 OLMo 3 32B 和 Apertus 70B。
为长上下文而构建的精简架构
Soofi S 是一款混合专家模型。它总共包含 316 亿参数，但每生成一个模型 token 仅激活约 32 亿参数。这使得其计算成本更接近 3B 模型，而非传统的 30B 模型。该联盟直接采用了英伟达 Nemotron 3 Nano 的架构，这是一种结合了 Mamba-2 层与标准注意力层的混合设计。
与传统 Transformer 架构的关键区别在于内存行为。在传统模型中，用于注意力计算、存储先前模型 token 的 KV 缓存会随上下文长度线性增长。当输入较长且并行请求较多时，重新加载该缓存会成为瓶颈。而 Soofi S 的 52 层中，仅有 6 层会维护这样的缓存。
实际收益体现在生成吞吐量上。在上下文长度为 40,000 个 token、32 个并行请求的情况下，Soofi S 每 GPU 每秒生成的 token 数量大约是 140 亿到 240 亿参数规模稠密模型的八倍。随着上下文增长，传统模型的吞吐量显著下降，而 Soofi S 在 4,000 到 256,000 个 token 范围内几乎保持平稳。在测量中，唯一表现出类似行为的模型是阿里巴巴的通义千问 Qwen3.5 35B-A3B，它也采用了混合架构。
围绕德语构建的训练数据混合
该联盟总共处理了约 27 万亿个 token，分为三个阶段。在第一阶段，模型从约 20 万亿个 token 中学习语言基础知识，这些 token 来自网络、代码、数学和特定领域文本的广泛混合。第二阶段紧随其后，使用约 6 万亿个来自更高质量来源的 token，旨在强化之前学到的模式。随后，一个较短的第三阶段通过训练长达一百万个 token 的超长文档来扩展上下文窗口。
对德语的刻意关注是核心。在第一阶段，德语占训练数据混合的 7.2%；在第二阶段，这一比例上升到 15.3%。在英伟达的 Nemotron 参考方案中，所有非英语语言合计仅占约 5%。
在数据来源方面，该联盟结合了来自 HPLT 的德语网络文本、开放许可的德语 Commons 语料库、FinePDFs 和 FineWiki 中的德语部分，以及商业授权的 Genios 语料库（包含来自 916 家德语出版物的 1.93 亿篇报纸文章）。机器翻译和合成生成的德语文本则构成了数据混合的其余部分。
在德语和英语方面均取得顶尖开源模型得分
根据报告，在与另外 16 个开源模型的评估对比中，Soofi S 在德语和英语的综合得分上均领先所有完全开源模型。这包括艾伦人工智能研究所的 OLMo 3 32B 以及苏黎世联邦理工学院和洛桑联邦理工学院的 Apertus 70B。与所有欧洲主权基线模型相比，该模型在该套件的所有德语基准测试中均表现领先，有时领先幅度达到两位数百分比。
在代码基准测试中，Soofi S 在 HumanEval 上得分为 73.8%，在 MBPP 上为 70.2%，在德语版 MBPP 上为 84.2%，均为开源模型中的最佳成绩。在针对德国特定区域知识的测试 INCLUDE-DE 中，Soofi S 与规模更大的 Qwen3.5 35B-A3B 并列第一，得分均为 61.2 分。与 Nemotron 基线模型相比，德语数据方案使语言能力提升了 15.1 分，科学测试 GPQA-Diamond 提升了 9.6 分，且未牺牲英语性能。
Soofi S 在德语竞赛数学方面表现不佳，在 Minerva MATH-DE 上得分为 56 分，远落后于 Qwen3.5 35B-A3B（76.5 分）和 Gemma 3 27B（65.6 分）。在 NaturalQuestions 的开放事实检索方面也表现落后。后者可能与其仅有 30 亿个活跃参数有关，这使其能够存储的世界知识少于稠密的 27B 模型。
RULER 长上下文测试还揭示了一个特定弱点：当模型需要从长文本中提取频繁出现的词语时，在上下文超过 32,000 个 token 后，Soofi S 的命中率降至约 3%，而可比的 Nemotron 模型仍能达到 60% 至 64%。作者将此归因于其长上下文训练数据包含许多长文档，但缺乏专为提取任务设计的合成数据。在其余十二项 RULER 任务中，两个模型的表现大致相同。
主权基础设施与有据可查的开放性
训练运行于三月至五月期间进行，在慕尼黑德国电信工业人工智能云上使用了多达 512 块 Nvidia B200 GPU，总计约 253,000 GPU 小时。根据报告，该设施完全使用可再生能源运行，冷却用水取自 Eisbach 运河，并将废热输送到周边的 Tucherpark 街区。Soofi S 是该基础设施上首批大规模训练运行之一。
Soofi 背后是由德国研究机构和企业组成的联合体，由德国人工智能协会协调，并作为欧洲 IPCEI-CIS 项目的一部分，由德国联邦经济事务和能源部资助。
参与方包括弗劳恩霍夫 IAIS 和 IIS 研究所、德国人工智能研究中心（DFKI）、达姆施塔特工业大学、维尔茨堡大学、L3S 研究中心、柏林应用科学大学，以及人工智能公司 Ellamind 和 Merantix Momentum。该项目目标是构建一个可在主权基础设施上运行并用于工业应用测试的开放欧洲人工智能模型系列。
研究人员将发布模型权重以及选定的中间检查点、完整的训练和评估代码，以及一份详细的数据清单，列出原始 token 数量、训练轮次数和各数据源的有效贡献。经过审查但被排除的数据源也一并记录在案。研究团队表示，这意味着 Soofi S 符合开放源代码促进会的《开源人工智能定义 1.0》。
一项更严格的欧洲开放数据定义提案要求每个训练 token 都必须可自由分发，但由于 Genios 数据占 1.3% 的份额且带有商业许可，Soofi S 未能满足该要求。报告称，约 99% 的训练数据组合可以独立重建。该模型发布的具体许可尚未最终确定。
正如主要作者 Michael Fromm 所写，Soofi S 将自己定位在覆盖多种语言的广泛多语言欧洲主权项目（如 EuroLLM 或 Teuken）与性能最高的国际开源权重模型之间。根据项目官网介绍，该联盟正在为下一阶段寻找行业合作伙伴，以便在涉及技术文档、代码生成和基于智能体的系统等应用中测试该模型。
Key Points
- A German research consortium has released the open-source language model Soofi S, which was trained entirely on Deutsche Telekom's AI cloud infrastructure.
- The model uses a resource-efficient hybrid architecture that activates only 3.2 of its 31.6 billion parameters per token, keeping processing speed constant even with very long inputs.
- With a strong focus on German training data, Soofi S outperforms other fully open models, such as Olmo 3 32B and Apertus 70B, in benchmarks for German, English, and programming tasks.
Soofi S is one of the first large language models trained entirely on Deutsche Telekom's Industrial AI Cloud in Munich. The open 30B model uses a lean hybrid architecture and a training mix deliberately weighted toward German.
A German research consortium coordinated by the KI Bundesverband (German AI Association) has released Soofi S 30B-A3B, an open language model that, according to its pretraining report, achieves the highest scores on English and German benchmarks among fully open models, surpassing previous leaders like OLMo 3 32B and Apertus 70B.
A lean architecture built for long contexts
Soofi S is a mixture-of-experts model. It contains 31.6 billion parameters in total but activates only about 3.2 billion per generated token. That puts its compute cost closer to a 3B model than a conventional 30B model. The consortium adopts the architecture of Nvidia's Nemotron 3 Nano without modification, a hybrid design combining Mamba-2 layers with standard attention layers.
The key difference from typical transformers is memory behavior. In conventional models, the KV cache that stores previous tokens for attention computation grows linearly with context length. With long inputs and many parallel requests, reloading that cache becomes a bottleneck. Only 6 of Soofi S's 52 layers maintain such a cache at all.
The practical payoff shows up in generation throughput. At a context length of 40,000 tokens with 32 parallel requests, Soofi S generates roughly eight times more tokens per second per GPU than dense models in the 14 to 24 billion parameter range. While throughput drops significantly for conventional models as context grows, Soofi S stays nearly flat from 4,000 to 256,000 tokens. The only model that shows similar behavior in the measurements is Alibaba's Qwen3.5 35B-A3B, which also uses a hybrid architecture.
A training mix built around German
The consortium processed about 27 trillion tokens in total, split across three phases. In the first phase, the model learns language fundamentals from roughly 20 trillion tokens drawn from a broad mix of web, code, math, and domain-specific texts. A second phase follows with about 6 trillion tokens from higher-quality sources, designed to sharpen the patterns learned earlier. A shorter third phase then extends the context window by training on very long documents of up to one million tokens.
The deliberate focus on German is central. In the first phase, German makes up 7.2 percent of the training mix; in the second phase, that share rises to 15.3 percent. In Nvidia's Nemotron reference recipe, all non-English languages combined account for only about 5 percent.
For data sources, the consortium combines German web text from HPLT, the openly licensed German Commons corpus, German portions of FinePDFs and FineWiki, and the commercially licensed Genios corpus containing 193 million newspaper articles from 916 German publications. Machine-translated and synthetically generated German texts round out the mix.
Top open-model scores in both German and English
In evaluations against 16 other open models, Soofi S leads all fully open models on aggregate scores for both German and English, according to the report. That includes OLMo 3 32B from the Allen Institute for AI and Apertus 70B from ETH Zurich and EPFL. Against every European sovereign baseline, the model comes out ahead on all German benchmarks in the suite, sometimes by double-digit margins.
On code benchmarks, Soofi S scores 73.8 percent on HumanEval, 70.2 on MBPP, and 84.2 on the German MBPP variant, the best results among open-source peers. On INCLUDE-DE, a test for Germany-specific regional knowledge, Soofi S ties for first place at 61.2 points with the larger Qwen3.5 35B-A3B. Compared to the Nemotron baseline, the German data recipe improves language proficiency by 15.1 points and the science test GPQA-Diamond by 9.6 points, without sacrificing English performance.
Soofi S doesn't do as well on German competition math, where it scores 56 points on Minerva MATH-DE, well behind Qwen3.5 35B-A3B (76.5) and Gemma 3 27B (65.6). It also lags on open factual retrieval in NaturalQuestions. The latter likely relates to having only 3 billion active parameters, which can store less world knowledge than a dense 27B model.
The RULER long-context test also reveals a specific weakness: When the model has to extract frequently occurring words from a long text, Soofi S's hit rate drops to around 3 percent beyond 32,000 tokens of context, while the comparable Nemotron model still manages 60 to 64 percent. The authors attribute this to the fact that their long-context training data contains many long documents but lacks synthetic data designed for extraction tasks. On the remaining twelve RULER tasks, both models perform about the same.
Sovereign infrastructure and documented openness
The training run took place between March and May on up to 512 Nvidia B200 GPUs at Deutsche Telekom's Industrial AI Cloud in Munich, totaling about 253,000 GPU-hours. According to the report, the facility runs entirely on renewable energy, is cooled with water from the Eisbach canal, and feeds waste heat into the surrounding Tucherpark neighborhood. Soofi S was one of the first major training runs on this infrastructure.
Behind Soofi is a consortium of German research institutions and companies, coordinated by the German AI Association and funded by the German Federal Ministry for Economic Affairs and Energy as part of the European IPCEI-CIS program.
Participants include the Fraunhofer Institutes IAIS and IIS, the German Research Center for Artificial Intelligence (DFKI), TU Darmstadt, the University of Würzburg, the L3S Research Center, the Berlin University of Applied Sciences, and AI companies Ellamind and Merantix Momentum. The project's goal is to build an open European AI model family that can run on sovereign infrastructure and be tested in industrial applications.
The researchers are releasing model weights along with selected intermediate checkpoints, the complete training and evaluation code, and a detailed data inventory listing raw token counts, epoch numbers, and effective contributions per source. Sources that were reviewed but excluded are also documented. According to the team, this means Soofi S meets the Open Source AI Definition 1.0 from the Open Source Initiative.
A stricter proposal for a European open-data definition, which would require every single training token to be freely distributable, isn't met because of the 1.3 percent share of Genios data, which carries a commercial license. The report says about 99 percent of the training mix can be independently reconstructed. The exact license for the model's release hasn't been finalized yet.
As lead author Michael Fromm writes, Soofi S positions itself between broadly multilingual European sovereignty projects like EuroLLM or Teuken, which cover many languages, and the highest-performing international open-weight models. According to the project website, the consortium is looking for industry partners for the next phase to test the model in applications involving technical documents, code generation, and agent-based systems.