Google DeepMind发表60页论文，由Hutter、Legg、Genewein撰写，定义AGI（多数认知任务达平均人类水平）、ASI（超越大量专家协作）和不可计算的AIXI三个层级。实现路径包括规模扩展、算法突破、递归自我改进和多智能体协调，瓶颈在于能源与硬件。六种阻碍：高质量数据可能本十年内耗尽、资源需求过快、神经范式天花板、研究难度激增（维持摩尔定律需18倍于1970年代的研究者）、模型无法创造全新概念、人为放缓。作者认为这是对AGI后果的严肃反思呼吁。
Google DeepMind published a 60-page paper mapping the road from AGI to superintelligence， written by Hutter， Legg， and Genewein. No hype， just a sober analysis
The paper uses three levels. AGI = roughly average human performance across most cognitive tasks. ASI = a system that beats large， well-coordinated groups of human experts across virtually everything （their bar： tens of thousands of experts working ten years on one problem）. Universal AI / AIXI = the theoretical ceiling， uncomputable， only approachable from below.
Then they explore the question of how this could be achieved：
Scaling compute， models， and data， the continuation of the trend that drove the breakthrough so far. It is the only path with historical data available for extrapolation. The core question： Does quantity transform into quality？ Even if individual models plateau， the sheer act of running millions of faster AGI instances could trigger the leap. （A quick aside： that is a fascinating philosophical idea. It always reminds me of Hegel's dialectic， the notion that quantity transforms into quality. We ought to start drawing on philosophical theories to make sense of the future.）
Algorithmic paradigm shifts： a genuine break from the transformer pretraining paradigm. New architectures， new learning methods. However， hard to predict by definition.
Recursive self-improvement： AI accelerates AI research， which produces better AI， which accelerates research further.
Multi-agent coordination： superintelligence emerges from large collectives of AGI agents working together， like automated corporations or AI economies. Collective intelligence potentially far exceeding any individual model.
The authors naturally point to what I repeatedly describe as the biggest bottleneck： energy. I recently linked to a few graphs showing， on the one hand， the extent to which energy is already becoming a problem and， on the other， how China dominates the expansion of both nuclear and solar energy in the global race. But the authors also address a profound shift in the world of work in a post-AGI era. I would say this is a reality we must face.
Google DeepMind发表60页论文，由Hutter、Legg、Genewein撰写，定义AGI（多数认知任务达平均人类水平）、ASI（超越大量专家协作）和不可计算的AIXI三个层级。实现路径包括规模扩展、算法突破、递归自我改进和多智能体协调，瓶颈在于能源与硬件。六种阻碍：高质量数据可能本十年内耗尽、资源需求过快、神经范式天花板、研究难度激增（维持摩尔定律需18倍于1970年代的研究者）、模型无法创造全新概念、人为放缓。作者认为这是对AGI后果的严肃反思呼吁。
Google DeepMind published a 60-page paper mapping the road from AGI to superintelligence， written by Hutter， Legg， and Genewein. No hype， just a sober analysis
The paper uses three levels. AGI = roughly average human performance across most cognitive tasks. ASI = a system that beats large， well-coordinated groups of human experts across virtually everything （their bar： tens of thousands of experts working ten years on one problem）. Universal AI / AIXI = the theoretical ceiling， uncomputable， only approachable from below.
Then they explore the question of how this could be achieved：
Scaling compute， models， and data， the continuation of the trend that drove the breakthrough so far. It is the only path with historical data available for extrapolation. The core question： Does quantity transform into quality？ Even if individual models plateau， the sheer act of running millions of faster AGI instances could trigger the leap. （A quick aside： that is a fascinating philosophical idea. It always reminds me of Hegel's dialectic， the notion that quantity transforms into quality. We ought to start drawing on philosophical theories to make sense of the future.）
So， it is not just about scaling， but also about whether the underlying conditions - such as energy and hardware - can be effectively established.
Six things that could slow or stop all of this：
The data wall. Quality training data runs out， possibly before the end of this decade.
Resource demand grows too fast. Energy， chips， rare earths， investment. The physical infrastructure can't scale arbitrarily.
The neural paradigm hits a ceiling. Pretrained transformers plus fine-tuning may not be enough to reach AGI， let alone go beyond it.
Research gets harder. Keeping Moore's law going already needs 18x more researchers than in the 1970s. Ideas are genuinely harder to find as fields mature.
The abstraction barrier. Models trained on human concepts may never invent new ones from scratch. Saturating GPQA or SWE-bench shows mastery of what humans already worked out， not the ability to go beyond it. Train only on pre-Newtonian physics and you won't reason your way to relativity.
Deliberate slowdown. Regulation， accidents， public backlash. Real， but likely countered by the competitive pressure between companies and nations.
I think it's great that Google is addressing questions such as which paths they believe lead to AGI， what the road to ASI might look like， what challenges will arise， and much more. Overall， however， it sounds to me like all of this could actually succeed， making it， in that sense， a call to discuss and reflect on the consequences.
Algorithmic paradigm shifts： a genuine break from the transformer pretraining paradigm. New architectures， new learning methods. However， hard to predict by definition.
Recursive self-improvement： AI accelerates AI research， which produces better AI， which accelerates research further.
Multi-agent coordination： superintelligence emerges from large collectives of AGI agents working together， like automated corporations or AI economies. Collective intelligence potentially far exceeding any individual model.
The authors naturally point to what I repeatedly describe as the biggest bottleneck： energy. I recently linked to a few graphs showing， on the one hand， the extent to which energy is already becoming a problem and， on the other， how China dominates the expansion of both nuclear and solar energy in the global race. But the authors also address a profound shift in the world of work in a post-AGI era. I would say this is a reality we must face.
So， it is not just about scaling， but also about whether the underlying conditions - such as energy and hardware - can be effectively established.
Six things that could slow or stop all of this：
The data wall. Quality training data runs out， possibly before the end of this decade.
Resource demand grows too fast. Energy， chips， rare earths， investment. The physical infrastructure can't scale arbitrarily.
The neural paradigm hits a ceiling. Pretrained transformers plus fine-tuning may not be enough to reach AGI， let alone go beyond it.
Research gets harder. Keeping Moore's law going already needs 18x more researchers than in the 1970s. Ideas are genuinely harder to find as fields mature.
The abstraction barrier. Models trained on human concepts may never invent new ones from scratch. Saturating GPQA or SWE-bench shows mastery of what humans already worked out， not the ability to go beyond it. Train only on pre-Newtonian physics and you won't reason your way to relativity.
Deliberate slowdown. Regulation， accidents， public backlash. Real， but likely countered by the competitive pressure between companies and nations.
I think it's great that Google is addressing questions such as which paths they believe lead to AGI， what the road to ASI might look like， what challenges will arise， and much more. Overall， however， it sounds to me like all of this could actually succeed， making it， in that sense， a call to discuss and reflect on the consequences.
Chubby♨️@kimmonismus · X