SemiAnalysis wrote a super long piece on OpenAI's Jalapeño chip.

Some revelations

• OpenAI may be designing infrastructure for: models measured in tens of trillions of parameters or context windows containing millions of tokens.

• They questioned if CUDA can survive when AI itself can rapidly write and optimize software for a completely new architecture.

• The chip showed for the first time that chips may no longer need perfect universal compilers if frontier models can write the hard parts themselves.

"If Jalapeño is a success, it will be a strong signal that the industry’s obsession over programming models and perfect, universal compilers are invalidated by frontier AI models."

• “OpenAI designs for perf/W.” OpenAI appears to be optimizing around a different scarce resource: not money, and not floor space, but electricity. Once power becomes the hard ceiling, tokens per watt starts looking like the real currency of AI infrastructure.

• Jalapeño isn't being designed as an isolated accelerator. OpenAI is building the networking architecture to make thousands of its own chips behave like one enormous inference machine.

• “Jalapeño smokes every other chip.” Specifically about token throughput per unit of datacenter power. In an industry increasingly constrained by electricity, that may be a more important victory than raw benchmark speed.

• Blackwell is almost the easier comparison. The much more provocative claim is that Jalapeño can already beat published results from Nvidia’s newer Vera Rubin generation on output-token throughput per megawatt.