原文 · 未翻译
How the hell is Groq raising more money?
Somehow, Palpatine returned.
Axios just dropped a bizarre scoop. Groq, the AI chip company company that was acquired by Nvidia in December of last year, is raising $650M. How, exactly, is a company that successfully exited raising more capital?
Well, technically, Nvidia did not acquire Groq. They licensed Groq’s technology and hired Groq’s key technical executives, but did not acquire the Groq corporate entity. That corporate entity continued to operate, focusing on maintaining Groq’s datacenters and their inference API. That API focuses on offering extremely fast inference on smaller models; their largest supported model is GPT OSS 120B, which is likely at least 10x smaller than frontier models like GPT-5.5 or Claude Mythos. This is a technical limitation of Groq’s architecture; without large amounts of high-bandwidth memory (HBM) in each chip package, the total cost to build and maintain a Groq cluster capable of serving a frontier model would be massive. However, their all-SRAM strategy lets them serve small models much faster than a conventional HBM-based chip, delivering more tokens-per-second at the cost of lower tokens-per-dollar. For certain applications, this makes sense.
More importantly, Groq has four large datacenter deployments that are already set up to serve inference workloads at scale. Building out new datacenters has been a major challenge for startups and hyperscalars alike, so Groq’s datacenters represent a major strategic asset. However, it remains to be seen if the Groq name gives them a legitimate leg up, and or if their brand association with the LPU chip and a specific high-speed, high-cost inference strategy limits them.
Thanks for reading zach's tech blog! Subscribe for free to receive new posts and support my work.
Datacenters matter more than ever
As inference demand surges, existing datacenters are quickly reaching full utilization and companies are trying to build new ones as fast as possible. But between regulation, power issues, and expertise, even large companies with experience building out datacenters are experiencing major delays in their datacenter construction projects. And for venture investors, who primarily invest in startups, getting direct exposure to datacenter demand is very difficult; because datacenters are so hard to build, very few startups are actually doing so.
Groq already has four functional datacenters, as well as the talent to build and operate more. When Nvidia licensed Groq’s technology, Groq’s chip design team, compiler team, and software team joined Nvidia, but Groq’s datacenter team stayed to maintain GroqCloud inference services. This makes the remaining Groq company a unique asset -- a private inference datacenter operator with clear operational expertise, plus what is likely an extremely low valuation due to most of their differentiating technology being acquired by Nvidia.