8 months after NVIDIA’s non-exclusive Groq licensing arrangement, Groq technology finally is appearing inside a rack-scale NVIDIA product.

Groq racks will be online this year.

So NVIDIA is splitting agentic AI work across specialized processors instead of treating the GPU as the whole machine.

Rubin GPUs handle heavy model computation, Groq 3 LPX targets latency-sensitive token generation, and Vera CPUs will run code, tools, data processing and simulation around the model.

Agents make token-generation latency far more consequential than it is in ordinary chat because later steps often wait for earlier ones to finish.

Nvidia's claimed 4x responsiveness (Nvidia's Groq 3 LPX vs. Cerebras' inference platform) improvement can therefore compound across a long task rather than just make individual responses appear faster. One big reason inference hardware will be fragmenting by workload.