Next week in San Jose, Nvidia is expected to pull back the curtain on its most significant strategic move in years. The story begins last December, when the chip giant quietly committed a reported $20 billion to license technology from startup Groq and bring its CEO, Jonathan Ross, and key staff into the fold. Ross is no ordinary hire; he was instrumental in creating Google's Tensor Processing Units, the most credible alternative to Nvidia's own processors.

This deal, overshadowed by holiday markets and Nvidia's own relentless deal flow, is about to take center stage at the company's annual GTC conference. Analysts believe the Groq technology will be the foundation for a new Nvidia chip designed specifically for inference—the process of running AI models in daily use, which is distinct from the training phase where Nvidia's GPUs reign supreme.

Inference has become the next frontier. As AI adoption spreads, customers are looking for cost-effective ways to handle millions of daily queries, creating a crowded and competitive market. While Nvidia still claims a large share of inference work, it faces pressure from AMD's GPUs, custom chips from tech giants like Meta and Amazon, and Google's formidable TPUs.

Groq's approach is different. The company built its Language Processing Unit (LPU) from the ground up for speed in real-time inference, using a distinct memory architecture. In interviews before joining Nvidia, Ross argued that while GPUs excel at parallel processing for training, inference has a sequential nature where raw speed is paramount. He even suggested Groq's chips could act as a "nitro boost" for existing GPU deployments.

Nvidia CEO Jensen Huang has drawn a parallel to the company's acquisition of networking firm Mellanox, which transformed Nvidia from a chip vendor into a full-stack computing provider. That integration fueled explosive growth; networking revenue now hits about $11 billion per quarter. The industry will be watching closely to see if the Groq deal can replicate that success, beginning with the vision Huang shares next week.

Source: CNBC