OpenAI has collaborated with Broadcom to design and manufacture its first custom-built inference processor, named Jalapeño. This processor is specifically tailored to meet the unique needs of OpenAI's inference systems, with the company's own AI models contributing to its development. Initial testing indicates that Jalapeño achieves significantly better performance-per-watt compared to current state-of-the-art alternatives. By developing its own custom chip, OpenAI aims to reduce its dependence on Nvidia's GPUs, following in the footsteps of Google and Amazon, which have also built custom chips for AI acceleration. The Jalapeño chip is optimized for inference workloads, which involve running pre-built AI models in response to user commands. OpenAI highlights the chip's low operating cost when running real-time coding models, which could lead to improved economics for the company. While more performance-intensive tasks may still rely on Nvidia hardware, optimizing the inference system is likely to be a key factor in the economics of AI going forward. By designing its own chip architecture, OpenAI can further optimize its infrastructure, from kernels and memory systems to networking and deployment systems, to make its models faster, more reliable, and more affordable for users.

Source: TechCrunch