In a significant development for data centers, OpenAI and Broadcom have collaborated on a custom-designed chip, dubbed Jalapeño, optimized for large language model (LLM) inference at scale. This application-specific integrated circuit (ASIC) was crafted from the ground up, leveraging OpenAI's expertise and insights from their research teams. The result is a chip tailored to the unique demands of LLMs, addressing the limitations of existing data center infrastructure. With a development cycle of just nine months, the Jalapeño chip promises to deliver substantially better performance per watt than current state-of-the-art solutions, according to early testing by OpenAI. While the companies acknowledge that comprehensive performance measurements are still underway, the initial results are promising. The Jalapeño chip represents the first iteration in a long-term partnership between OpenAI and Broadcom, with plans for ongoing refinement and optimization. As the demand for efficient LLM inference continues to grow, the introduction of specialized chips like Jalapeño is expected to play a critical role in shaping the future of data center architecture and design.

Source: Ars Technica