Huawei has introduced a new computer architecture it says could eventually let up to one million AI chips operate as a single machine. The company presented Peerium at its annual CONNECT conference on Sept. 17.

For scale, SpaceXAI describes its Colossus cluster as the world's most powerful AI training system, with 200,000 interconnected chips. Huawei says computers built on Peerium could train models with up to 40 trillion parameters, while competing hardware such as NVIDIA's GB200 NVL72 and Cerebras' CS-3 have reported training at the trillion-parameter scale.

HiSilicon chief scientist Liao Heng told the Associated Press that six or seven frontier AI labs in China are expected over the next two years, all aiming at foundation models of 10 to 40 trillion parameters, requiring infrastructure at that scale.

Peerium rests on three elements, according to Huawei: nested parallelism, which splits workloads across hierarchical groups of chips; unified memory addressing, letting processors reach memory on other servers; and peer-to-peer interconnects for more direct communication between components.

Huawei argues existing interconnect technology cannot support the architecture at its intended scale. Rotating chairman Eric Xu told the AP that NVIDIA's setup offers 200GB per second of inter-rack bandwidth, which he said cannot make one million processors work as one computer. Huawei developed its own interconnect, UnifiedBus, which it says delivers up to 800GB per second between racks.

Huawei also announced the Atlas 950 SuperPoD, built on Peerium and powered by more than 8,000 Ascend 950DT AI accelerator chips. Xu said AI training tests are underway on these systems and show more efficient use of available computing power than clusters not based on Peerium and UnifiedBus. Huawei expects to supply the SuperPoDs by early next year at the latest, with SuperClusters linking multiple SuperPoDs to follow; one SuperCluster with 256,000 computing cards is currently being deployed and tested.

Source: Llive Science