Traditionally, the AI chips used to train foundation models are housed in separate servers and communicate over a network to share data and divide workloads. But the new architecture, Peerium, uses a high-speed interconnect system designed to maintain efficient communication between chips across multiple racks, even as the number of chips scales into the hundreds of thousands.
Representatives from Huawei said in a statement that computers powered by Peerium could train models with up to 40 trillion parameters. For comparison, competing hardware, like GB200 NVL72 from NVIDIA and CS-3 from Cerebras, have reported training at the trillion-parameter scale, rather than the tens-of-trillions.
Peerium's performance relies on three elements: nested parallelism, unified memory addressing, and peer-to-peer interconnects, Huawei representatives said in the statement.
All three principles are designed to make more efficient use of computing resources so that a single system can coordinate larger numbers of chips to handle demanding AI workloads.
"With such a low inter-rack communications bandwidth, there is no way you can make one million processors work as one computer," he said.
"When one million processors work as one computer, this lays the foundation for better and more efficient AI training and inference," Xu added. "Therefore, I see UnifiedBus as the path for the future AI era. I believe, in the years to come, other players would certainly follow suit."
Stitching together a computing network
Related stories
Scientists build specialist 'AGI processor' that they believe will power the next wave of AI agentsScientists say they've eliminated a major AI bottleneck — now they can process calculations 'at the speed of light''Beyond human intuition': AI designs chip components 500 times smaller than what engineers could ever imagine
Huawei expects to begin supplying the SuperPoDs by early next year, at the latest, before its SuperClusters, which link multiple SuperPoDs to form a larger AI computing system, based on Peerium, eventually come online. Representatives said that one SuperCluster with 256,000 computing cards — hardware modules that add processing capacity — is currently being deployed and tested.
Models with large numbers of parameters demand greater computing resources and infrastructure, which isn't always desirable as it increases costs. In the summer, researchers used an IBM quantum computer to reduce the perplexity of Meta's Llama 3.1 8B language model — a measure of how well it predicts the next word or token — while keeping the number of parameters almost unchanged.
Hence then, the article about huawei unveils supercluster architecture designed to make 1 million ai chips work as a single computer was published today ( ) and is available on Live Science ( Middle East ) The editorial team at PressBee has edited and verified it, and it may have been modified, fully republished, or quoted. You can read and follow the updates of this news or article from its original source.
Read More Details
Finally We wish PressBee provided you with enough information of ( Huawei unveils 'SuperCluster' architecture designed to make 1 million AI chips work as a single computer )
Also on site :
- ‘Boy Meets World’ Star Rider Strong Reveals He Got ‘Completely Screwed’ During The Show’s Final Season
- Walmart’s Fluted Cabinet Brings Mid-Century Modern Charm and Tons of Storage to Small Bathrooms, Now 50% Off
- Is ‘I Play Rocky’ Original or Adapted? Stallone’s ‘Rocky’ Origin Story Faces a ‘Barbie’-Style Oscar Dilemma (EXCLUSIVE)