Exascale wafer-scale supercomputer has 13.5m cores
Wafer-scale chip designer Cerebras Systems has developed an AI supercomputer with 13.5 million cores capable of exacale processing.
The Andromeda supercomputer consists of 16 Cerebras CS-2 systems and uses Cerebras MemoryX and SwarmX technologies to provide over 1 Exaflop of AI compute and 120 Petaflops of dense compute at 16-bit half precision.
The CS-2 system is a 15U rack with s 850,000 AI optimized compute cores, 40GB of on-chip SRAM, 20 PB/s memory bandwidth and 220Pb/s interconnect. Each rack consumes a peak power of 23kW, giving a total power usage of 368kW. These are fed by 18,176 3rd Gen AMD x86 EPYC processor cores in 284 chips.
The SwarmX fabric, which links the MemoryX parameter storage solution to the 16 CS-2s, provides more than 96.8 terabits of bandwidth.
Andromeda has 1.6 times as many cores as the largest supercomputer in the world, Frontier, which has 8.7 million cores.
The integration of hundreds of thousands of cores on a single wafer enables the AI supercomputer to demonstrate near-perfect linear scaling on large language model workloads relying on simple data parallelism alone, unlike supercomputers that use GPUs.
Near-perfect scaling means that as additional CS-2s are used, training time is reduced in near perfect proportion. This includes large language models with very large sequence lengths, a task that is impossible to achieve on GPUs.
One of Andromeda’s first users achieved near perfect scaling on the GPT-J AI large language model at 2.5 billion and 25 billion parameters with long sequence lengths.
Access to Andromeda is available now, and customers and academic researchers are already running real workloads and deriving value from the leading AI supercomputer’s extraordinary capabilities, including:
“AMD is investing in technology that will pave the way for pervasive AI, unlocking new efficiency and agility abilities for businesses. The combination of the Cerebras Andromeda AI supercomputer and a data pre-processing pipeline powered by AMD EPYC-powered servers, together will put more capacity in the hands of researchers and support faster and deeper AI capabilities,” said Kumaran Siva, corporate vice president, Software & Systems Business Development, AMD.
Andromeda can be used simultaneously by multiple users. Users can easily specify how many of Andromeda’s CS-2s they want to use within seconds. This means Andromeda can be used as a 16 CS-2 supercomputer cluster for a single user working on a single job, or 16 individual CS-2 systems for sixteen distinct users with sixteen distinct jobs, or any combination in between.
Andromeda is deployed in Santa Clara, California, in 16 racks at Colovore, a leading high performance data centre.
Related articles
- 14,336 ARM cores in chiplet-based waferscale AI engine
- 1400 RISC-V cores for on-chip machine learning
- Wafer-scale boost for optical photonics
Other articles on eeNews Europe
- AMD pushes CXL, chiplets with latest data centre processor
- Eliyan raises $40m to shake up chiplet interconnect
- IBM sees 2023 as quantum inflexion point
- Consortium forms Rapidus to get Japan back into chip race at 2nm
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
