xAI now has 200000 GPUs at Memphis AI Supercomputer cluster

Generative AI company xAI has officially announced that its AI Supercomputer cluster is already operating with 200000 Nvidia GPUs to support the latest model – Grok 3.

The company is hosting this new computational power at its Memphis facility. Named Colossus, the cluster was up and running with 100k Nvidia Hopper GPUs in the beginning.

The cluster was built and operational in 122 days and running in 19 days after the first server deliveries. In December, the company announced an expansion to 200k GPUs but didn’t reveal the time.

Today at the Grok 3 unveiling, the xAI team shared details and confirmed that the next phase of cluster development was completed in just 92 days, adding another 100000 GPUs. This addition has led to a total of 200000 AI GPUs at the Colossus supercomputer cluster.

“We had another phase, which we haven’t talked about publically, as this is the first time we’re talking about this when we doubled the capacity of the data center,” said an xAI official during the livestream.

xAI Supercomputer cluster with 200000 GPUs (Source – xAI)

The latest computing power has helped the company to improve its reasoning capabilities and performance. It will also power the text-to-voice model for its voice assistant.

This data center uses the Nvidia Spectrum-X ethernet networking platform for hyperscale AI factories. It equips Spectrum SN5600 Ethernet switch with a port speed of up to 800Gb/s based on Spectrum-4 switch ASIC.

Elon Musk, founder and CEO of xAI has also shared plans to build another supercomputer cluster, which will support the next-gen model.