xAI is training 7 models including Imagine V2 and a 10-trillion parameter mega model

xAI is making a lot of advancements in generative AI and Large models, as it’s now confirmed that the company is training 7 models, including Imagine V2 and a 10 trillion parameter variant.

CEO Elon Musk said xAI is training these models at the Colossus 2 AI supercomputer, which has more than 700,000 GPUs actively deploying training power to the corresponding models.

Training seven models simultaneously is rare and increases the cost of operations. Therefore, most of the labs only train one big model at a time, but xAI seems to be trying something new.

Here are the details

Imagine V2

xAI has a more advanced video generation model that will succeed with improved quality, realistic texture, and better instruction following. Though Musk hasn’t said anything about the improvement, these are the very likely reiterations in the next Imagine version.

2 Variant of 1T

Two different versions of a 1 trillion parameter model are; they’re similar in size but could have different training.

2 variants of 1.5T

Same here, it has two variants of 1.5 trillion parameters, which makes them more capable than the 1T models.

6T

This is something new to the AI world. Processing 6 trillion parameters at once is a humongous task, making it the largest model that could launch. Most of the models have 1-2 trillion parameters, and a 6T model could do some unimaginable things.

10T

This model is the biggest that any company has ever mentioned. This is frontier-scale, the largest that humanity has ever attempted to train. When asked about the training of this model, Musk said its pre-training phase is 2 months, which shows the speed and robustness of the xAI team is trying to scale things at its lab.

Following the pre-training, the model has to go through fine-tuning, safety testing, and more.

After revealing these 7 models, Musk mentioned that xAI has “some catching up to do”, which suggests that the company is trying to catch up with OpenAI, Google, and Anthropic, which are currently leading in the competition.