Today, we’re thrilled to announce our first submission of MLPerf® Training v5.0 results. As a peer-reviewed industry benchmark suite, MLPerf® Training by MLCommons® is one of the most trustworthy sources of data about AI cloud performance in the industry.

We achieved impressive results by training a Llama 3.1 405B model on 512 and 1,024 NVIDIA Hopper GPU clusters interconnected with NVIDIA Quantum-2 InfiniBand networking, which demonstrates our capabilities to deliver predictable training performance at large scale.

About MLPerf Training benchmarksAbout MLPerf Training benchmarks

For many AI and ML professionals today, AI cloud is an obvious and inseparable part of their production pipelines. At the same time, it remains a very complex and sophisticated system, where performance and reliability are hard to measure and compare. This creates a big challenge for potential customers of AI clouds when it comes to comparing options and making decisions about where to invest millions of dollars.

The MLPerf® benchmarks address this problem perfectly. Developed by industry and academic engineers, and open to community reviews, MLPerf® benchmarks create a credible measurement system that reveals ML model performance in realistic deployment scenarios across various workloads.

“We appreciate how MLCommons serves as a reliable compass for our industry, offering a clear framework to measure AI infrastructure performance. It helps us demonstrate our capabilities as a cloud provider, while also underscoring the rapid pace required to remain competitive in today’s AI landscape.”

— Narek Tatevosyan, Director of Product Management at Nebius

Large-scale training of the 405B modelLarge-scale training of the 405B model

We submitted results for training the Llama 3.1 405B model, one of the largest and most challenging-to-train models from the latest MLPerf® Training benchmark suite. We ran this benchmark on two multi-host clusters, built on the NVIDIA Hopper architecture and interconnected with NVIDIA Quantum-2 InfiniBand networking:

• 64-node cluster with 512 NVIDIA H200 GPUs

• 128-node cluster with 1,024 NVIDIA H200 GPUs

Even on the verge of proliferation of NVIDIA Blackwell platforms, NVIDIA HGX™ H200 platforms are an excellent choice for distributed model training — they deliver robust performance and exceptional cost-effectiveness at scale.

“We are thrilled to welcome Nebius as a first-time MLPerf Training submitter. We are particularly impressed by their achievement of training Llama 3.1 405B — the largest open-weight model in our benchmark suite — on substantial clusters of 512 and 1,024 GPUs.”

— David Kanter, Founder and Head of MLPerf at MLCommons

Nebius serves GPU clusters, accelerated by NVIDIA, to industry-leading AI labs and continuously improves our expertise in delivering compute capacity for massive ML training. This made us confident that we would be able to demonstrate solid performance results on clusters of this scale.

Achieving top-tier training performanceAchieving top-tier training performance

MLPerf® Training benchmarks measure training step time — the amount of time it takes to complete one training step — from loading a batch of data to updating the model’s weights. Shorter step time means more optimized compute infrastructure, which results in faster and less expensive training.
We achieved 124.5 min and 244.6 min training step time for Llama 3.1 405B on the 128-node cluster and 64-node cluster, respectively¹.

Figure 1. Training step time decreases by 1.97x when doubling from 512 to 1,024 GPUs