Nebius monthly digest, August 2024
Nebius monthly digest, August 2024
We recently reduced the prices of NVIDIA H100, invited everyone interested to reserve H200, launched a public preview of Managed Service for MLflow and released a new talk about deploying generative AI models in production.
Price drop: Get NVIDIA H100 GPUs for less
One VM equipped with H100 is now priced at $3.50 per hour when paying as you go, down from the previous price of $4.85.
Therefore, the prices for VMs and clusters reserved for 3, 6 or 12 months are also down. For instance, you can get a cluster for 12 months at less than 3 dollars per H100 GPU hour.
Check out the new pricing and get your cluster right away.
This fall, we’re adding H200 SXM GPUs to Nebius
Currently, H200 is the most powerful GPU for your AI and HPC. Its key gains come from:
- Memory. H200 is the first GPU with 141 GB of HBM3e memory — nearly doubling the memory capacity of H100 SXM.
- Performance. The increased memory bandwidth of 4.8 TB/s allows for better utilization of processing power, making it the new preferred GPU for large models.
- Data access speed. With the shared filesystem, you can achieve up to 20 GB/s read speeds from one node, which is crucial for training and inference. This is 6 times more than what’s available with the H100.
According to an NVIDIA research, H200 shows up to 2x the LLM inference performance over H100: 1.4x in case of Llama2 13B, 1.6x with GPT-3 175B, and 1.9x with Llama2 70B.
Prices start from $2.50 per GPU hour. You can reserve these new powerhouse GPUs today.
Managed Service for MLflow is now in public preview
MLflow is a renowned industry tool that streamlines workflows in the model development cycle. We made MLflow more accessible to a broad audience of ML enthusiasts by providing it as a managed solution.
You can learn more or request access here.
Talk by Nebius: what it takes to train a GenAI model
Several weeks ago, our Product Director Narek Tatevosyan gave a talk at The AI Summit Series in London. He shared an insider’s perspective on the key steps, essential tools and challenges involved in bringing a generative AI model from concept to production. In this talk, he broke down the real story behind data preparation, experiments to prepare foundational model training, a pre-training process, fine-tuning and inference.
The latest on our blog
- Machine learning experiments: approaches and best practices. ML experiments help you discover the most optimum model version for your specific use case. Read the article to learn about different types of experiments and what you need to watch out for when conducting them.
- Choosing storage for deep learning. Drawing from Nebius’ and our clients’ extensive experience, this guide and research aims to help engineers choose the most fitting storage solutions.
- How transformers, RNNs and SSMs are more alike than you think. By uncovering surprising links between seemingly unrelated LLM architectures, we could open the door to effective idea exchange and greater efficiency.
Explore Nebius AI
See also
In July, we introduced our LLM R&D team and shared more of the team’s experience. We also launched a GPU auction and published diverse materials, including guest post about the London Institute’s research based on our infra.
Our June’s highlights included offering up to 8 GPUs via self-service, decoding GPU pricing and adding L40S to our portfolio. We also detailed our Terraform provider, discussed LLM dataprep, announced DVC.ai partnership and introduced feature requests.
In May, we published a guide on using DVC in Nebius, discussed Slurm vs K8s and fundamentals of LoRA, picked the best GPUs for CV, hosted a webinar on applying RAG — and welcomed our tech partner Weights & Biases with its Launch agent in Marketplace.