Nvidia's New AI Chip is Five Times More Powerful Than the H100; Aims to Become the Next Apple or Microsoft

Nvidia on Monday unveiled two exciting AI breakthroughs: a new AI chip with five times the computing power of the H100 chip, and a platform-level product to challenge Microsoft and Apple.

Nvidia announced that the new AI chip architecture is named Blackwell, and the first chip based on this architecture is called GB200. Blackwell uses TSMC's 4NP process and is the first GPU to adopt a multi-chip module design. With 208 billion transistors, it is more than twice as many as the previous generation chip, known as Hopper. GB200's computing power has been significantly upgraded, achieving 20 petaflops in AI performance, five times that of H100. This new product is expected to start shipping later this year.

"Hopper is fantastic, but we need bigger GPUs," Nvidia CEO Jensen Huang said at the conference.

The GB200 is sizable due to a combination of two B200 GPUs and one Grace CPU. However, thanks to a customized TSMC process, this integration is highly efficient, resulting in a remarkable surge in power. Additionally, the GB200 also features 192GB of HBM3E memory.

Like previous generations, new GB200s can be combined to create a server with substantial computational power. The GB200 NVLink 2 contains 72 Blackwell GPUs and other Nvidia parts designed to train AI models.

Nvidia suggests that the system can support a model with 27 trillion parameters, significantly more than the largest models at present, such as GPT-4, which contains approximately 1.7 trillion parameters. Larger models, more parameters, and increased data can unlock more AI capabilities.

Among the many organizations expected to adopt Blackwell are Amazon Web Services, Dell Technologies, Google, Meta, Microsoft, OpenAI, Oracle, Tesla, and xAI. Nvidia disclosed that Amazon Web Services plans to build a server cluster with 20,000 GB200 chips.

Nvidia did not reveal the price of the GB200 or the full system, but it is anticipated to be costlier than the previous generation and command stronger pricing power. Currently, Nvidia's H100 retails for between $25,000 and $40,000, while the entire system's cost can reach up to $200,000.

The announcement did not end there. Nvidia also introduced a software product called NIM, designed to help developers reduce deployment times from weeks to minutes.

Nvidia's new NIM software simplifies the AI model deployment process by packaging algorithmic, system, and runtime optimizations, and adding industry-standard APIs. This enables developers to integrate NIM into their existing applications and infrastructure without extensive customization or the need for specialized expertise.

Nvidia executives suggest that the company is evolving from purely a chip provider to more of a platform provider, akin to Microsoft or Apple, enabling other companies to build software on top of their technology. As Huang put it, "Blackwell"s not a chip, it"s the name of a platform."

"The commercial product for sale was the GPU, and the software was all intended to help people use the GPU in different ways," said Nvidia's VP of Enterprise, Manuvir Das. "Of course, we are still doing that. But what"s really changed is that we now have a substantial commercial software business."

Das indicated that Nvidia"s new software will streamline the process of running programs on any of Nvidia"s GPUs, even older ones that might be better suited to deploying, rather than building, AI.

More importantly, NIM will enable AI to run on laptops equipped with GPUs, instead of servers in the cloud, heralding promising future prospects for AI PC applications.