AWS Introduces IRHX Liquid Cooling System for Nvidia AI GPUs

Amazon Web Services (AWS) has developed a custom liquid cooling system to address the thermal challenges posed by Nvidia’s next-generation AI GPUs. The In-Row Heat Exchanger (IRHX) is designed to efficiently cool high-performance GPUs used in training large AI models, which are known for their high power consumption and heat generation.

The IRHX system circulates chilled liquid close to the server rows, effectively removing heat from the densely packed GPUs. This innovative solution is suitable for both new data center applications and retrofitting existing ones, eliminating the need for a complete redesign. Dave Brown, Vice President of Compute and Machine Learning Services at AWS, highlighted that traditional cooling solutions were inadequate for their needs, as they would have wasted valuable data center floor space and used water inefficiently.

AWS has also introduced P6e instances featuring Nvidia’s GB200 NVL72, a supercomputing platform containing 72 Blackwell GPUs in one rack. These instances are designed to handle the computationally intensive tasks associated with large AI models and generative AI. By integrating the IRHX with the GB200 NVL72 system, AWS customers can access unmatched computing power at scale, enabling faster and more efficient training of AI models.

The development of the IRHX is part of AWS’s broader strategy to own more of its infrastructure. In recent years, AWS has invested heavily in developing its own chips, storage systems, and networking gear to power its cloud services. This approach allows Amazon to reduce its reliance on third-party suppliers and balance operational performance with cost. The IRHX launch further solidifies AWS’s position as a leader in cloud infrastructure innovation.

Other tech giants, such as Microsoft, Google, and Meta, are also investing in custom hardware and cooling systems tailored to AI workloads. However, AWS’s extensive global footprint of data centers and experience in building and deploying custom hardware at scale give it a significant advantage. The IRHX could enhance AWS’s AI-ready infrastructure, making it more efficient, sustainable, and scalable.