Amazon Develops In-Row Heat Exchanger for Efficient AI Computing

Amazon's cloud computing division has developed specialized hardware to address the high energy consumption of next-generation NVIDIA GPUs, which are extensively used in AI-related computing tasks. These GPUs, while powerful, require additional cooling equipment due to their significant energy consumption. Amazon had initially considered building data centers equipped with liquid cooling systems to fully utilize these high-power NVIDIA GPUs. However, AWS's Vice President of Compute and Machine Learning Services, Dave Brown, noted that this process was too time-consuming and that available solutions did not meet their needs. Brown stated, "They either take up too much data center floor space or significantly increase water usage. While some of these solutions might work in small-scale scenarios for other service providers, they simply do not have enough liquid cooling capacity to support our scale."

In response, Amazon engineers designed and developed the "In-Row Heat Exchanger" (IRHX), a device that can be inserted into existing and new data centers. Previous generations of NVIDIA chips were sufficiently cooled using traditional air cooling methods. Brown mentioned that customers can now access this AWS service through a compute instance named P6e. These new systems are paired with high-density computing hardware designed by NVIDIA. The NVIDIA GB200 NVL72 installs 72 NVIDIA Blackwell GPUs in a single rack, working together to train and run large AI models.

Amazon has previously launched several self-developed infrastructure hardware solutions. The company has developed custom chips for general computing and AI, as well as its own storage servers and network routers. By running its own hardware, Amazon reduces its dependence on third-party suppliers, which helps improve profitability. In the first quarter, AWS achieved its best operating profit margin since 2014, contributing significantly to Amazon's net profit.