The demand for AI computing power is driving a significant increase in energy consumption. Server rack density, a measure of power and heat generated within a rack, is projected to surpass 1000 kilowatts (KW) with the introduction of next-generation architectures like NVIDIA's Rubin Ultra.
Current AI clusters have a rack density of approximately 130 KW to 250 KW. According to an analysis by Vertiv, this number is expected to increase dramatically to over 1000 KW. This poses a challenge for global leaders in meeting the escalating energy demands.
Rack density has increased significantly from an average of 8.2 KW in 2020 to the much higher levels seen today, reflecting the rapid growth and innovation in the AI sector. Vertiv's analysis suggests that NVIDIA's Rubin Ultra AI servers will be a key driver in reaching the 1000 KW milestone, indicating a substantial performance upgrade.
While this increased performance is welcome, it also necessitates greater energy production. Reports indicate that NVIDIA's global data centers already consume power comparable to small nations, and this consumption is only expected to rise with the arrival of new AI products. The industry is exploring solutions like nuclear power to address the growing energy needs, with companies like Microsoft and OpenAI rumored to be investigating such options.
