For AI, energy is nothing, and energy is everything
The serendipity of measuring data centers in watts
How serendipitous is it that data centers are measured in watts?
Everyone I know in the energy industry is talking about data centers. This isn’t surprising, given that data centers have quickly become the fastest growing source of electricity demand worldwide. Here’s a March 2024 forecast by SemiAnalysis, which has really done its homework in order to put this outlook together. (Seriously, while most data center forecasts are pretty iffy, these guys seem to have a rigorous methodology.1) This includes every class of data center, spanning hyperscalers & colocaters, cloud & AI.
So, global data center capacity is on track to grow by >3X within the next four years. Total energy consumption is also on track to roughly triple, growing to about 4.5% of global electricity demand.
What’s somewhat surprising is that everyone I know in the AI and data center world (admittedly, that’s way fewer people than I know in energy) is now also talking about energy.
Why is this surprising? Because energy is still a very small part of the COGS (cost of goods sold) for computing. Compare, for example, the cost of rent which data center tenants are currently paying for capacity in “Tier 1” US markets2 with the cost of power generation capacity in the country’s largest auction for it — in the PJM power market — alongside an estimate of the actual cost of new generation capacity in Texas. Data center capacity is waaay more expensive than power generation capacity.
It’s important to note that typical data center rent does not include the cost of any computing hardware — it pays for space, fiber connectivity, cooling, security, backup power, and a grid connection. High performance chips themselves can be even more expensive than all the rest of these elements combined. Hence, the cost of a fully-loaded, high-performance computing system completely dwarfs the cost of power supply infrastructure.
Here’s another perspective: leading AI research firm Epoch AI’s estimate of the cost breakdown for training “frontier” AI models. The cost of energy appears to be roughly on par with the cost of smoothies in the breakroom. It’s about 10-20% of the cost of NVIDIA’s magic chips.
So, energy is currently a trivial share of the total cost of computing — especially for AI workloads. Yet, access to energy is no longer a trivial concern. As I wrote back in February, the electric power sector is navigating a narrow “gauntlet” — an increasingly precarious path to keep up with surging demand amidst bottlenecks to infrastructure expansion. This challenge has now been widely publicized, prompting public questions about the social impacts of AI development, and about the ambitious carbon commitments trumpeted by many of AI’s leading proponents.
There’s an obvious opportunity here.
Because power is such a small portion of the overall cost of high performance computing, and because leading-edge AI developers are anticipating that their products will be extremely profitable, those developers ought to be willing to pay a substantial premium to secure the power they need. And, because these companies tend to be filled with people who want to do something about climate change, they appear to be willing to pay an especially high premium for low-carbon power.
Hence, we’ve seen the big three hyperscale cloud computing providers, which are also pushing the boundaries of AI — Microsoft, Amazon, and Google — making forays into nuclear power. See, for example:
Microsoft’s 20-year offtake agreement with Constellation to restart Three Mile Island Unit 1 by 2028.
Google’s agreement with Kairos to deploy 500 MW by 2035.
Amazon’s agreement with Energy Northwest to deploy X-energy's small modular reactors, and investment in X-energy's recent $500m fundraise.
Herein lies the opportunity for the power sector: Enlist the booming data center industry to help navigate the “electricity gauntlet” which the quest for superior AI is partially responsible for instigating. Nuclear is just one example of an emerging technology category which deep-pocketed AI investors could help advance by paying a premium, and possibly by taking on a share of the risk for the first few projects. I’ve been encouraged to see electric utilities developing programmatic ways to do this — for example, the “Accelerating Clean Energy” tariff framework which Duke Energy has put forward with support from all three of the hyperscalers mentioned above. Duke is one of the country’s largest utility companies, with a market cap of ~$87 billion. Why not ask a company like Google, whose market cap is $2.15 trillion, to shoulder some of the financial burden of the infrastructure deployment they’re asking to accelerate?3
In sum: The combination of an urgent demand signal and more attention from policymakers which the AI sector has brought to the energy sector are undoubtedly a source of pressure. But, it’s possible that these forces can also be harnessed as accelerants for the energy transition.
Interested in nuclear? Check out the second post in my series of Big Questions in energy & climate tech:
“When will we get serious about nuclear, or something nuclear-esque?”
From SemiAnalysis on their forecast methodology: “Our approach forecasts AI Datacenter demand and supply through our analysis of over 3,500 datacenters in North America across existing colocation and hyperscale datacenters, including construction progress forecasts for datacenters under development, and for the first time ever for a study of its type, we combine this database with AI Accelerator power demand derived from our AI Accelerator Model to estimate AI and non-AI Datacenter Critical IT Power demand and supply. We also combine this analysis with regional aggregate estimates for geographies outside North America (Asia Pacific, China, EMEA, Latin America) collated by Structure Research to provide a holistic global view of datacenter trends. We supplement the regional estimates by tracking individual clusters and build outs of note with satellite imagery and construction progress, for example the up to 1,000 MW development pipeline in Johor Bahru, Malaysia (primarily by Chinese firms) – just a few miles north of Singapore.”
Tier 1 — e.g. Nothern Virginia, Dallas, Phoenix.
On 11/18, Duke Energy’s market capitalization is $87 billion, while Google’s is $2.15 trillion.
Super article Andy. Interesting how AI will drive energy consumption but I also wonder about the eventual energy impact of quantum computing entering the space. My bigger point is while energy demand will drive investments in energy production, I remain concerned about the ability to get the energy from the point of production to the point of consumption. Innovations and investments will need to be made to maximize throughput on existing energy corridors. As EIP knows, this is an area ripe for growth to feed our nations desire for all uses of energy, not just computing. I'm glad that at least the "big data guys" are considering this but demand growth will include many other actors that may not have the same low energy cost hurdle as data centers.