Sam Altman, CEO of OpenAI, recently made a comparison between how much energy had been consumed by humanity over the millennia and the energy consumption of artificial intelligence (AI) inference.
In an interview at the AI Summit in India, he suggested we consider the energy needed for a human to do an inference query. “It takes 20 years of life – and all of the food you eat during that time – before you get smart.”
The implication being that AI is a massive shortcut in the evolution of the human race, where a human in today’s society is able to make adult decisions.
But datacentres are power-hungry, driven by the need to supply more powerful AI. The International Energy Agency predicts that energy demand from datacentres will more than double by 2030, and electricity demand from AI-optimised datacentres is projected to more than quadruple by 2030.
This is having real-world consequences. Electricity markets operate differently around the world, but in the US, the power demands of datacentres and the grid upgrades they require are being directly blamed for price rises being endured by residential customers, according to a ConsumerAffairs analysis of the US Energy Information Administration’s (EIA) Electric power monthly report.
Energy consumption is one of the many reasons that communities are pushing back against datacentre developments. It is set to get worse as chip technology improves. The graphics processing units (GPUs) that AI model builders rely on are set to become more power-hungry. Nvidia’s roadmap assumes the 1MW rack is not far away, while the company is championing the transition from 48V or 54V DC at the rack to 800V DC power for datacentres.
While this transition may ultimately lead to more efficient power use, it also means a wider overhaul of datacentre infrastructure: more powerful GPUs will mean more storage, more networking and more cooling. All of these point to greater energy consumption, even as GPU efficiency increases.
So, where does this leave enterprises looking to build out their AI capabilities while not trashing their sustainability reputations or alienating their end customers?
Arguably, the biggest problem for IT and business leaders is general blowback against datacentres, rather than their own AI use, which, as Rabih Bashroush, professor of digital infrastructure at the University of East London, notes, is relatively low. “Enterprises don’t represent the largest workload for AI,” he says.
Nevertheless, the power demands of AI are shaping how infrastructure is being built out.
Powering what, exactly?
For Nscale, one of the European darlings of the neocloud operators, access to power is as important as access to the GPUs on which AI depends.
“That is the biggest constraint that we see,” Nscale’s chief revenue officer, Tom Burke, said during the Vast Forward event in February.
The company’s datacentre network is centred on Norway, whose cold climate and abundant hydroelectric power offer distinct advantages when running power-hungry, heat-generating AI infrastructure.
He noted that the power footprint of GPUs has driven broader infrastructural changes.
“We look at the heat transfer requirements of the chips. You saw a transition from air-cooled datacentres to liquid-cooled datacentres, and with that came what used to be two-year release cycles consolidated from Nvidia down to six-month release cycles because of how fast this innovation was coming,” he said.
This suggests that a combination of centralisation and engineering expertise from cloud providers is helping to drive down power demands.
But the University of East London’s Bashroush says centralised AI infrastructure is not the only game in town. “There is a lot of open source AI that companies are downloading and running internally. I know many companies are doing this. So, what’s the direction of travel?”
At the same time, enterprises are looking to use specialised models, which are much more efficient than the general-purpose models being offered by the likes of ChatGPT.
And the drive for data sovereignty will also shape demand, he says, as it further bolsters the case for distributed infrastructure and specialised models.
“I need to think twice before I tell you enterprises will be consuming a lot of cloud AI,” he adds.
IT hardware providers are shifting their product focus to meet demand for decentralisation. Karim Abou Zahab, principal for sustainable transformation at HPE, says: “Enterprises are increasingly looking at where AI runs and how efficiently it can be deployed closer to their data and operations.”
Existing edge locations will also have pre-existing power – the datacentre boom means getting a new grid connection, which means waiting in line for years.
But, says Zahab, that also means IT decision-makers must treat efficiency as a factor from the outset. “Software-driven optimisation is critical to ensure compute is fully utilised and energy isn’t wasted through idle or over-provisioned infrastructure.”
This means looking at the entire IT estate, he says: “The data fed into models, the software used to interact with and train them, the right equipment, datacentre resources, and the energy sources powering them.”
This might mean doubling down on the Nvidia ecosystem.
Speaking at Vast Forward, Vast cofounder Jeff Denworth highlighted the impact of Nvidia’s Bluefield 4 Smart NICs, which can carry Vast’s storage software platform.
“For every 1,100 GPUs, you don’t have to deploy another 256 physical Vast C node servers,” he told the audience. “So, your cost saving is off the charts. Your power saving is also quite considerable. We can reduce power for your infrastructure by about 75%.”
Alternatively, IT decision-makers may want to consider new models. James Sturrock, director of systems engineering at Nutanix, says workload optimisation is key, so companies need to be modernising infrastructure to reduce energy consumption and improving utilisation to avoid over-provisioning.
“For example, organisations adopting modern, software-defined infrastructure have reported energy reductions of around 50% compared to legacy environments,” he adds.
A turn-off?
But there are even simpler strategies for efficiency and optimisation when running smaller models using less data, away from the hyperscalers’ infrastructure, says Bashroush. “Once you run it this way, you’re switching it off when there’s no one in the office.”
But there are other ways to think about efficiency. As Bashroush notes, it has an impact on the workforce. AI has the potential to reduce headcount, which increases productivity and removes the need for the resources associated with supporting a larger workforce. He says: “Ultimately, in the enterprise space, the net of AI is very positive from an economic perspective.”
It’s also important to consider just what we mean by AI compute. HPE’s Zahab points out that the EIA estimated that in 2024, AI was still only responsible for 15% of datacentre energy demand. Most demand still comes from standard compute workloads.
That said, he says that inferencing energy use is set to outpace training. Inferencing, according to Zahab, is projected at roughly 162.5TWh versus 87.5TWh by 2030. For Zahab, this gives an opportunity – and a runway – to reduce costs and carbon footprints, if efficiency is prioritised from design through to deployment.
Of course, this all raises Jevons paradox. While English economist William Stanley in 1865 used Jevons paradox economic theory to explain why more, not less coal would be used as steam engines increased in efficiency, the more efficient AI infrastructure becomes, the more we’re likely to end up consuming. And that, again, raises the question of what we are really consuming.
As Bashroush says, enterprise AI workloads are a fraction of total cloud and datacentre workloads. It’s a truism that cat videos are the biggest consumer of datacentre resources. But what percentage of videos being uploaded to YouTube are based on AI?
“AI is doing a lot of good stuff. We are doing a lot of research. It’s expediting a lot of things, saving us a lot of time,” he says. “But in reality, what percentage of electricity is being spent on that stuff? A lot less than video, images and media.”
That means the same people who might complain about an AI datacentre development on their doorstep need to consider their own AI-fuelled media consumption and its impact on carbon emissions.
“How do we make it more transparent about the impact?” Likening the challenge to food labelling, he says: “We’re not forcing people to eat less sugar, but we are giving them the opportunity to make an informed decision.”