High-performance computing (HPC) sounds like spectacular applications, power and speed. But it is also about mundane things like sockets, transformer stations, cooling water and electricity bills running into the millions. (Okay, that is pretty spectacular again.) So what is actually going on, energy-wise, in a supercomputing centre – and how can we compute more energy-efficiently? An FAQ for everyone who is not particularly keen on launching data centres into space, as Captain Musk is planning to do.
Bettina Benesch
According to the International Energy Agency (IEA), data centres worldwide consume 415 TWh of electricity per year, which corresponds to around 1.5 per cent of total consumption.*
A conventional data centre typically requires about 10 to 25 megawatts (MW) of power, while a hyperscale, AI-focused facility can have a capacity of 100 MW or more and thus use as much electricity per year as around 100,000 households. The IEA expects data centre electricity consumption to triple by 2035, with a strong geographical concentration in the USA, China and the European Union.
The largest scientific HPC clusters in Austria are VSC-4, VSC-5 and MUSICA. As part of the EuroCC project, Austrian and European companies also run their workloads on them. These three clusters are managed by Austrian Scientific Computing (ASC); their average power consumption is between 1 and 1.5 MW.
Electricity consumption will also rise in Austria, as several international companies – including Google – are building large data centres. Grid capacity will need to be expanded for this. Whether this can be achieved in a climate-neutral way remains to be seen.
So it is sure that electricity consumption will increase – and that brings us to the next question:
Yes, in several ways: from faster, more accurate weather forecasts for wind and solar power, through optimised industrial processes, to smarter control of heating and cooling in buildings.
In industry, AI could speed up product development, reduce costs and improve quality, according to the International Energy Agency. Widespread adoption of existing AI applications to optimise processes in industry can lead to energy savings equivalent to more than the total energy consumption of Mexico today (approximately 353,5 TWh/year). The IEA has also calculated that AI can help to control heating and cooling in buildings intelligently, which could lead to electricity savings of up to 300 TWh. This corresponds to the annual electricity production of Australia and New Zealand combined.
Overall, HPC systems are becoming more efficient in terms of the ratio of performance to energy consumption; in other words, how many Flop/s per watt a system delivers. For the three major HPC clusters mentioned above, this breaks down as follows: VSC‑4 with 2.7 PFlop/s (Rmax), VSC‑5 with 3.31 PFlop/s (aggregated/accumulated Rmax performance) and MUSICA with 45.11 PFlop/s (aggregated/accumulated Rmax performance).
The first major Austrian supercomputer, VSC‑1, had a performance of 35 teraflop/s and drew around 400 kW of continuous power. Today, a single GPU in the new MUSICA system already offers more computing power at a fraction of that electricity consumption: While VSC‑1 required around 5.1 kW per TFlop/s (about 0.2 GFLOP/s per watt), MUSICA gets by with roughly 16 watts per TFlop/s, which corresponds to around 61 GFLOP/s per watt. In terms of compute performance per watt, MUSICA is therefore more than 300 times as efficient as VSC‑1.
The largest share of the electricity goes into the computing hardware itself. For the Austrian ASC HPC clusters, the ratio is 10:90: ten per cent of electricity demand goes into cooling, 90 per cent is available for computation.
When we read news about the huge data centres run by Amazon or Google, we often hear about rivers being used for cooling and warmed up by the data centres. In Austria, the approach is much more environmentally friendly, namely water cooling in a closed loop. In concrete terms, this looks like this:
At the ASC, ten per cent of electricity demand goes into cooling, while 90 per cent is available for computation.
At the ASC, the current systems VSC-4, VSC-5 and MUSICA are operated using mixed water cooling, the so-called direct liquid cooling. Copper pipes carry water (plus glycol) directly to the hottest components in the server – i.e. GPUs, CPUs, memory modules. The water is relatively warm at around 40 to 50 °C. Wherever heat is generated in the server, it is transferred directly into the coolant, carried away and pumped up to large dry coolers on the roof. There, fans blow outside air over the coils and cool the glycol solution down again. Around 80 per cent of the waste heat – in the case of MUSICA even around 90 per cent – is removed directly from the system via water. The remaining 10 to 20 per cent is still removed via air: fans draw in the warm server air and pass it through another system in which water at around 18 °C flows, bringing the air back down to room temperature. All of this happens in a closed system; there is no need to bring rivers into the story.
Air is a convenient cooling medium, but physically unfavourable: it has a low heat capacity, requires high volumetric flow rates and demands a lot of energy for fans and chillers. One advantage of water cooling is that the closer the water is to the heat source, and the higher its permitted temperature, the more efficient the cooling becomes.
Energy is already a limiting factor worldwide. At the ASC too, every purchase of a new cluster raises the question: Can we make this work? What needs to be rebuilt so that more transformers can be installed? How much electricity can be drawn at the site at all?
The global construction of huge data centres, which we are currently witnessing, calls for more energy sources. This is one reason why the phase-out of nuclear power is being delayed even in Europe – although Europe tends to focus more on producing electricity in a CO2-neutral way. In the USA things are different: there, it is not uncommon for data centre operators to use gas turbines to generate electricity. But whether Europe, the USA or China: energy is now a location factor, on a par with network topology and storage architecture. And in the coming years, we will continue to focus on computing performance – but the benchmark will increasingly be “Flop/s per watt” (Flop/W) rather than just “Flop/s”.
This is also why the Green500 list was created, which ranks the most energy-efficient HPC systems worldwide.
At the ASC, the reuse of waste heat is essentially planned, but not currently implemented. There have been concrete considerations to feed the heat into the district heating network and use it to heat the building. At present this fails due to a technical detail: the district heating network operates at significantly higher temperatures than the data centre can provide, namely at up to 120 °C. The slightly warm water would therefore have to be heated up again before it could be fed in, which makes little sense energetically. That it can work in principle is shown, among others, by the operators of the LUMI HPC system in Finland: there, waste heat is fed into campus or district heating networks and used to heat buildings.
Because three things have to fit together:
Firstly: the temperature level, as mentioned. HPC cooling water typically comes out of the racks at 40 to 50 °C, whereas classic district heating networks often run at 70, 90 or even up to 120 °C. To feed waste heat into these networks, you therefore need heat pumps or additional heating; otherwise the temperature does not match.
Secondly: consumers nearby. Waste heat can only be used economically if the heat consumption is local – for example in adjacent buildings, campus heating or connected neighbourhoods with a suitable network.
Thirdly: planning from the very beginning. If integration into district heating or campus networks is not planned from the outset, retrofitting later is expensive, because the existing flow temperatures are unsuitable and there are no pipes. Missing approvals and business models can also play a role. Austria and other countries therefore still need some time before waste heat from data centres can be used efficiently.
However, there are already successful international examples where data centres feed their waste heat into district heating networks. We already mentioned LUMI in Finland, but there is also Stockholm, for example, where over 20 data centres supply around 1.5 per cent of district heating demand and help reduce CO2 emissions. Studies by the International Energy Agency show that, in theory, data centres in Europe could supply up to ten per cent of nearby building heat demand with waste heat.
Very strongly. In principle, the following holds: the better a piece of software runs, the faster a job finishes and the less energy is consumed. By optimising the code, you save time, and whoever saves time saves electricity.
Typical efficiency pitfalls include:
Recommended countermeasures include:
Directly at the ASC’s own training centre, which also offers AI training. Most courses take place online; all include hands-on training on European HPC systems. As part of the EuroCC project, these training courses are free of charge, including for companies from EU and/or Euro HPC JU member countries. EuroHPC JU
Information on the trainings is available here.
+++
* Energy and AI, International Energy Agency, 10 April 2025: https://www.iea.org/reports/energy-and-ai