From Mainframe to Exascale Claudia Blaas-Schenner
From Mainframe to Exascale: The Exciting Future for HPC Enthusiasts
Claudia Blaas-Schenner, one of the leading experts in high-performance computing (HPC), takes us on a journey through 30 years of supercomputing at TU Wien: from the beginnings of numerically intensive computing to the current era of exascale systems and the rise of artificial intelligence (AI). Claudia also tells us how she thinks HPC will continue to develop and what opportunities there are for anyone looking to build a career in HPC and AI.
Interview by Bettina Benesch
Claudia, you have been working with supercomputers at TU Wien for 30 years. How did it all start?
I studied technical physics and then continued my research at TU Wien. When you want to calculate something in physics, you need a big computer. That’s how I got into high-performance computing (HPC). And the more you do something regularly, the better you get at it, so HPC gradually became my second specialisation.
Was there talk of HPC 30 years ago?
We mostly referred to it as numerically intensive computing, but I wrote my first parallel program in the summer of 1992.
What has changed since then?
Compared to today, the machines were much slower; the supercomputer back then was less powerful than my current mobile phone. The most important step was that the interconnects on the clusters became faster, meaning the connection between the individual parts of the computer. That was in the early 2000s. Since then, parallel computing has evolved massively, and the era of HPC began. A few years ago, fast GPUs (graphics cards) came onto the market, giving artificial intelligence a boost. A lot has happened in these 30 years. What used to take a week of computing time now takes less than a minute.
How do you think HPC will develop in the next ten years?
It will continue to evolve at a rapid pace. We have seen a lot of development in recent years, especially in Europe since the EuroHPC Joint Undertaking was established, with Austria also being a member. Europe now has three major pre-exascale systems: LUMI in Finland, LEONARDO in Italy, and MARENOSTRUM 5 in Spain.
“
What used to take a week of computing time now takes less than a minute.
„
All three systems are among the top ten on the TOP500 list of the world’s fastest supercomputers. This would have been unimaginable a few years ago when China, Japan, and the USA dominated the market. Today, Europe has three flagship HPC systems, and we will probably have our first exascale system in Europe in 2024, JUPITER in Jülich, Germany.
“
The hype with large language models (LLMs) comes from the fact that now we have enough resources to compute such models.
„
What role does artificial intelligence play in HPC?
AI and HPC will increasingly converge and benefit from each other; the interplay here is very complex. Artificial intelligence has been used in HPC for decades in simulations, especially in ensemble simulations where many different calculations run simultaneously, each with only one or two parameters different from the adjacent calculation.
For example, I can calculate a phase diagram of copper and nickel, which form an alloy. I have to do many calculations, each for a different concentration of copper and nickel. AI then helps me with the evaluation. It can also be used to monitor the ongoing operation of HPC systems and for error detection or efficient scheduling.
So AI has been around for a while. Is what we are experiencing now just a hype?
There is a difference compared to earlier: we live in the data age and have more and more data available. Not just the data we generate ourselves through simulations and then evaluate, but also data that is measured, for example, by companies.
The more data you have, the more important AI becomes. If you have three numbers, you can look at them as a human and know what they mean and what to do with them. If you have three million numbers, you look pretty desperately at your three million numbers. That’s when you need help, and that’s what you do with machine learning and AI.
In 2019, we had a deep learning hype. This was only possible because systems were available that could produce a good result in a reasonable time. This requires enormous computing power. And the hype we have seen since 2022 with large language models (LLM) also comes from the fact that the resources are now available to compute such LLMs.
HPC is a boost for AI, can we say that?
Yes, at the moment, HPC is a strong boost for AI and vice versa. We also see at our training events that interest in AI is increasing significantly.
You have been offering training at the VSC since 2015. What courses do you currently have in the programme?
What we are known for and where our good reputation comes from is parallel programming, such as MPI, OpenMP, or MPI+X. These were our first courses, and they are still our flagship. The top course is the intro course for the VSC, which every new user takes to get oriented. The course runs four or five times a year and is always fully booked. And for several years, we have, of course, had training that covers newer topics such as AI, deep learning, large language models, and – always fully booked, too – Python for HPC.
“
Our top courses are on parallel programming, introduction to VSC and Python for HPC. They run several times a year and are always fully booked.
„
HPC is a very broad domain. Is there something like a typical career in HPC?
There is not really a clear job profile in HPC, and many projects across Europe are working on defining it. It is a complex topic because it requires very different skills to work in the HPC field. On the one hand, there are experts who manage the HPC systems, which requires special skills in system administration. On the other hand, there are specialists who build a system from scratch, and others who cover the area of application engineering or programming. Then it also depends on whether it is a classic simulation you are doing or training AI and analysing data. So the jobs are very diverse, and you definitely need a team that combines the whole skillset.
The skills are different, you say – but are there qualities that people need who want to pursue a career in HPC?
It is certainly good if you like to try new things, are willing to dive into new topics, and are not afraid to make mistakes. Since it is teamwork, the key thing is being able to communicate well. Communication skills are essential.
“
Anyone aiming for a career in supercomputing should ideally pursue a PhD in a STEM field.
„
How many people in Austria have HPC skills?
There are maybe 100 to 500 who know HPC well; people with basic knowledge are about 5,000 to 10,000.
Will more HPC experts be needed in the future?
The market definitely needs many more experts, and they are on their way. Currently, less qualified people are being hired, and they learn on the job.
What educational path would you recommend to a young person who wants to go into HPC?
A master’s, or even better, a PhD in a STEM subject, like physics, chemistry, or mathematics, and an interest in HPC. Or possibly just an HPC master’s like the EUMaster4HPC. An internship at an HPC centre would be excellent, working there for six months to get practical experience. And of course, you have to love to learn new things.
Short bio
Claudia Blaas-Schenner studied Technical Physics at TU Wien, where she also earned her PhD and completed her postdoc. She started with supercomputer calculations in 1990 and has since delved deeper into the field. Today, Claudia Blaas-Schenner is one of the world's most recognised HPC experts. In 2015, she founded the training programme at Austria’s supercomputer, the VSC, and has been its director ever since.
She is one of the top experts in MPI and chair of the EuroMPI/Australia 2024 conference.
MPI stands for Message-Passing Interface, a standard without which parallel computing on clusters would be impossible. EuroMPI is an international conference with a very high reputation and a high standard of quality. Only about 50 percent of the paper submissions are accepted. The programme committee consists of representatives from Europe, the USA, Japan, and Australia.
Training at the Vienna Scientific Cluster (VSC)
Austria’s flagship supercomputer, the VSC, was put into operation in 2009. The first training events were held in 2015 and were fully booked immediately. Today, the VSC offers around 35 different courses annually, from the VSC intro course to MPI and C++ to Python for HPC and large language models. About 1,200 people attend the events each year. The courses are free for all VSC users and employees of Austrian universities or anyone in Europe as long as the course is co-financed by EuroCC. Only for courses not covered by EuroCC do participants from the industry and non-Austrian participants pay a small course fee.
The VSC training programme: events.vsc.ac.at
About the key concepts
Believe it or not, High-Performance Computing (HPC) is actually a relatively old concept: the word "supercomputing" was first used in 1929, and the first mainframe computers appeared in the 1950s. However, they had far less capacity than today's mobile phones. The technology really took off in the 1970s.
HPC systems are used whenever the personal computer's memory is too small, larger simulations are required that cannot be run on the personal system, or when what was previously calculated locally now needs to be calculated much more frequently.
The performance of supercomputers is measured in FLOPS (Floating Point Operations Per Second). In 1997, a supercomputer achieved 1.06 TeraFLOPS (1 TeraFLOPS = 10^12 FLOPS) for the first time; Austria's currently most powerful supercomputer, the VSC-5, reaches 2.31 PetaFLOPS or 2,310 TeraFLOPS (1 PetaFLOPS = 10^15 FLOPS). The era of exascale computers began in 2022, with performance measured in ExaFLOPS (1 ExaFLOPS = 10^18 FLOPS). An ExaFLOPS equals one quintillion floating-point operations per second.
As of June 2024, there were only two exascale systems in the TOP500 list of the world's best supercomputers: Frontier at Oak Ridge National Laboratory and Aurora at Argonne National Laboratory, both in the USA. In Europe, there are currently three pre-exascale computers, which are precursors to exascale systems. Two European exascale systems will be operational shortly.
VSC (Vienna Scientific Cluster) is Austria's supercomputer, co-financed by several Austrian universities. The computers are located at the TU Wien university in Vienna. From 2025, the newest supercomputer, MUSICA (Multi-Site Computer Austria), will be in use at locations in Vienna, Linz, and Innsbruck.
Researchers from the participating universities can use the VSC for their simulations, and under the EuroCC programme, companies also have easy and free access to computing time on Austria's supercomputer. Additionally, the VSC team is an important source of know-how: in numerous workshops, future HPC users, regardless of their level, learn everything about supercomputing, AI and big data.
EuroHPC Joint Undertaking is a public-private partnership of the European Union aimed at building a Europe-wide high-performance computing infrastructure and keeping it internationally competitive.
EuroCC is an initiative of EuroHPC.
Each participating country (EU plus some associated states) has established a national competence centre for supercomputing, big data and artificial intelligence – EuroCC Austria is one of them. They are part of the EuroCC project, which brings technology closer to future users and facilitates access to supercomputers. The goal of the project is to help industry, academia and private sector adopt and leverage HPC, AI and high-performance data analytics. EuroHPC also supports the EUMaster4HPC project, an educational programme for future HPC experts.
The EuroHPC JU launched EUMaster4HPC, a European master’s degree programme, as a pilot. The first students graduated in the summer of 2024. It is a joint project of various universities. TU Wien is involved as a contributing university: its staff provide expertise for the project, but no courses are held at TU Wien. The master’s programme lasts four semesters and requires students to be mobile, as they change universities after the first year.