Supercomputer cools off with groundwater

Glen Paul: G’day, and welcome to CSIROpod. I’m Glen Paul. As you’ve probably noticed computers generate a lot of heat, which is why the little fan keeps spinning away in your PC, and your legs get hot when you rest a laptop on them for an extended period. A supercomputer, with its thousands of packed together processes at work, generates heat at a far greater level, requiring technology such as liquid cooling to keep it operational. This is how Australia’s largest supercomputer, the Pawsey Centre supercomputer, currently under construction in Kensington, South Perth, will be cooled, but with one significant difference – the process will be geothermal, meaning groundwater will be pumped into the cooling system.

Joining me on the phone to discuss the geothermal project is UWA Professor working at CSIRO, Professor Klaus Regenauer-Lieb. Klaus, just how hot do these supercomputers get?

Prof. Regenauer-Lieb: Well the actual chips get very hot. They can vary in their tolerances. I had a supercomputer that was actually switching off at 45°. Modern computers can tolerate much higher temperatures, and indeed there is an idea, and already a prototype, where the heat of a supercomputer can be used to heat houses to beneficial use. But this is just prototype, so at the moment we’re still having to reject the water.

Glen Paul: OK, so you’ve pipes running through the system, are they large pipes or small pipes that get right in amongst the processors and so on?

Prof. Regenauer-Lieb: Well there are two different systems. One system is basically a system where a dielectric fluid is surrounding the whole board, not just the chips, the whole board, and the heat is basically being carried away through convection in that fluid, and then the heat outside is exchanged with another heat exchanger, so that’s a system that is on trial. But most supercomputers are basically just putting small pipes of water over the chips, hopefully sealed, and then the heat is then brought to another heat exchanger where, in our case for instance, we are just rejecting 30° heat, so we push in 20° cold water and the other end of the pipe there is 30° coming out, so it has to be cooled back to 20°.

Glen Paul: So you can’t just put a big fan in the room and blast the computer with cold air?

Prof. Regenauer-Lieb: The problem with new generation supercomputers is that they get more and more dense, and they create more heat in very small areas, so one rack produces for instance now 63 kilowatts of power and heat output, and that is actually too hot for cooling with air, so you’d have to physically put a tornado through, so that’s why supercomputers now change to a good old technique that was used way back, and that’s water cooling.

Glen Paul: How do you cool the water down that comes out of the ground? Obviously it’s going to be warm, do you use a cooling tower?

Prof. Regenauer-Lieb: The classical solution for this 20° would be either a cooling tower or a refrigerant cooling system that actually maintains that temperature. So each supercomputer has its own temperature that it wants to reject heat at, it starts at 20°, so they seem to be moving to higher and higher temperatures, as I said up to 60° already, but at the moment we’re still living with 20° as a requisite.

Now the good thing is that the ground underneath our feet here in Australia has an average temperature over the year of 20°, and that average temperature is reached at 15 metres depth, so we could use the ground as a very efficient sort of battery, a thermal battery.

Glen Paul: Now I understand that in the past only chilled water could be used for this process. Why has this changed?

Prof. Regenauer-Lieb: Ah, that’s basically because the chilled water was required to provide cooling for air, so the chilled water needs to have around about a temperature of 7°, this was then used to basically cool air and the air was funnelled through the supercomputer, so it’s just an air cooled supercomputer. And the 7° has to be taken because of two facts, one is basically if there’s moisture in the air you need to go below the dew point so all the moisture will condense on the chilled water fins, so you will actually not have moisture on the chips. That’s one component. The other one is basically air isn’t a very efficient coolant, so you need a much colder environment to actually transfer the heat.

Glen Paul: Well I imagine getting water into your A$80 million supercomputer wouldn’t make for a good day.

Prof. Regenauer-Lieb: Yes.

Glen Paul: How much water then will this system save on the regular water supply, if you just had to draw the cooling water out of the tap?

Prof. Regenauer-Lieb: So the cooling water out of a tap would normally go through a cooling tower, that cooling tower would basically reject waste water into the sewage system, and also evaporate water. So you can’t recycle the water through the cooling tower forever, and you do chemical treatment of that cooling tower as well to avoid legionella diseases.

So this system replaces the cooling tower, and it saves 38 megalitres, so 38.5 million litres of water per year, which is on the order of 15 Olympic size swimming pools that is actually otherwise wasted.

Glen Paul: Fantastic. And the computer is being built to support the world’s largest ever radio telescope, the Square Kilometre Array, how powerful will the computer be, and when is it expected to come online?

Prof. Regenauer-Lieb: It’s actually almost coming online as we speak, so we will have the first celebration I guess in the middle of the year, so very soon the first equipment will arrive then. The building is finished, it looks really nice, and all the cooling works are going to come online very soon as well, so by October/November we’ll expect the first benchmarks of the supercomputer, we’ll have to first install all the software and make sure that everything works, then we’ll be coming online.

But recall this is only a 1 per cent solution of the full Square Kilometre Array. This is only the servicing the 1 per cent solution, so what you can expect is a much bigger supercomputer in the future when the full Square Kilometre Array comes online.

Glen Paul: Wow, and how powerful is this one in supercomputer terms?

Prof. Regenauer-Lieb: Oh, it’s actually going to be amongst the top five in the world, so it’s on the order of 100,000 cores, so it is not the top supercomputer in the world, but it is actually quite powerful. It is actually a computer that puts Australia from being behind New Zealand in terms of supercomputer power to way ahead of New Zealand, so amongst the world leaders in supercomputing.

Glen Paul: We can’t have that (chuckles), being behind New Zealand.

Prof. Regenauer-Lieb: Well this is only due to The Lord of the Rings of course. I mean all the supercomputers that have congregated in New Zealand were basically made for moviemaking.

Glen Paul: Well they are good films, but maybe our computer will prove to be more “precious”. Thank you very much for chatting with me today, Klaus.

Prof. Regenauer-Lieb: It’s my pleasure.

Glen Paul: Professor Klaus Regenauer-Lieb. And to find out more about the research, or to follow us on other social media, just visit www.csiro.au.