Microsoft is now submerging servers into liquid baths

Microsoft is starting to submerge its servers in liquid to improve their performance and energy efficiency. A rack of servers is now being used for production loads in what looks like a liquid bath. This immersion process has existed in the industry for a few years now, but Microsoft claims it’s “the first cloud provider that is running two-phase immersion cooling in a production environment.”

The cooling works by completely submerging server racks in a specially designed fluid. The fluorocarbon-based liquid works by removing heat as it directly hits components and the fluid reaches a lower boiling point (122 degrees Fahrenheit or 50 degrees Celsius) to condense and fall back into the bath as a raining liquid. This creates a closed-loop cooling system, reducing costs as no energy is needed to move the liquid around the tank, and no chiller is needed for the condenser either.

“It’s essentially a bath tub,” explains Christian Belady, vice president of Microsoft’s data center advanced development group, in an interview with The Verge. “The rack will lie down inside that bath tub, and what you’ll see is boiling just like you’d see boiling in your pot. The boiling in your pot is at 100 degrees Celsius, and in this case it’s at 50 degrees Celsius.”

This type of liquid cooling has been used by cryptominers in recent years to mine for bitcoin and other cryptocurrencies. This method inspired Microsoft to trial its use over the last few years, using it to test against spikes of cloud demand and intensive workloads for applications like machine learning.

Most data centers are air cooled right now, using outside air and cooling it by dropping it to temperatures below 35 degrees Celsius using evaporation. This is known as swamp cooling, but it uses a lot of water in the process. This new liquid bath technique is designed to reduce water usage. “It potentially will eliminate the need for water consumption in data centers, so that’s a really important thing for us,” says Belady. “It’s really all about driving less and lower impact for wherever we land.”

This tub of servers also allows Microsoft to more tightly pack hardware together, which should reduce the amount of space needed in the long term compared to traditional air cooling. Microsoft is trialing this initially with a small internal production workload, with plans to use it more broadly in the future. “It’s in a small data center, and we’re looking at one rack’s worth,” says Belady. “We have a whole phased approach, and our next phase is pretty soon with multiple racks.”

Microsoft is going to be mainly studying the reliability implications of this new cooling and what types of burst workloads it could even help with for cloud and AI demand. “We expect much better reliability. Our work with the Project Natick program a few years back really demonstrated the important of eliminating humidity and oxygen from an environment,” explains Belady.

Project Natick saw Microsoft sink an entire data center to the bottom of the Scottish sea, plunging 864 servers and 27.6 petabytes of storage into the water. The experiment was a success, and Microsoft had just one-eighth the failure rate of a land-based data center. “What we’re expecting with immersion is a similar trend, because the fluid displaces the oxygen and the humidity, and both of those create corrosion … and those are the things that create failure in our systems,” says Belady.

Part of this work is also related to Microsoft’s environmental pledge to tackle water scarcity. The company has committed to replenish even more water than it uses for its global operations by 2030. This includes Microsoft using an on-site rainwater collection system at its offices and collecting condensation from air conditioners to water plants. Nevertheless, Microsoft withdrew nearly 8 million cubic meters of water from municipal systems and other local sources in 2019, compared to a little over 7 million in 2018.

Microsoft’s effort to address its water usage will be extremely challenging given its trend toward more water usage, but projects like two-phase immersion will certainly help if it’s rolled out more broadly. “Our goal is to get to zero water usage,” says Belady. “That’s our metric, so that’s what we’re working towards.”