Shouldn't your servers be sweating?

The concept of ‘sweaty servers’ is interesting in more ways than one and cannot easily be dismissed, especially with the justification that air conditioning units do work better when the room they are cooling is hotter. Thinking about it, it’s just as well that this is the case as datacenter managers are already grappling with the increased heat load dissipated by server racks and in turn, the increase in computing power per server. The datacenter is ‘naturally’ getting hotter.

However, if a server that is developed to handle 38 degrees Celsius must be made bigger, we have a challenge on our hands. The challenge lies in the fact that we’re faced with a lack of physical floor space in datacenters today. That is the reason that rack-optimised servers are so popular today. What’s more, calling for manufacturers to make servers that are more capable of running happily in temperatures in excess of 30 - or at the higher end 38 degrees Celsius - comes with its own risk but frankly I feel actually there is a bigger argument.
Most of the electricity that a server draws is converted into heat. The heat generated by each of the many individual circuits within the server are a direct reflection on energy efficiency. Simply put, the more efficient the circuit design the less heat is produced. Overheated parts typically have a shorter life-span and running systems at higher temperatures will have a direct impact on the availability of that server.

I fully understand the issue with noise and comfort in the server room, and also with the over-arching cost of running server hardware. I think the answer lies with more intelligent and efficient designs. Integrating multiple components for example can reduce power draw and therefore heat. Creating cooling zones within the server itself also impacts the energy a system consumes not just the energy used to heat it up but the energy used to cool it down too, and then there is of course the question of water cooling too. There certainly isn't a magic bullet, but we at IBM are really trying to take a very holistic approach to what is a serious challenge for our customers. We want to help our customers evaluate existing facilities, plan an energy-efficient data center, take control of power management and see significant energy savings. Thinking of making your server sweat? If you want my advice, proceed with caution!

Updated by 1000158494 on Aug 11, 2009 9:36 AM

Good points getting CPU'S running at ambient temps with passive cooling is the way forward, after all is that not that what IBM set out to do with the PS3 CPU, ok that one didn't go quite as planned but nether the less getting a CPU to run at half that at ambient temps is still an achievement.

Posted by CA on Aug 10, 2009 11:33 PM

I'm no electrical engineer but we clearly can make computers run in hostile environments.

I'm not sure how well today's average rack-mounted server would cope if it were forced to run at 30+ degrees but, given that a ground-up approach is taken to CPU design every three years or so, within five or six years it's not unreasonable to suppose that most data centres will be in a position to run ten degrees hotter than they do now.

Updated by manek on Aug 11, 2009 8:25 PM

I am supposed to be an electronic engineer, although it's a long time since I designed anything useful. However, during the time I was employed as a hardware engineer, it was building UPSs where the operating temperature of some of the components could be as high as 80 degrees C. Clearly you can run electronic components hotter than 20 degrees, but should you?

As has already been pointed out, running hotter reduces reliability. As has also been already pointed out, the answer to reducing the amount of cooling you need is to not generate as much heat in the first place, but that will happen as the technology improves.

One big win would be to agree a standard for water cooling sooner rather than later. This would solve many of the problems people have with datacentres — the ambient temperature and noise — and probably improve cooling efficiency into the bargain. All it takes is a specification for the connections, pressure and flow rate which, if central heating engineers can do it, I'm sure leading server manufacturers can.

Updated by Jonathan Bennett on Aug 12, 2009 12:11 PM

Hm mixing water with server's is not really a good idea i think because although it may work for desktop as in the form of self contained applicative use, this would be a costly fail for servers.

The reason why I state this is because these self contained water blocks are still to big to be deploying across blades with 8 or more sockets not to mention all the water tubing involved in amongst all them racks, and given the fact that the actual pumps and reservoirs are now consolidated together in an aid to reduce the sheer amount clutter of the older versions, means that the internal pumps will draw more voltage to sustain the pressures required to keep a strong flow of fluid throughout the now vast tubing required especially with regards to server CPU loads.

There's also the added weight to consider, when you start to think about these units could potentially add up to roughly the same weight as half a pound of suger sat on top of each of these sockets across the blades along with tubing spouting from them.

I think the CPU's manufactures have come along enough way with there implementations of Moore's Law to be able to give us another 10 C with passive heat sinks, the only other thing to do is to adopt a new way to deploy the stations.

For example instead of deploying the blades horizontally why not vertically? on top of this why not deploy an over head heat extractor per cluster of racks like you get over your kitchen cooker, but with one integrated powerful fan leading off into a air ducting system that can be either redirected back into the building and used as heating in the winter, and during the summer directed out of the building.

This type of approach would reduce the dependency we currently have on air conditioning and thus save energy costs along with energy used during the winter.

You might even be able to squeeze a few more blades into the over all rack chambers and still save on your costs as well as carbon foot print size.

Posted by CA on Aug 11, 2009 9:58 PM

The answer is thermionics...

It's certainly possible to design high temperature electronics.

"The maximum working temperature for semiconductors can be estimated from their intrinsic carrier density, which depends on the band-gap of the material. When the intrinsic density reaches the doping level of the devices, electrical parameters are expected to
change drastically. For the high-voltage regime (1000V), the theoretical limit for silicon is 150C; for discrete devices below 100V, it is about 250oC. Materials used up to 300oC include bulk silicon and silicon-on-insulator (SOI) technologies; for higher temperatures, gallium arsenide (GaAs), silicon carbide (SiC), and diamond show promise, and devices have been demonstrated at 500oC."

From http://ehw.jpl.nasa.gov/Documents/PDFs/publications%20pdf/CameraReadyKeymeulen.pdf , which has lots more!

Updated by Rupert Goodwins on Aug 13, 2009 9:34 AM

Thanks Rupe.

I suspect the equation here then is cost: there may well come a point where it's worth Intel/AMD et al building a chip for high temperature use when the capital costs of high-temperature servers, amortised over three or four years, fall below the operational costs of standard components, with their high demands for cooling - even taking into account the added space that may be required. I wouldn't be surprised if IBM's got a patent or two tucked away somewhere, waiting for the moment when the market's ready.

This is, of course, all predicated on capitalism's never-ending growth being set to continue as per situation ante...

Posted by manek on Aug 12, 2009 5:32 PM