Shitposting

Okay, while I do suspect the 16oz number is pulled from someone's ass (or just very old, many efficiency improvements ago), there are a couple of things to consider:

a) A single B300 won't even run frontier models. It only has 288 GB of HBM. The DGX B300 server has 2.1 TB, which I suspect is not enough either (running Qwen 3.8 Max lossless (BF16) requires 4.9 TB just to load the model). But these are run in clusters anyway and any single user's prompt only uses a tiny percentage of the compute from tons of servers. OpenAI and Anthropic are going for size with their models so the most expensive ones probably need even more than Qwen.

b) While the GPUs themselves are directly cooled by closed loops, the water inside those closed loops is often cooled by evaporative coolers, or even AC and evaporative cooling in tandem (that allows the evaporation to happen outside the datacenter itself). That's because evaporative cooling is the most energy efficient form of cooling.

There are companies offering very complex heat systems that take advantage of multiple forms of cooling because they're making 100+ kW racks these days and talking of 500-1000 kW racks in the future. Ridiculous numbers, honestly.

This is from some company's website that offers cooling solutions: Notice the outside "cooling tower" - those can be dry or wet. In a hot and dry climate, you'd want wet. It'll cool really well. In a colder and more humid climate, dry might work better. They also optionally let you heat the aquifer apparently, which is bound to cause issues for everyone living nearby over time.