this post was submitted on 16 Mar 2025
969 points (98.6% liked)
Programmer Humor
22752 readers
580 users here now
Welcome to Programmer Humor!
This is a place where you can post jokes, memes, humor, etc. related to programming!
For sharing awful code theres also Programming Horror.
Rules
- Keep content in english
- No advertisements
- Posts must be related to programming or programmer topics
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
So, this means they either have a local copy on disk of whatever database they're querying, or they're dumping a remote db to disk at some point before/during/after their query, right?
Either way, I have just one question - why?
Edit: found the thread with a more in-depth explanation elsewhere in the thread: https://xcancel.com/DataRepublican/status/1900593377370087648#m
So yeah, she's apparently toting around an external hard drive with a copy of the "multiple terabytes" large US spending database, running queries against it, then dumping the 60k-row result set to CSV for further processing.
I'm still confused at what point the external drive overheats, even if she is doing all this in a "hot humid" hotel room that she can't run any fans I guess because her kids were asleep?
But like, all of that just adds more questions, and doesn't really answer the first one - why?
My one question would be "How?"
What the hell are you doing that your hard drives are overheating? How do you even know it's overheating as I'm like 90% certain hard drives (except NVMe if we're being liberal with the meaning of hard drive) don't even have temperature sensors?
The only conclusion I can come to is that everything he's saying is just bullshit.
Hard drives do get hot and need some cooling but not at 60k rows. Its either made up or their computer case is made of thermal cladding
You could query 60,000 rows on a low tier smart phone. Makes no sense at all.
Can we think of any device someone might have that would struggle with 60k? Certainly an ESP32 chip could handle it fine, so most IoT devices would work...
Right? There's no part of that xeet that makes any real sense coming from a "data engineer."
Terrifying, really.
Unless the database was designed by someone who only knows of data as that robot from Star Trek, most would be absolutely fine with 60k rows. I wouldn't be surprised if the machine they're using caches that much in RAM alone.
Imo if they can't max out their harddrive for at least 24 hours without it breaking, their computer was already broken. They just didn't know it yet.
Any reasonable SSD would just throttle if it was getting too hot, and I've never heard of a HDD overheating on its own, only if there's some external heat sources, like running it in a 60°C room
Hard Drives might do it if the enclosure is poorly designed (no ventilation), but I can't imagine a situation where it would overheat like that that quickly, even in a sealed box. 30k is nothing in database terms, and if their query was that heavy, it would bottleneck on the CPU, and barely heat the drive at all.