Selfhosted

Die

"Today, the Apocalypse happened. All that we love and hold dear came to an end. Blood in the streets worldwide, panic, hunger, devastation reigns supreme."

"What about my database server? Is it still up?"

More seriously though I have Meshtastic. I can just ask the public channel for anything I need

At some point, you gotta just accept that things are gone and start hunting for the next radroach to eat. I guess the corpo speak for this is acceptable losses and/or risk management.

In the most extreme cases, the final backup of my most important files are on my phone. With all the compromises we're forced to make there, I still refuse to buy one without an SD card slot, so I have swappable 1TB with me at all times. Importantly, it's also not the Source of Truth, so if it's lost I'm still recoverable, but if it's the last piece of electronics above sea level at least I still have that.

But for power management, I just have some UPSes that sustain a graceful shutdown and that's about it. If I'm on the lam, I would rather the 20TB of manga and anarchist zines be destroyed (read: crypto keys lost) than try to figure out how to carry it with me. Maybe the offsite backup strategy will finally get tested once I've established an alternate identity.

I'll go to the Winchester, have a nice cold pint, and wait for it all to blow over.

I actually wrote a blog about this a few months back. It was after a 12 day war (Israel+USA attacking Iran) and 40 day internet blackout, and then we got into another war (Israel+USA attacking Iran) and a 90-100 day(lost track) internet blackout.

It isn't exactly "how to survive the apocalypse" guide but it was a really helpful guide for myself and my friends and helped me keep working in those blackout days.

It's isn't focused on hardware, just software, since I'm a software engineer.

https://alavi.me/blog/we-need-apocalypse-proof-software/

I have a UPS that allows the server to survive an apocalypse of 15 minutes or less.

Servers are going to be the last thing I think about. I'll be busy trying to survive and help my loved ones. I suggest reevaluating your priorities immediately.

work with my community to build a better world from the ashes

Kinda just planning on dying in a ditch...

Abut 35+ years ago, I stuck a finger up and didn't like the way the wind was blowing. I decided to do something about it. While I am a prepper, I do not prep for EOTW scenarios. If we start dropping nukes, point me towards the blast cloud and let this universe recycle the energy it takes to keep this meat bag alive, into something else.

I do, however, prep for inclement weather, shortages, civil unrest, pandemics, etc. I have solar and whole house generators. I grow my own food, raise my own livestock, can and freeze my vegetables, meats, and such. During the pandemic, I rarely ventured off the compound as there was no real need to. I've long since turned my dining room into a pantry and it is well stocked and rotated. I stock medicinal supplies, things that would be needed in a disaster scenario, not gadetry. I have taught myself the skill of making very good alcohol, which can be used medicinally, and for barter. I stock a lot of staples, things that can be turned into multiple meals; flours, sugar, corn meal, etc.

I would say that my servers would be a minor issue or concern in a disaster scenario. I would most likely depend on Ham radio and CB communications, vs the internet. We would be back to living like say mine, or your, grandparents did. Very lean and close to the bone, relying on what we could scratch together to survive, such as Victory Gardens, etc.

We live in a world of convenience, and while that's great and all, we get used to the notion that we will always be able to go to the grocery to pick up food supplies, and that is a false comfort. For anyone interested, I'd start with extending your pantry. Make wise purchases. Don't fall for all the gizmos and gadgetry surrounding prepping. They'll sell you a sack full of crap you'll probably never use, or be useless when the time comes.

I'm planning on running towards the bright flash and hoping I end up part of the first wave of mass casualties....

What's your contingency plan for the apocalypse?

I will be one of the first to die. thought about moving, but honestly I would rather die than ~~survive~~ exist on this shithole after a nuclear war.

I have a drawer full of old cables.
Scsi, printers, scart, modem, I'm set!

Screw electronics. I'll finally get time to play my 100 board games, pen and paper roleplay games and all the stuff I currently don't do, because I'm doomscrolling all day. And I might have to ask the neighbour to bring their accordion and sing some Lady Gaga for me until Spotify comes back online. I think I'd be fine.

Just a word of caution, It'll be dark in the supermarket at that time. The electronic cash terminals cease to work and half the food is going to spoil within a few hours. So get some cash, rice, noodles, oil, ketchup and canned food. And you'll need some sort of water supply.

If power is down for good, then getting water is the main priority. If the pumpa don't run the water tower is losing pressure fast. I have 40 litres jugged up in the basement at all times for the first few days.

When that dries out my neighbour has a well that we've hooked up to five properties. Mostly for gardening, but it is potable. The pump needs power, though, so I'd pull an extension cord over to my caravan.

My caravan has 400W of solar, 300Ah LiFePO4 and a 1.5kW inverter. Also a meshtastic node with an antenna on the roof. That'll keep the food cold, and laptops charged. It can run a microwave or hotplate, too. I've got 20kg of propane if I need to conserve power.

Eventual goal - solar with battery backup for the house with isolation ability from the grid. Here in Aus you can have (1) solar tied to grid, (2) solar with batteries tied to grid, and (3) solar with batteries with a grid isolation switch. Only (3) allows you to power your house when the grid goes down.

If my place gets flooded then, due to the terrain, it's going to be a much bigger problem than data loss (even if it is all my family photos and videos). I think that will be the least of my concerns at that point. That said, I do have off-site backups and I'm also locally archiving to m-discs, so both the flood and EMP problem are not insurmountable in that respect.

Probably the one thing I do need to do is print out a lot of the more recent photos so I have hard copies of ones I want to keep.

Step 1. Get a lot of money

Step 2. Ignore the laws and get myself a super comfy RTG-powered hidden bunker with a Faraday cage

Step 3. Put my servers in there, take drives full of everything from the outside

Step 4. Live comfortably ever after

If i am lucky enough to avoid dying il die anyway within a few years. being dependent on medication that expires, sucks 😔😞

If this sort of event were to happen, I would have bigger things to worry about than my data. What use would it be anymore? What about drinking water and food? What about my ADHD meds? Shits about to get very twisted if I stop these meds. I fear for everyone else. In fact, if this sort of event was to happen, everyone should be worried about what happens when I'm not on my meds. Data be damned.

At first I was going to say, the 3 2 1 Backup rule won't stop the planet from being destroyed by a meteor. But then I remembered the data on Voyager1.

In that level of extreme disaster, honestly not going to be caring. But I do have a layered approach to less extreme more ~~realistic~~ likely scenarios.

Neighbors and Community

The most important thing in a real emergency. We know our neighbors, chat with them on the street and in line for the weekly ice cream truck. We have several close friends within an easy walk or bike ride and are part of a local social club that we go to every week. We’ve had the emergency chat with many of them.

Power

15 minute UPS on my NAS will get me through small power bumps. I also have a large backup battery meant for camping with solar panels that lets my partner and I go indefinitely without city power for our medical devices, with enough to spare most days to keep our phones topped off. I’m currently using it a a oversized UPS for my desktop, but in a real emergency I’ll shut that down and move it to the bedroom.

Longer term, we’re planning on getting solar+house scale battery. I had one before and it got us though multiple days without power as long as we were careful.

Food, water and general supplies

55 gallon food safe drum of drinking water with the tablets that keep it safe for years. I have a todo item that reminds me to rotate it out every three years. We have two emergency bins, one with a hand crank/solar/usb powered radio and flashlight and assorted emergency supplies. The other has freeze dried hiking meals. They were the cheapest per meal per year of shelf life last time I did the math.

Medications

A real gap. I can’t get more than a one month supply of my meds, similar for my partner. While neither of us have immediate life threatening problems without them, we’d both be in rough shape in different ways. Don’t know what to do about this.

Backups

My desktop, my partners laptop, the NAS, and my VPS all have offsite backups to another country halfway around the world. I test recovery annually, and use healthchecks.io to notify me if they stop doing their daily backup. I need to finish getting my laptop backup running, but it’s been low priority as I mostly use it as a thin client for my desktop and keep a few files synced with Syncthing.

VPS

A few critical services run on it instead of my at-home NAS in case our home internet connection fails. It’s physically located several hundred miles away. Again, backed up elsewhere so I can relatively quickly recover it if needed.

NAS

Hot-swappable 4-disk raid with a spare sitting in the closet. That should get me through most issues, with the offsite backups for things that don’t. It also pings healthchecks with a few daily self diagnostics.

RaspPi

Really just running PiHole, so the only data to back up is the split dns config which lives in my notes on my desktop. Seems like a weak point, but could be replaced by the NAS, router, or my laptop pretty quickly.

Mobile devices

Backed up to their corresponding corporate overlords, except for photos and videos which go to immich on the NAS. I wish I had a better solution here.

Me

I have a notes directory describing the setup with configuration, docker files and playbooks for the various services in a local git repo on my desktop. I have printouts of the assorted recovery codes and a letter explaining all this in my filing cabinet alongside my will and advanced directives. We have enough technical friends that my partner can ask one to help, or just point an LLM at the note files and have it walk them through most things. I’ve audited the notes and git history for credentials and it’s clean. Just IPs and machine names, lists of services on each, clean docker files and basic maintenance instructions.

I think my biggest gap is what to do in a dual-failure case where I lose my home internet connection, and my desktop ssd fails. My data would be safe in the offsite, but I wouldn’t be able to reinstall Debian. My laptop would let me take care of most things for a while, but maybe I need to set up a mirror…

If WW3 breaks out I am going to the nearest pharmacy and finding ways to get all the narcotics and going peacefully asleep, even though I do not live in the middle of Bum fuck nowhere I can see it from here. I wont get the pleasantries of a painless death of having a nuke dropped anywhere near me but I will suffer from the massive amount of radiation poisoning, all my data would be gone as is from the electromagnetic shock so yeah no worries from me.

Asides from my kiwix clone of Wikipedia, in the apocalypse there’s not much value to most of the things I self host.

I self host backups, code forge, some AI tools (all my AI chat and completion are local now).

But realistically, in an apocalypse situation I’m going to leave my suburban home and migrate somewhere safer and more directly connected to food+shelter, and probably spend my days dealing with trying to survive.

My self hosting is primarily designed around avoiding US based tools and systems, so that I have more control over privacy and don’t find policy I disagree with.

Generation enough to maybe keep the house from total death in winter for a week or so, but it won't be pretty.

A few months of non-perishable food.

Some spare parts on the shelf

Some offline backups

But reality is, if the apocalypse really happens, it'll look like the bronze age collapse on steroids, and we'd be lucky if only 90% of people died without our supply chains, especially once winter hits.

Say ww3 kicks off and power goes off - how are you keeping your servers up? Solar panels and batteries?

My servers run on a power station and then solar panels with a battery.

What if there’s a biblical flood and you dont have the means to build an arc? All your servers are destroyed beyond repair?

I have a boat.

What if you heard the Feds are coming to cart you and your servers away cos they suspect you of bad mouthing Emperor Tromp?

Don't live in the US. Servers are encrypted and backed up off-site.

What if theres a war and Luxembourg (you know, the enemy) let’s of an EMP pulse that kills your servers and all the infrastructure (power, internet…). How do you access all those cherished pics on Immich?

It takes A LOT for an EMP to fry electronics. The most important pictures can be printed. Already have a lot of albums. Some of my pictures is stored on discs.

In an apocalypse or an emergency, none of this stuff really matters though. You are better of making sure you have enough food, water and shelter. In almost any event, your servers are just going to sit there, waiting to be turned on again, when hopefully everything is back to "normal".

I would immediately try to set up mutual aid and start building small local communities. If there ever is a chance for true anarchism to take root, it's during a period where people will value mutual aid more than anything. This doesn't just include necessities like food and water but also entertainment.

In the grand scheme of things the only thing on my server stack that's really worth anything is immich. The rest will have very little value to anyone once I'm gone. Plan is to create printed books from the photos and those should stay accessible for the future generations, our archive just needs a ton of work on creating those photos and possibly adding descriptions on who's on the pictures and when they're taken.

I don't really plan for ww3 nor solar flare frying half of the planet, but one thing that's a real problem is that if something happens to myself. My wife or kids don't know how to manage/access a majority of the stuff there is even if their everyday digital life is using network and services in it I've built. They'll be just fine without pihole or jellyfin, but data in immich/nextcloud is valuable and bus factor for the digital environment is pretty low.

I should at least verify that all server passwords are on my bitwarden vault and set up dead mans switch on that. Then they can at least get someone to pull the data out of the systems or even hire someone to maintain them. Best option would be if one of the kids would learn the ropes, but so far it doesn't seem like they're interested on anything like that.

None of it will matter. That said, short term FUBAR can be mitigated. I have a semi-portable solar setup that currently runs my internet, NAS, Fridge and to some degree, the A/C.

To prep for the serious mess, I have to get Faraday bags for my server, drives, and solar generator and have enough time to bag everything before the EMP hits.

I have survival guides in book form as well as on the NAS, and I work the land enough to bring the soil into balance and build its fertility, plus feed the bees so they'll be there when I need them.

Kill a relative or two as a final favour. Check if the wife still wants to live, check if I can be bothered to carry on. If so I’d either head into the mountains or down to the coast (get a boat) and try and get away from population centres for a year.

Edit: Sorry, just seen “selfhosted”. I’ve got about a terabyte of stuff on sd cards - super packable. Those, a handset, a wind-up radio and solar phone charger.