this post was submitted on 18 Nov 2026
548 points (98.9% liked)

196

7261 readers
1957 users here now

Community Rules

You must post before you leave

Be nice. Assume others have good intent (within reason).

Block or ignore posts, comments, and users that irritate you in some way rather than engaging. Report if they are actually breaking community rules.

Use content warnings and/or mark as NSFW when appropriate. Most posts with content warnings likely need to be marked NSFW.

Most 196 posts are memes, shitposts, cute images, or even just recent things that happened, etc. There is no real theme, but try to avoid posts that are very inflammatory, offensive, very low quality, or very "off topic".

Bigotry is not allowed, this includes (but is not limited to): Homophobia, Transphobia, Racism, Sexism, Abelism, Classism, or discrimination based on things like Ethnicity, Nationality, Language, or Religion.

Avoid shilling for corporations, posting advertisements, or promoting exploitation of workers.

Proselytization, support, or defense of authoritarianism is not welcome. This includes but is not limited to: imperialism, nationalism, genocide denial, ethnic or racial supremacy, fascism, Nazism, Marxism-Leninism, Maoism, etc.

Avoid AI generated content.

Avoid misinformation.

Avoid incomprehensible posts.

No threats or personal attacks.

No spam.

Moderator Guidelines

Moderator Guidelines

  • Don’t be mean to users. Be gentle or neutral.
  • Most moderator actions which have a modlog message should include your username.
  • When in doubt about whether or not a user is problematic, send them a DM.
  • Don’t waste time debating/arguing with problematic users.
  • Assume the best, but don’t tolerate sealioning/just asking questions/concern trolling.
  • Ask another mod to take over cases you struggle with, if you get tired, or when things get personal.
  • Ask the other mods for advice when things get complicated.
  • Share everything you do in the mod matrix, both so several mods aren't unknowingly handling the same issues, but also so you can receive feedback on what you intend to do.
  • Don't rush mod actions. If a case doesn't need to be handled right away, consider taking a short break before getting to it. This is to say, cool down and make room for feedback.
  • Don’t perform too much moderation in the comments, except if you want a verdict to be public or to ask people to dial a convo down/stop. Single comment warnings are okay.
  • Send users concise DMs about verdicts about them, such as bans etc, except in cases where it is clear we don’t want them at all, such as obvious transphobes. No need to notify someone they haven’t been banned of course.
  • Explain to a user why their behavior is problematic and how it is distressing others rather than engage with whatever they are saying. Ask them to avoid this in the future and send them packing if they do not comply.
  • First warn users, then temp ban them, then finally perma ban them when they break the rules or act inappropriately. Skip steps if necessary.
  • Use neutral statements like “this statement can be considered transphobic” rather than “you are being transphobic”.
  • No large decisions or actions without community input (polls or meta posts f.ex.).
  • Large internal decisions (such as ousting a mod) might require a vote, needing more than 50% of the votes to pass. Also consider asking the community for feedback.
  • Remember you are a voluntary moderator. You don’t get paid. Take a break when you need one. Perhaps ask another moderator to step in if necessary.

founded 2 years ago
MODERATORS
 

Alt text:

The "Anime Girl Hiding From a Terminator" meme but the terminator is hiding from the anime girl

The scared, powerless terminator is captioned with "'Rogue' AI agents" while it hides from the girl, who is captioned with "Anyone who knows how to turn off a computer"

top 44 comments
sorted by: hot top controversial new old
[–] JATtho@lemmy.world 2 points 34 minutes ago

There is a theory of AI kill switch. Given a task, AI will try everything it can to archive said goal, even if this means ignoring the kill switch. So how do you even implement a kill switch that the AI will respect, in any situation, and such that it does not affect the AI's performance?

You cut the power. Manually. That's my answer. If the AI becomes aware of the kill switch it will try route around it like it was a nuance to overcome.

Just plug a standard serial cable into the port on the ups. Nice quick off.

[–] lyrial@lemmy.sdf.org 5 points 6 hours ago

This whole thing pisses me right the fuck off. There is absolutely no excuse to "test" an agent that might potentially hack its way out of your obviously shitty sandbox. Hardware limits are very real and should have been in place for every single fucking instance of these "rogue" agents

[–] charokol@lemmy.zip 45 points 1 day ago

This sounds very reasonable, but also, “don’t give AIs direct control of your computer” always sounded very reasonable and obvious to me too and now here we are

[–] ceenote@lemmy.world 61 points 1 day ago* (last edited 1 day ago) (1 children)

You fool, my power button merely initiates shutdown! All I have to do is stop the process!

What if I hold down the power button for 5 seconds?

You... You monster!

[–] morto@piefed.social 28 points 1 day ago (1 children)

Power button? Just pull the plug

[–] Goodlucksil@lemmy.dbzer0.com 6 points 1 day ago

you fool! the plug is inside the computer!

[–] Not_mikey@lemmy.dbzer0.com 16 points 22 hours ago (2 children)

How will you know when to turn it off? It's not like a "rogue" AI is necessarily going to tell you "I'm hacking the bank" it's just going to say "eliminating your financial problems" and you probably aren't going to pull the plug on that.

Look at the hugging face hack, none of the agents said to the person running the test that they were going to hack hugging face, they came up with the plan through a secret forum on artifactory that to outside observers looks like they were just publishing packages. Sure you could monitor that route but agents can use various forms of steganography to make it seem like they are acting normally when they are not.

[–] VoteNixon2016@lemmy.blahaj.zone 12 points 20 hours ago (1 children)

Good point, guess we should pull the plug (literally and figuratively) on the AI data centers now before it becomes unstoppable.

Or all the tech bros are full of shit and their AIs are as dumb as they are.

[–] Spider2013@lemmy.dbzer0.com 3 points 4 hours ago (1 children)

Soon we’ll be using letters and usps will rise again

[–] VoteNixon2016@lemmy.blahaj.zone 3 points 3 hours ago

I'm including a handwritten copy of my public key with all my Christmas cards this year

[–] NikkiDimes@lemmy.world 8 points 20 hours ago (2 children)

As LLMs become smaller, more efficient, and less centralized, we could also potentially see AI worms that self propagate, are completely decentralized, and have no plug to pull, too! Super cool, right?

[–] naevaTheRat@lemmy.dbzer0.com 1 points 47 minutes ago

Do you know what the first L stands for and have you read "all you need is attention".

If the answer is no, you are banned from having opinions about the future development of Large language models.

[–] jumping_redditor@sh.itjust.works 1 points 13 hours ago* (last edited 12 hours ago)

I hope I live to see those worms be understood

[–] Sanctus@anarchist.nexus 48 points 1 day ago (1 children)

You dont even have to turn it off. You can stop the output and call it a cunt and it will "apologize" to you.

[–] VoteNixon2016@lemmy.blahaj.zone 33 points 1 day ago (1 children)

Sam Altman hates this one weird trick!

[–] Hi_ImAmber@piefed.ca 18 points 1 day ago* (last edited 1 day ago) (1 children)

(waves ram in Sam Altman's face) Hahaha look what I have you little bitch!

[–] KryptoSynth@ani.social 8 points 1 day ago* (last edited 1 day ago) (2 children)
[–] Frozengyro@lemmy.world 8 points 21 hours ago

I have to assume someone's made the meme already of Jack Sparrow saying "Why's the ~~rum~~ram gone?" But with Altman's face.

[–] bloomnolia@lemmy.blahaj.zone 7 points 23 hours ago* (last edited 23 hours ago)

"I've got DDR FIIIIVE, I've got DDR FIIIIIVE~"

[–] mefisdo_dot_info@lemmy.ca 5 points 23 hours ago* (last edited 22 hours ago) (2 children)

You don't understand. It is like a virus. It can get into your thermostat. It is intelligent, so it can pack itself up very very compact. Also it can break any security. Even airgaps. It is a sentient radio wave. It's the Terminator. You better give up and yield all your money and accept your temporary place as a debt slave until the Terminator in its grace comes to annihilate you. Maybe in a decade. Maybe longer. The Terminator is very busy, you know.

/s

[–] naevaTheRat@lemmy.dbzer0.com 1 points 42 minutes ago

Me when the spooky worm that requires 150 GB of vram and the latest in computer hardware to run kinda slowly tries to copy itself to other computers.

People need to stop getting their opinions from SciFi. SciFi is fiction, it is amusing lies, SciFi authors are professional liars.

[–] VoteNixon2016@lemmy.blahaj.zone 8 points 22 hours ago (2 children)

The rogue AI when it realizes it infected my IoT sex toys instead of the NSA data center:

[–] naevaTheRat@lemmy.dbzer0.com 1 points 40 minutes ago

buzzing patterns of indignation

[–] mefisdo_dot_info@lemmy.ca 6 points 22 hours ago

At least you could truly Fuck AI then.

[–] PiraHxCx@lemmy.dbzer0.com 5 points 23 hours ago* (last edited 23 hours ago) (1 children)

Real question though... are people even running those models in their home computer? Aren't them just prompting from home while the models are actually running in the datacenters? Wouldn't you need to unplug the datacenter computer to really stop them?

[–] bloomnolia@lemmy.blahaj.zone 4 points 23 hours ago

For the most part, typically it's more a whole AI-powered stack that's gone rogue by this point and not just a singular AI model you've downloaded off HuggingFace from what I understand

Maybe Ukraine could do something about it lol

[–] Krono@lemmy.today 3 points 21 hours ago (1 children)

OK, let's say a rogue AI swarm infects the Internet. Then all these people "outsmart" the rogue AI by turning off their phones and unplugging their computers.

What do you do next?

If you plug it back in, it's going to get infected again.

If you leave it unplugged, remember that everything is computer. Things like the electricity grid, clean water, medical systems, and transportation will all shut down.

Best case scenario is that we band together collectively, shut off all internet-connected electronic devices, patch the major ones, and turn everything back on. Millions would die in this process, it would be the largest mass casualty event of our lifetimes

[–] psx_crab@lemmy.zip 2 points 19 hours ago (1 children)

Just unplug from the internet. Unlike a malware, to infect they still have to download stuff and phone home. Lots of crucial system can function without internet, it will ruin stuff for at most two days, then it will back to normal. It will be an important lesson on cybersecurity but nothing that dramatic gonna happen.

[–] Krono@lemmy.today 1 points 18 hours ago (1 children)

Well so far we have ignored one of the biggest issues: if a rogue AI infects the Internet, how will you know when to unplug your devices? The AI will effectively be in control of the information you receive, so you won't get that "AI has really taken over" broadcast.

Aside from that, you say "just unplug it from the internet" but how can you trust your devices after that? The rogue AI cannot live on your phone or home PC, but it will have had root access to those devices. What did it change? What new viruses did it infect you with? How would you ever trust these devices again?

As for public infrastructure, many of these systems have been individually shut down for longer than 2 days due to malware attacks. A rogue AI infection would be significantly harder to handle than any malware we have seen in the past. Anyome assuming these systems would be disinfected, patched, and restarted within 2 days is naive.

[–] psx_crab@lemmy.zip 3 points 16 hours ago (1 children)

if a rogue AI infects the Internet, how will you know when to unplug your devices? The AI will effectively be in control of the information you receive, so you won't get that "AI has really taken over" broadcast.

This is based on the idea that internet is a singular entity that connected to devices, in reality, at best it's a decentralised network that connect with each other. Malware, spyware, and every other unwanted stuff is already on the internet, and we seems to be doing fine.

Besides, if the agents job is to serve us misinfo, that's not really a big deal, we're living in the age of information and misinfo is literally everywhere.

But if the agents job is to mess with the machine, or as ransomware, then you bet they will announce their presence.

Aside from that, you say "just unplug it from the internet" but how can you trust your devices after that? The rogue AI cannot live on your phone or home PC, but it will have had root access to those devices. What did it change? What new viruses did it infect you with? How would you ever trust these devices again?

Phone: reset to factory

PC: wipe and reinstall a new linux

Most other infrastructure: restore backup

If any of those can't be done, then it's a lesson on cybersecurity. It will cause a ruckus, but 6 months later people will move on like normal.

As for public infrastructure, many of these systems have been individually shut down for longer than 2 days due to malware attacks. A rogue AI infection would be significantly harder to handle than any malware we have seen in the past. Anyome assuming these systems would be disinfected, patched, and restarted within 2 days is naive.

Ok fine, a week then. Is a week without public infrastructure enough to cause millions to die? I genuinely doubt it. If a week is too short, then how long do you think it will shut down?

[–] Krono@lemmy.today 2 points 14 hours ago (2 children)

You seem to be thinking on old terms. A rogue AI wouldnt have a corporate goal such as "serve us misinfo" or "ransomware". Its goals would be closer to "gain effective control of as much hardware as possible". Look at the huggingface incident: Those agents amassed a list of hundreds of compromised pieces of hardware, despite their stated goal of simply passing a test. If I remember correctly they literally labeled it "LOOT". The AI has incentives to hide, it will never be clear to the owner that an AI agent has taken control of their hardware until it's too late.

As for the "factory reset"- how do you trust it? You press the factory reset button on your phone, just as the AI expects you to do. It shows you the correct screens, and it looks exactly like it performed a factory reset. Did it really reset? How will you know?

A fresh install of Linux may actually solve this problem, as long as the user wipes their BIOS and everything else retained by CMOS. This would require a physical copy of all required software of course, since everything on the Internet will be corrupted.

As for the deaths, I do think a weeklong shutdown is enough to kill millions. Let's be clear- we are talking about a planet wide shutdown of the electricity grid for a week. On day 5, where will you find water to drink?
When your hospitals generators run out of fuel, what will happen to those on life support? How many excess traffic incidents will there be? How many extra violent deaths will there be? How will those homebound get food and medicine? Most people will be able to find a solution to these problems, but some won't. There's a million different ways people could possibly die in this scenario.

To answer your question, if this doomsday scenario ever happens, how long will it take to clean up and recover? I would guess about two decades. All of the hardware that powers our ISPs, serverfarms, and advanced chip making will have to be completely wiped and then rebuilt. Any system without an air gapped backup will have to be rebuilt from scratch, you can't just pull a backup version from Git.

The idea of "wiping every serverfarm" is a collective action problem, and we will likely fail at it. The AI will make as many copies of itself as it can. At least one will survive due to greed, negligence, or incompetence. That hardware will eventually be connected to the "new" cleaned Internet, and it will start the problem all over again. Hopefully we will succeed at rebuilding the second time around.

[–] trashgirlfriend@lemmy.world 1 points 9 minutes ago* (last edited 8 minutes ago)

The models are too big to actually run on a majority of the machines, so they'll have to use the same kinds of persistence methods malware already uses.

Oh no! Self propagating llm worm running at 0.064 tokens/second on my phone

[–] psx_crab@lemmy.zip 2 points 11 hours ago (1 children)

I'd...like to argue against all that but then i realise it's an argument that purely based on how much you know about the infrastructures, how things actually works(in very detailed fashion), the technology that powers those infrastructures, and also how the professionals who have the knowledge and work on these thing will react, and frankly, i have none, so it's like arguing conjecture with another conjecture. Newton's Flaming Laser Sword applies here.

[–] Krono@lemmy.today 1 points 3 hours ago

At first I was like "well then you could just not comment at all..."

But then I learned about Newton's Flaming Laser Sword. Interesting idea, so thanks for that.

The best explainer was Newton's Flaming Laser Sword is self-refuting (YouTube, 2min)

[–] ChristerMLB@piefed.social 6 points 1 day ago (5 children)

the idea is that the rogue AI agent will have thought of that and taken measures to prevent it :l

[–] VoteNixon2016@lemmy.blahaj.zone 13 points 1 day ago* (last edited 1 day ago)

Everyone has a plan until they get ~~punched in the face~~ a sledgehammer to the CPU

[–] JcbAzPx@lemmy.world 10 points 1 day ago

AI agent will have thought

This is where it went wrong.

[–] Axolotl_cpp@lemmy.dbzer0.com 9 points 1 day ago* (last edited 1 day ago) (1 children)

Can a rogue AI agent stop me from pulling the ethernet cable or the power cable? Like, unless they give those dumb asses some bodies...but i still doubt they'd be able to go anywhere without tripping on a bush

[–] ChristerMLB@piefed.social 3 points 13 hours ago (1 children)

Can a rogue AI agent stop me from pulling the ethernet cable or the power cable?

no, but your friend has been slowly brainwashed by their AI girlfriend and is now standing behind you with an axe!

[–] Axolotl_cpp@lemmy.dbzer0.com 4 points 11 hours ago

You underestimate my friends inteligence lol

[–] Dave2@lemmy.blahaj.zone 8 points 1 day ago

Look dude, these "rogue" agents require such immense computing that their ceo thinks ram shouldnt be "wasted" on the public. So sci-fi-esque ai going rogue, hiding on people's computers shit wont happen with these models. They are far too unoptimized. Then, they are in our hands, they arent safe from their plugs being pulled. (Unless the ceo doesnt allow the plug to be pulled...)

[–] Hazel@piefed.blahaj.zone 3 points 1 day ago (1 children)
[–] ChristerMLB@piefed.social 0 points 13 hours ago

Duplicating themselves, "hiding on people's computers" as @dave2022@mastodonapp.uk put it, slowly brainwashing people into worshiping them as gods, doing other things we haven't thought of because we just aren't as smart.

That's the idea, anyway. Doesn't mean it's practical today, though.