Man, AI agents are remarkably bad at "self-awareness" like this, I've used it to configure some networking on a Raspberry Pi, and found myself reminding it frequently, "hey buddy, maybe don't lock us out of connecting to this thing over the network, I really don't want to have to wipe the thing because it's running a headless OS".

It's a perfect example of the kind of thing that "walk or drive to wash your car?" captures. I need you to realize some non-explicit context and make some basic logical inferences before you can be even remotely trusted to do anything important without very close expert supervision, a degree of supervision that almost makes it totally worthless for that kind of task because the expert could just do it instead.

[–] Confused_Emus@lemmy.dbzer0.com 21 points 2 days ago (2 children)

AI agents are remarkably bad at "self-awareness"

Because today’s “AIs” are glorified T9 predictive text machines. They don’t have “self-awareness.”

[–] JackbyDev@programming.dev 13 points 2 days ago

Finally, T10

[–] definitemaybe@lemmy.ca 15 points 2 days ago (1 children)

I think "contextual awareness" would fit better, and AI Believers preach that it's great already. Any errors in LLM output are because the prompt wasn't fondled enough/correctly, not because of any fundamental incapacity in word prediction machines completing logical reasoning tasks. Or something.

[–] JackbyDev@programming.dev 9 points 2 days ago

Ah, of course. The model isn't wrong, it's the input that's wrong. Yes, yes. Please give me investment money now.

[–] Earthman_Jim@lemmy.zip 27 points 2 days ago (2 children)

Hey, maybe if we're lucky, Claude will accidentally lock the world out of using nukes forever.

[–] r1veRRR@feddit.org 3 points 1 day ago

Hey, Claude isn't actually used there anymore. Anthropic had some insanely basic red lines, and the Department of War wasn't a fan. All the murdering efforts are now done by OpenAI.

[–] TwilitSky@lemmy.world 6 points 2 days ago (2 children)

Or, more likely, Claude will launch them.

[–] DeadDigger@lemmy.zip 12 points 2 days ago

Would be funny if it also forgets to open the hatches

[–] fartographer@lemmy.world 2 points 2 days ago

Not if I launch them first! Where's the 9v battery for the rocket engine igniter?

[–] qjkxbmwvz@startrek.website 4 points 2 days ago

"...I really don't want to have to wipe the thing because it's running a headless OS"

I feel like logging in as root on a headless system and hoping you type the command(s) to restore functionality is a rite of passage.

[–] A_norny_mousse@piefed.zip 4 points 2 days ago (1 children)

AI agents are remarkably bad at “self-awareness”

🤔 what does it say when you tell it something like "look, this is wrong, and this is why, can you please fix that"? In a general sense, not going into technical aspects like what OOP is describing.

[–] Hazzard@lemmy.zip 4 points 2 days ago (1 children)

It's usually pretty good about that, very apologetic (which is annoying), and usually does a good job taking it into account, although it sometimes needs reminders as that "context" gets lost in later messages.

I'll give some examples. In that same networking session, it disabled some security feature, to test if it was related. It never remembered to turn that back on until I specifically asked it to re-enable "that thing you disabled earlier". To which it responds something like "Of course, you're right! Let's do that now!". So, helpful tone, "knew" how to do it, but needed human oversight or it would have "forgotten" entirely.

Same tone when I'd tell it something like "stop starting all your commands with SSH, I'm in an SSH session already." Something like "of course, that makes sense, I'll stop appending SSH immediately". And that sticks, I assume because it sees itself not using SSH in its own messages, thereby "reminding" itself.

Its usual tone is always overly apologetic, flattering, etc. For example, if I tell it bluntly I'm not giving my security credentials to an LLM, it'll always say something along the lines of "great idea! That's a good security practice", despite directly suggesting the opposite moments prior. Of course, as we've seen with lots of examples, it will take that tone even if actually can't do what you're asking, such as in the examples of asking ChatGPT to give you a picture of a "glass of wine filled to the very top", so it's "tone" isn't really something you can rely on as to whether or not it can actually correct the mistake. It's always willing to take another attempt, but I haven't found it always capable of solving the issue, even with direction.

[–] Bronzebeard@lemmy.zip 7 points 2 days ago

Meh, they apologize then proceed to continue making the same mistake. Repeatedly.

[–] Luminous5481@anarchist.nexus 41 points 3 days ago (3 children)

this seems like something I would do too, which is why I haven't installed fail2ban

[–] ramasses@social.ozymandias.club 12 points 2 days ago* (last edited 2 days ago) (2 children)

You should really install fail2ban, just for peace of mind. If you dont, at the very least setup SSH keys.

[–] hexagonwin@lemmy.today 4 points 2 days ago

unless I have an insecure password or a security vulnerability in sshd, how would it be a problem? I haven't had any issues for the last 6 years.

[–] Luminous5481@anarchist.nexus 3 points 2 days ago

I was making a joke. I already have rate limiting protecting my Authelia login page.

[–] ozymandias117@lemmy.world 4 points 2 days ago

I set up, and prefer, iptables rules to rate limit logins.

I have mine set so you can connect up to 5 times per 15 minutes.

Blocks bots well enough, and if I really mess up, I just wait 15 mins

[–] smiletolerantly@awful.systems 2 points 2 days ago* (last edited 2 days ago) (2 children)

The nice thing about SSH key-based access is, I either have the key and login succeeds, or I have no business trying to log in.

That's why my remote root server bans via fail2ban after a single failed login.

Yes I've had to write support to get a KVM. Yes it's still configured like this.

[–] baguettefish@discuss.tchncs.de 3 points 2 days ago (2 children)

i am a tailscale enjoyer, which means i can set up tailscale ssh once on each machine and then from another machine just login over tailscale

[–] InternetCitizen2@lemmy.world 2 points 2 days ago (2 children)

How's that different from normal ssh?

[–] glibg10b@lemmy.zip 1 points 1 day ago

With Tailscale, you don't have to open your SSH port to the whole internet. It's actually kind of silly that many servers are still exposing ports for private services on the internet

[–] baguettefish@discuss.tchncs.de 1 points 2 days ago

you can disable the need for a password or key if you like, and you also don't really need fail2ban, since nothing is actually port forwarded anywhere

[–] smiletolerantly@awful.systems 2 points 2 days ago* (last edited 2 days ago)

Eh, the machine is actually in one of my wireguard nets anyways, but for different purposes.

[–] probablymissing@lemmy.world 0 points 2 days ago (1 children)

as a nixos enjoyer, i have no idea how to setup ssh keys. fail2ban and a regular password for me.

yes, i have locked myself out of my own server for hours at a time because i'm an absolute tool.

[–] smiletolerantly@awful.systems 3 points 2 days ago

Ehm... I'm also on Nixos and I'd say it's super trivial.

services.openssh = {
  enable = true;
  settings = {
    PasswordAuthentication = false;
    PermitRootLogin = "no";
  };
};

users.users.<name>.openssh.authorizedKeys.keys = [ list of pubkeys ideally read from file in repo ];

[–] Maiq@piefed.social 14 points 3 days ago

So many times have I been able to wing it.. Setting up kitty terminal, just find out where the default config lives and copy to where it expects it to be under your user. Read the config and change what you want. Easy peezy. Download gnu-stow, stow -h, setup a small test directory, trial and error. Bam done.

When it comes to security winging it just isn't an option. The turn of phrase is RTFM, and the arch wiki is my first stop. Setup fail2ban, sshd, snapper, hibernation btrfs swap the same day. Had some misconfiguration in my jail but reading systemctl output gave explicit errors that easily help you figure out your config. In my case I didn't use the [sshd] headder.

Still have an issue when waking from hiberhation where my desktop leaks for one second before the lock screen kicks on, still need to setup a service to enact an on wake sleep 1. My adhd has made me chase other squirrels.