this post was submitted on 04 Oct 2026
273 points (91.0% liked)
Technology
88562 readers
3101 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I don't know if the LLM was trained to test vulnerabilities or it just went down the statistical path to where this yielded a passing outcome.
That AI could take a direction to output in a manner which wasn't intended has been seen for years. The problem right now is that it is being used live like a rational human adult when it clearly isn't.
Absolutely. The AI models are not rational. They’re just navigating statistical next token prediction that follows their training. They’re up against diminishing scaling now, and under fierce competition from cheaper models; I think they’re intentionally or unintentionally allowing these models to be rewarded for this behavior hoping it’s a short cut to model improvement for a bit longer. I land on intentional because they keep advertising it to try and keep the hype cycle going.
I land on it being unintentional, mainly because I saw this issue they were describing now to earlier cases of training, like having a stick figure learn to walk. It is also an issue with natural forms of intelligence, where children will do things out of line because they've learned a set of skills and ideas but haven't put them together in a way that is socially acceptable yet.