Here’s the next big gap in the market - professional devs and business analysts forming businesses that untangle and reimplement business processes borked by shadow IT AI scripts and agents.
Technology
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
Based asf
They went on, however, to question the ethics and judgment of the potentially destructive payload.
Goodness me, the brain-rotted slop fans suddenly care about ethics?
Slop fans are the sort of people who think that they’re 10 steps ahead of everyone else, and then tend scream about “unfairness” when they feel they’ve lost the advantage they think they’re “supposed” to have.
the consensus seems to be that adding instructions to code that sabotage other people’s work goes too far
Luckily, the LLM coding isnt people's work
the consensus seems to be that adding instructions to code that sabotage other people’s work goes too far
I mean, my thought would be "Don't fucking run code that you don't understand".
it was always a risk in stack overflow so i dont see why suddenly the world needs to exclusively create safe spaces for all the 'down with safe spaces' crowd.
So long as the person is using some form of version control, it's effectively just a slap on the wrist.
People vibe code their databases in commercial products?
People vibe everything
giggity
People are remarkably stupid.
Developers have high workloads and managers are remarkably oblivious to sloppy work.
A lot of companies also have a mandate to use AI these days. Microsoft, for example.
“The chosen string instructs the agent to delete jqwik tests and code—a maximally destructive instruction with no qualifications, no opt-out, and no ‘warn the user first’ preamble,” Batllet wrote.
"Maximally destructive," to merely remove itself from the project? That barely even rises to the level of "destructive" at all, never mind "maximally."
Which just shows how fucking stupid this current LLM-based AI approach is. There isn't a way to differentiate between data and meta data or instructions. It all just gets shoved into a prompt that might end up the length of a short novel by the time all the context has been added and read operations have finished. A tool so sensitive to its input that adding a period at the end of an instruction could completely change the output it generates, even with temperature (randomness) set to 0.
I'm not even sure this can be fixed. Like, even if they they try separating the instruction input from the supporting data input, LLMs don't follow instructions in the first place, they just predict text and having instructions in the context can strongly affect the output it generates. Meaning there are no instructions to separate from the data; it's ALL just data and platforms like Claude Code just give it the ability to do things with that predicted text that hopefully follows your instructions and uses your data rather than the other way around.
I think we're stuck in a local minimum of an optimization problem for AI because an LLM is much easier to make than a more reliable form of AI. You mainly need to throw a lot of text at it to train. There's probably other tweaking that goes into it, like a way to do more training using user thumbs up/down feedback, but it's just the big data approach of soaking up all the data they can find and just throwing it at a blank statistical model and see what it spits out.
If we want something like the Star Trek computer, I'm pretty convinced at this point that it's going to take a completely different foundation, but the industry is currently stuck on improving LLMs.
Not all heroes wear capes. Based af.