I wonder about the timing of this. I just got a backup NAS out at my mom's house some miles away and for one or two beautiful days I was sending Rsync differential backup jobs through the vendor interface for backups over Wireguard. The NAS is still on my network over WG, comes back up in that way after a reboot…but for the last week, those backup jobs just break with a useless error. I haven't had the time to look under the hood at logs but I've been assuming this was slopping config on my part cause I'm new at it. But it would almost be a relief if it was just a bad update (before the graver implications of the situation set in on my mind). I wish I had enough background in this stuff to be useful, but I'm just a bystander and a grateful, useless end user.
Fuck AI
"We did it, Patrick! We made a technological breakthrough!"
A place for all those who loathe AI to discuss things, post articles, and ridicule the AI hype. Proud supporter of working people. And proud booer of SXSW 2024.
AI, in this case, refers to LLMs, GPT technology, and anything listed as "AI" meant to increase market valuations.
For those who don't know, "tridge" is legendary.
He casually reverse engineered Microsoft's SMB protocol, creating Samba, back when windows file sharing was a key part of Microsoft's lock in. He also isn't just the maintainer of rsync, he invented the algorithms it uses. People who worked with him consider him a genius and a guru.
How much you want to bet he's just bombarded by the "ai security reports arms race" I saw on here a couple days ago, where people use LLMs to find security holes in open source projects (likely a form of 'fuck the dev' training)? I mean, for hundreds of reports to come in, some of which I'm sure are legitimate, is overwhelming to a team... and he's just one dude.
Edit. Looks like I may have been right. User Chairman Meow posted an excerpt from Discord that basically says that. Even legends get lonely, it seems.
Yep. A solo dev working on a project. Legitimate security flaws found by people who don't know much of anything about coding, but can prompt an LLM. They don't even understand the bugs they're submitting, so if he has questions they can't help.
His choice is either to spend all of his free time trying to patch these bugs, or to look for help. It's very hard to find help as a solo dev on an unsexy but essential tool. So, he turned to LLMs to help. And, who knows, maybe he's able to use them slightly more responsibly than other devs. But, LLMs almost inevitably lead to their own bugs because LLMs are always confident, and are designed to produce something that looks as much as possible like real working code, but without any actual thought or analysis behind them.
Which makes it all the more disturbing that he has turned to slopmachines.
If you read the discord chat logs, it makes sense. He's being bombarded by security vulnerabilities discovered via LLMs, from people who barely know how to code and can't even explain the flaw that their LLM discovered. He's a solo maintainer, and his choice is either to leave these security vulnerabilities open, or to turn to LLMs to try to keep up with the need for patches.
I don't think he made the right choice, but I think he's probably a much better programmer than me.
I don't think he made the right choice, but I think he's probably a much better programmer than me.
I'm a senior dev that works with LLMs these days and been running dozen people teams before and reading slop code is a skill that needs to be built through months/years of work no matter how good of a programmer you are - it's a different skill set.
This is about to be a big thing. LLMs are very good at finding exploits and creating scripts to exploit them. Now a script kiddy is much more powerful. Companies are trying to figure out how to respond. Red Hat's Project Lightwell is one such project.
Just gonna copy what tridge said:
bottom line is if you want to be useful then pick holes in the test suite, find things it doesn't cover, find interactions between options it doesn't pin down, report those and offer fixes for that.
Why ask for forks or alternatives?
You may not like it, but this is what 10x productivity looks like.
This is negative productivity. It worked before, and now it doesn't.
But when it worked there was no work being done. The repo just stayed there, working. Doing nothing.
A few LLM commits have kickstarted the process of a lot of people checking their rsync versions, choosing the correct one. And so on. That is work that wasn't being done before, and now it is done thanks to LLMs. Truly a wonder of our times.
Move fast and break things. Features over stability.
Makes sense for a lean startup. Not so much for a widely used utility for backing up important data.
I'm starting to think that I don't want to use Arch anymore and thus always be among the first to get all the new slop.
And now with supply chain attacks being all the rage it's like being in a convertible with its top down tailgating a flatbed filled with portapotties.
What is it about LLMs that makes so many devs' brains melt?
Studies have already shown that the moment you start relegating code to LLMs you kinda just start using them as a crutch even if you don’t need them.
Staff Engineer here. Our CTO told us in March two things. One, if we didn't get on board with AI then we would be unemployable in 3 months and two, we had to use AI for everything. Literally everything. I asked (as a senior engineer of 19 years) if that included simple bug fixes I see that take minutes vs 30+ describing the problem. The answer was "absolutely". Our budget is $400K /month to Anthropic and we exceeded that 3 weeks into May
It's always the damn suits.
Pump those numbers, make them regret the decision.
Also that’s an insane budget for AI.
Our budget is $400K /month to Anthropic and we exceeded that 3 weeks into May
Fucking hell, that's so much money to burn on management's AI addiction. Have to wonder how your finance department feels about burning almost half a million a month.
Also, wild that management is telling you that not letting your skills degrade by handing everything off to an AI is what'll make you unemployable.
They think once the ball is rolling, then they can phase out the humans.
They think that AI usage is like training a junior dev, that it starts out hopeless but over time can operate without the expertise.
They don't realize that invoking AI doesn't work that way, that the context window is the only accumulation of anything germain to your codebase, and that the model doesn't evolve based on that interaction.
So they don't care about the skills, they want to get to the point where they can toss a prompt into Claude and have it all taken care of, thinking that their employee usage of it somehow accelerates that outcome.
Here it seems like panic in the face of things like the CopyFail/DirtyFrag/Fragnesia/ssh-keysign-pwn stuff.
That if he didn't let AI 'fix' the issues it can find first, then someone will hit rsync with devastating CVEs.
Problem is he saw that the tool was offering to 'fix' things that perhaps weren't quite right and saw a credible proposal to implement fixes, but the fixes were for bugs no one cared about or noticed and weren't security related, but incurred side effects that people did notice.
If you have a non-security bug that's been in place since 2019 and the only thing that noticed was an LLM analysis of your codebase, it may be best to let sleeping dogs lie...
The project's issue tracker has been pretty wild recently, for example https://github.com/RsyncProject/rsync/issues/929
Wherein backups falling consists entirely of one self report of the users self written backup script not working followed by him seeing commit messages indicating usage of ai with zero effort to show work diagnose the cause or bisect to failing commit despite poster being a hobbyist who dabbles in programming.
Trust me bro.
You should try reading the rest of the comments section. It's not just this one dude.
Honestly what happened to language models is a shame. Good tools perverted to try and do every job. LLMs dont really have a place and eat up so much resources with what effects to a okay scaffolding tool in code, and a piece of shit liar everywhere else. I remember seeing this shit being used in medicine almost 15 years ago thinking thats gonna be a cool technology to we expand. It was fucken not.
Neural networking has so much potential in so many places, yet of course the industry collectively zoomed in on LLMs specifically and is trying to sell them as a panacea to the world's problems.
As though a mechanical parrot knows anything about good coding practices, or literally anything outside of mimicking speech patterns.