TechTakes

I just want to point out that every single heavily downvoted, idiotic pro-AI reply on this post is from a .ml user (with one programming.dev thrown in).

I wonder which way the causation flows.

From the blog post referenced:

We do not provide evidence that:

AI systems do not currently speed up many or most software developers

Seems the article should be titled “16 AI coders think they’re 20% faster — but they’re actually 19% slower” - though I guess making us think it was intended to be a statistically relevant finding was the point.

That all said, this was genuinely interesting and is in-line with my understanding of the human psychology that’s at play. It would be nice to see this at a wider scale, broken down across different methodologies / toolsets and models.

I have an LLM usage mandate in my performance review now. I can’t trust it to do anything important, so I’ll get it to do incredibly noddy things like deleting a clause (that I literally always have highlighted) or generate documentation that’s more long-winded than just reading the code and then go to the bathroom while it happens.

For each time saved, you're having that one kink that will slow you down by a fuck ton, something that AI just can't get right, something that takes ai 5 hours to fix but would've taken you 10-20 to write from scratch

Anyone who has had to unfuck someone else’s work knows it would have been faster to do the work correctly from scratch the first time.

@dgerard I normally consider myself a 10x developer. With the 10x speedup of AI I now consider myself a 100x developer. I can replace an entire small business worth of developers with just myself and my LLM bot assistance. Just pay me $100 million up front no strings and I'll prove it to you! /s

@dgerard@awful.systems who is your illustrator? These are consistently great.

ahahaha holy shit. I knew METR smelled a bit like AI doomsday cultists and took money from OpenPhil, but those "open source" projects and engineers? One of them was LessWrong.

Here's a LW site dev whining about the study, he was in it and i think he thinks it was unfair to AI

I think if people are citing in another 3 months time, they'll be making a mistake

dude $NEXT_VERSION will be so cool

so anyway, this study has gone mainstream! It was on CNBC! I urge you not to watch that unless you have a yearning need to know what the normies are hearing about this shit. In summary, they are hearing that AI coding isn't all that actually and may not do what the captains of industry want.

around 2:30 the two talking heads ran out of information and just started incorrecting each other on the fabulous AI future, like the worst work lunchroom debate ever but it's about AI becoming superhuman

the key takeaway for the non techie businessmen and investors who take CNBC seriously ever: the bubble starts not going so great

Devs are famously bad at estimating how long a software project will take.

No, highly complex creative work is inherently extremely difficult to estimate.

Anyway, not shocked at all by the results. This is a great start that begs for larger and more rigorous studies.

As someone that has had to double check peoples code before, especially those that don't comment appropriately, I'd rather just write it all again myself than try and decipher what the fuck they were even doing.

Megacorp LLM death spiral:

1. Megacorp managers at all levels introduce new LLM usage policies.
2. Productivity goes down (see study linked in post)
3. Managers make the excuse that this is due to a transitional period in LLM policies.
4. Policies become mandates. Beatings begin and/or intensify.
5. Repeat from 1.

5% "coding"

95% cleanup

The N=16 keeps getting buried. Deliberate?