TechTakes

2057 readers

380 users here now

Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

founded 2 years ago

MODERATORS

dgerard@awful.systems

254

AI coders think they’re 20% faster — but they’re actually 19% slower (pivot-to-ai.com)

submitted 18 hours ago by dgerard@awful.systems to c/techtakes@awful.systems

32 comments fedilink hide all child comments

podcast version
video version

you are viewing a single comment's thread
view the rest of the comments

[–] mspencer712@programming.dev -5 points 17 hours ago (3 children)

The N=16 keeps getting buried. Deliberate?

[–] dgerard@awful.systems 14 points 14 hours ago (1 children)

this user has been removed for commenting without reading the article

being from programming dot dev is just the turd on top

[–] froztbyte@awful.systems 11 points 14 hours ago (1 children)

programming.dev: statistical sampling excellency (worst edition)

[–] self@awful.systems 11 points 13 hours ago (1 children)

programmers learned what N means in statistics and immediately realized that “this N is too small” is a cool shortcut to sounding smart without reading the study, its goals, or its conclusions. and you can use it every time N is smaller than the human population on earth!

[–] blakestacey@awful.systems 13 points 13 hours ago (1 children)

This N is too small: ~N~

[–] OpenStars@piefed.social 5 points 12 hours ago

Skill issue - this N is even smaller:

spoiler

[–] Feyd@programming.dev 28 points 17 hours ago

You're acting like this is a gotcha when it's actually probably the most rigorous study of AI tool productivity change to date.

[–] blakestacey@awful.systems 21 points 17 hours ago (2 children)

Paragraph 2:

METR funded 16 experienced open-source developers with “moderate AI experience” to do what they do.

[–] HedyL@awful.systems 22 points 16 hours ago

... and just a few paragraphs further down:

The number of people tested in the study was n=16. That’s a small number. But it’s a lot better than the usual AI coding promotion, where n=1 ’cos it’s just one guy saying “I’m so much faster now, trust me bro. No, I didn’t measure it.”

I wouldn't call that "burying information".

[–] swlabr@awful.systems 14 points 17 hours ago

. Debate me bro? (jk)