this post was submitted on 23 Jun 2026
52 points (87.1% liked)

Futurology

4309 readers
5 users here now

founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
[–] mindbleach@sh.itjust.works 2 points 3 weeks ago (1 children)

Genuinely compact models are hitting benchmarks just a couple months behind the big boys. And eventually we'll get better at decoupling data from processing - so the model can do a regular-ass search of a regular-ass database and pull details into its context as needed. Ideally while also decoupling that context from the prompt, because apparently these things can have a hundred attention heads, and still nobody thought of having two text input fields.

[–] fluffykittycat@slrpnk.net 1 points 3 weeks ago (1 children)

You'd think they'd have done that by now, and maybe some symbolic AI too

[–] mindbleach@sh.itjust.works 1 points 3 weeks ago

All focus has been forced onto LLMs and diffusion, even though only diffusion works properly. And those LLMs better iterate on the exact same mechanisms we've tweaked for the last six years, because all results will be compared to the state of the art, right the hell now, not a comparable level of development or compute.