this post was submitted on 13 Dec 2025
77 points (84.7% liked)
Programming
24097 readers
289 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I'm perplexed as to why there's so much advertising and pushing for AI. If it was so good it would sell itself. Instead it's just sort of a bit shit. Not completely useless but in need of babysitting.
If I ask it to do something there's about a 30% chance that it made up the method/specifics of an API call based on lots of other similar things. No, .toxml() doesn't exist for this object. No, I know that .toXml() exists but it works differently from other libraries.
I can make it just about muddle through but mostly I find it handy for time intensive grunt work (convert this variable to the format used by another language, add another argparser argument for the function's new argument, etc..).
It's just a bit naff. It cannot be relied on to deliver consistent results and if a computer can't be consistent then what bloody good is it?
I do wonder why so many devs seem to have so wildly different experiences? You seem to have LLM's making up stuff as they go, while I'm over here having it create mostly flawless code over and over again.
Is it different behavior for different languages? Is it different models, different tooling etc?
I'm using it for C#, React (Native), Vue etc and I'm using the web interface of one of the major LLM'S to ask questions, pasting the code of interfaces, sometimes whole React hooks, components etc and I get refactored or even new components back.
I also paste whole classes or functions (anonymized) to get them unit tested. Could you elaborate on how you're using LLM'S?
I really don't feel like getting in depth about work on the weekend, sorry.
Yeah man, I was going to say there's already too much talking about work on a Saturday in this thread than I like. 💢
Naaw, just when things started to get interesting...
We're in the middle of a release and last week was a lot. I shouldn't have stepped into the thread!
I suspect it mostly relates how much code base there is on internet about the topic. For instance if you make it use a niche library, it is quite common that it makes up methods that don't exist in that library but exists in related libraries. When I point this out, it also hallucinates saying "It was removed after version bla". I also may not be using the most cutting edge LLM (mix of freely available and open source ones).
The other day I asked it whether if there is a python library that can do linear algebra over F2, for which it pointed me to the correct direction (Galois) but when I asked it examples of how to do certain stuff it just came up with wrong functions over and over again:
In the end it probably was still faster than google searching this but all of these errors happened one after the other in the span of five minutes, so yeah. If I recall correctly, some of its claims about these namespaces, versions etc were also hallucinated. For instance vstack also does not exist in Galois but it does exist in a very popular package called numpy that can do regular linear algebra (and which this package also uses behind the scenes).
It's the language and the domain. They work pretty well for the web and major languages (like top 15).
As soon as you get away from that they get drastically worse.
But I agree they're still unambiguously useful despite their occasional-to-regular bullshitting and mistakes. Especially for one-off scripts, and blank-page starts.
It's the models that make the difference. Up until like Nov it's all been really shit
But I've been doing this for years.