Web Development

They are not the automated from 0 to 100 coders that some people claim them to be. But they are quite capable, definitely much more capable than what anyone could have imagined ten years ago. Given well defined problems they can excel at even relatively complex tasks. I pointed Claude at a latex file of a somewhat complicated nonparametric statistical estimate calculation to look for any mistakes and it was actually able to find some. I then pointed it at a code that replicates the calculations and it was also able to correctly identify some issues with the code. I think this is the way one should use LLMs, not let it loose on coding tasks. In the former way you won't even be able to burn through your first tier account quota where as in the latter the LLM will likely end up getting in weird loops burning tokens like there is no tomorrow. Also this method of sane usage of LLMs is much more suitable for open local LLMs. I don't think there is any doubt anymore that LLMs can be very useful tools, not just for doing stuff but learning it too. People should move past the stage of invalid criticisms like "they are just stochastic parrots" and move to more serious matters like environmental impact, greedy fucking CEOs pretending LLMs are replacements for humans, degredation of skills, getting lazy at checking AI code and the unsustainable AI bubble that tech companies are pushing for.

part of the issue as well is that when they get something completely broken, people just re roll the output until they get something that’s broken in ways they don’t notice. Or re roll parts of it, or tell the system to judge if the output is broken and re roll the parts that it judges are broken automatically. Or increase the size of the context window to get it closer to that upper limit of accuracy.

All this together can get a more functional output with less effort, and as people find these tricks it gives them the illusion of an upward trend in capability, like this is all solvable issues that will improve as time goes on. Big problem with that though, theses tricks and methods explode the compute cost rapidly. That’s all fine and dandy when everyone is getting their compute costs for these tools subsidized by these model providers, but eventually they will need to charge the real cost of running this. The compute providers that host the model providers are also running at a loss, trying to help grow the market segment and maximize their market share. And then places that have the datacenters in them are giving tax breaks and discount utilities to attract new construction.

Everyone except the people making the chips is selling at a loss, and as people pile on usage to make up for the fundamental limitations of these systems, the demand balloons, validating to the providers at all levels that this is a growing market they should invest more in to.

But eventually… they need to make money. The bill comes due on all the debt and investment. What happens to the people who have fully embraced these to run their businesses? Or to all the people who have built their skill set around using these systems? It’s a crisis, a series of crisis, each time a debt wall gets hit by someone in the supply chain. A half decade of technical debt that just got really expensive to deal with, and not enough experienced people to handle it, since all the grey beared retired and not enough new people got brought in to replace them because the entry level work was automated.

This is very obvious unless you are in tech leadership, in which case your job is now to push this at all costs and suppress dissenting voices.

You know the feeling that you want to rewrite a project? But you know that most rewrites are a bad idea.

Be it your own, old code. Or code you inherited.

There is a small chance that the world realizes that they went in the wrong direction and nothing can get fixed. That will be the time of rewrites.

No, I don't expect this to be very likely. The agent code will remain, and human programmers get yelled at for not fixing it fast enough.

It's so nice to see some people speaking reason. If only any of those people ran multibillion dollar companies.

This alarm's being rung for over a year now, so "calling it now" means finally reading the writing on the wall

if it's broken in a way that can't be detected, is it actually broken?

all software is broken in some way. if the rate of bugs generated by llm and the severity of those bugs drops below the rate you would expect from a human programming team, then llm is offering something competitive.

No one paying the bills cares