change my view

This is objectively false. Here's why.

A Formal Language is one that is rigorously defined in logic. It has a grammar which rigidly defines the set of strings (or patterns of symbols) that are in the language. Math and computing is built on formal languages. It's not intuitive, but a sufficiently well defined problem statement is a grammar, and the set of solutions to the problem is the language. So in a very literal sense, being able to "speak" a formal language is the same as calculating the solution to a problem.

A Natural Language on the other hand is contextual. It evolves over time. It's not random, but it is arbitrary. There is not hard line between what is a valid sentence in a natural language, and what's not. Me speak with broke worded, but still can understand you. There is a pattern there, if it were random there would be no meaning, but it is impossible to formally define.

LLMs and AI in general are specifically suited to this last part: approximating solutions to problems where there exists a pattern, but the pattern is impossible to rigorously define. Looking at a photo and knowing whether it's a cat, creating artwork with a specific aesthetic, or speaking a natural language are what AI excels at. But calculating specific solutions to well defined problems is not, for that we built calculators.

All that said, humans brains are better at natural language than being calculators (that's why we eventually invented calculators), so I think there may come a day when we make an AI that is capable of designing and building a calculator. And at THAT point, AI will handily replace programmers. But that is a much harder problem it would seem.