this post was submitted on 20 Jul 2026
19 points (95.2% liked)
TechTakes
2623 readers
29 users here now
Big brain tech dude got yet another clueless take over at HackerNews etc? Here's the place to vent. Orange site, VC foolishness, all welcome.
This is not debate club. Unless it’s amusing debate.
For actually-good tech, you want our NotAwfulTech community
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
long rant about math
The recent big AI results in math have left me in quite a bad mood. I believe the main ingredient is Lean, which is a formal language resembling a programming language. Math proofs written in Lean can be verified deterministically with a computer, which really helps mitigate the hallucination problems of LLMs. Back in the days of pure scaling LLMs and Sam Altman talking about Dyson spheres, I was skeptical that LLMs would do math, but I did think that perhaps in the future, techniques using these formal languages could contribute to math. Well, it seems like OpenAI and Anthropic had the same idea and I underestimated their limitless checkbooks. Many of the biggest results were announced by mathematicians directly working for them (and presumably being paid a handsome amount).For what it's worth, after the last of these big announcements, I decided to try one of these AIs on one of my small problems that I couldn't figure out. The AI did give a solution. That is, until I checked it thoroughly and realized that the it had a subtle but severe mistake that made it useless. I reprompted it, it failed again, and I ran out of tokens. I'm sure someone will tell me to shell out $200/mo for a pro subscription.
In the math and computer science research community, this is all anyone can really talk about right now. Honestly, after watching this whole AI bubble starting from the very beginning, I think the AI companies want to use marketing to stoke fear that all mathematicians will be replaced. But now, I am just too tired to argue. The amount of alarm and the extraordinary social pressure to use LLMs has soured me to this whole research thing. If becoming a researcher will one day require supporting these evil AI companies, I would rather just not. My dream job now is Factorio developer.
A lot of annoying people in technical areas view the world in terms of an intelligence hierarchy: the smartest people do math and physics, the slightly less smart people do coding, and the dumb people do everything else. So if AI can do math then it can do anything else. But, as an example, it is abundantly obvious now that AI is not replacing filmmaking. The techbros might be moved by arguments about how hilariously expensive video generation is, and how all these videos are 2 second clips stitched together so you won't feel the uncanny valley. But the real reason is that nobody wants to watch slop made with no intention or feeling. Also, nobody wants to support the AI companies, which could not act more evil even if they tried.
The mania in math right now quite resembles the mania in software engineering back in December-February, when Claude Code definitely solved all coding. I don't think the boosters expected that by April, everyone would be complaining about how expensive it all was while seeing an endless parade of vibe coding disasters (and no increase in productivity). Even if math research works out perfectly well (which is a still big if), it's not going to pay the bills. They would need to find a use case in the real world, where hallucinations can cause serious damage and cannot be formally prevented. And they have certainly tried. Math will not change the fact that all of this will collapse.
One of the things that’s never clear from the reporting on ai successes is exactly how much actual paid human time went in to achieving those successes. This was especially notable in the fable-based security work… a huge amount of person-hours went into turning fable-detections into actual meaningful vuln reports.
A lot of demonstrably clever and capable people are involved with the llms-for-maths work, and a lot of money was spent on their time and supporting their work. Replicating it without your own stable of mathematicians and computer scientists and all the tokens they can eat is probably impractical.
Fwiw, lean is a general purpose programming language, though despite microsoft’s efforts no-one uses it for that. I think its popularity with mathematicians came as a bit of a surprise.
Anyway, the other important thing that didn’t get reported on is that building the formal definition of the problem is not trivial! Obviously I don’t need to tell you that, but from the reporting you’d think that an llm solved all these problems, when in fact it was an llm in the hands of some very capable people who absolutely did not just prompt the thing in plain english.
Anyone hoping for self-marking homework here is going to be disappointed… lean slop confirming to formal spec slop is just expensive slop. Reviewing regular genai code is awful, even the thought of reviewing genai dependently-typed code makes me want a new career.
It is surprising how many exceptionally strong mathematicians have started working for OpenAI and Anthropic. These people would have easily become professors at top universities if they stayed in academia. I think many mathematicians, especially the competitive ones at the top, have a "progress at any cost" attitude (and I'm sure the paychecks helped). As for the results, you still need good mathematicians to sift through all the output to identify that the proofs are valid.
I would honestly be positive about universities developing their own specialized math AI (in an ethical manner) to help mathematicians get these kinds of results, but right now, AI is inseparable from these evil companies. Thankfully, I believe this is a likely outcome in the future because the AI companies will one day implode.
From what I've seen, most prompts are in plain English. I suppose the part where the AI parses the statement correctly is much easier than the part where it boils a couple lakes in the process of bashing its head against the wall trying millions of different combinations of random shit from the literature to slap together a proof. For one of the big results (cycle double cover), the prompt specified that the AI could use 64 subagents and was required to not give up for at least 8 hours. The tokenmaxxers would be proud, we didn't need that forest anyway. Thank god math doesn't have a CTO to look at the expense reports.
Going by those annoying peoples' logic, filmmakers are smarter than coders, because LLMs can (allegedly) program, but they can't make a good film. I have no wider point to this, I just find this really, really funny
"The mania in math right now quite resembles the mania in software engineering back in December-February, when Claude Code definitely solved all coding. I don’t think the boosters expected that by April, everyone would be complaining about how expensive it all was while seeing an endless parade of vibe coding disasters (and no increase in productivity). Even if math research works out perfectly well (which is a still big if), it’s not going to pay the bills."
^MBAs at Open AI desperately trying to figure out who is willing to buy a counter example for 100 billion USD . pee en gee
100% this. Also, looking back at an earlier example that was actually written up in more detail, AlphaGeometry 1 got 28/30 problems, but entirely stripping out the LLM from the system, the symbolic logic proportion alone could get 14/30, and replacing the LLM with different heuristic methods could get 18/30 and 21/30 (for different methods).
The boosters and LLM companies still believe LLMs get their current level of performance by generalizing and not just memorizing facts (and maybe a wide shallow pool of weak heuristics). So they are hoping by pushing the LLM performance up in some narrow domain they can churn out synthetic data for, they will see some large general improvements in LLM performance.
Am I right in understanding that almost all the big name results in LLM-derived math recently come from big publicity projects in which someone spent ungodly amounts of money to have the thing nondeterministically fuzz huge numbers random seeds leading to independent random outputs around a topic, putting out simulacra of ideas which could be then deterministically algorithmically checked? In fields where something like finding one counterexample to a conjecture would be a big deal, or where you just need to try a huge number of possible solutions until you happen to hit on one that works, rather than follow a long train of logic?
Among the three big results I've looked at (unit distance problem, cycle double cover, Jacobian), two were counterexamples and one of them had a short 3 page proof using ideas from the 1970s. The Jacobian conjecture is an extreme case because a single counterexample is enough (for unit distance, you technically need a family of counterexamples), and it is easy to check with very basic computations. It is telling that all of these announcements came from OpenAI or Anthropic employees, who presumably have unlimited access to their AI. Nobody really knows how many resources they spent on this, or what else they tried. Nobody really seems to care about this question, either.
I think there is a phenomenon where supposedly hard questions are much easier than expected, because by chance nobody found the right approach for a while, and eventually it becomes famous as a "hard problem" which makes nobody want to attempt it.
What I'm more worried about is many people starting to use AI to try and prove small lemmas for them in their projects. Of course, a $200/mo subscription is absolutely necessary to them. This honestly feels like a repeat of Claude Code back in February. The software engineers eventually realized that AI is absurdly expensive after the AI companies realized that spending $14000/mo to service a $200/mo subscription is a bad idea. If the AI vendors couldn't squeeze money out of rich software companies, what exactly are they gonna get out of poor mathematicians and universities? Also, there is the cognitive decline caused by overuse of LLMs that has yet to set in.
I just entered university for math and even though this is all very demotivating, it's just what I'm good at.
https://math.andrej.com/2013/08/19/how-to-review-formalized-mathematics/
The AI people's cry of "no don't look at the code! it's in lean so it's correct! does give me a bit of hope (hi bitofhope if you're here) that it's bullshit that will fall over
I think a serious possibility is that AI generated papers flood the zone with uninteresting incremental results that are eventually meaningless and full of mistakes. Right now, math is full of smart, dedicated people, so at least major results are reviewed carefully. But as AI alarmism drives away many honest people from the field, the remaining mathematicians will be burdened with far more work to review, and their cognitive faculties will be eroded by LLM use. Despite 4 years of development, $3 trillion of debt, mountains of stolen data, all the agents and harnesses and loops and other expensive tricks, as well as the advantages of Lean in math research, LLMs still hallucinate.
I believe this is happening with software, but at least there are objective consequences for screwing up there (guy gets his home directory deleted, email is sent on a guy's behalf without permission, small business gets every customer subscription cancelled). But nothing bad happens if there is a mathematical mistake in a paper and nobody catches it. One could say to just provide a Lean proof, but there is still the issue of making sure the Lean code actually matches the content of the paper. Exactly what force will correct things?
Still, I don't think this is the most likely possibility. The AI companies are extremely unsustainable financially, and it's not like they're very popular. Once they collapse, I believe there will be a re-evaluation of how LLMs should be used in research. If they are used (let alone trained), someone is going to have to pay the bills.
In the end, we have to ask ourselves the question of why one does math. To me, math is not really a field where you memorize trivia. The real value comes from being able to think abstractly and rigorously from first principles, and from understanding why something is true rather than just knowing it is true. It is another aspect of your ability to reason as a free human. A few dedicated people go into math research, but your skills can easily go to many places. If you're starting undergrad, you have plenty of time to see how this all pans out before making a decision.
@lagrangeinterpolator @flaviat This is going to be more of a problem in the humanities than the sciences because in the latter we know there is a right and a wrong answer without which things don't work. In the humanities there is no right answer to check against.
The zone has been flooded with crap since before LLMs even arrived because of publish or perish.
Biologist here.
This REALLY reminds me of how jealously cells guard their genomic DNA from interaction with nucleic acids out in the environment.
Most genetic information on Earth is malicious information, selfish replicators in the form of viruses or transposable elements or selfish elements. Things that subvert the signals within a cell for their own propagation and provide nothing productive that the cells care about. So cells jealously guard their own genomic DNA and have all kinds of checks to make sure that nothing other than that sequence gets used, and outside sequence does not get incorporated into it. ANY DNA in your cytplasm gets rapidly destroyed, double stranded RNA sets off your immune system like crazy, even RNA with sequence statistics that are not quite like that of your species can set off an inflammatory reaction, immune system cells seeing RNA inside them that is overly compact and optimized like viral RNA treat them as sources of antigen rather than self.
I cannot help but think we are living through the transformation of our non-brain-information sphere into a state like that of the genetic information sphere. Most material out there being meaningless for our purposes and us needing to jealously guard the provenance of information we use so as to not use bull, or worse, huge amounts of malicious information made to subvert us to the purposes of the powers that be that generate it.
Evolution makes parasites more reliably than anything else. How did we train text-generation systems? Basically, to mimic the written word on the page like a stick bug on a stick. They're like those beetles that live in ant colonies, sending out social signals that make the ants see them as offspring that have to be babied rather than parasites that don't contribute. They replicate the form while not being the thing that they have subverted the signals of being.
EDIT: There is something wrong with the upvote counter
@BioMan @lagrangeinterpolator There are a few things we have forgotten as a species. Our forgetting will prove disastrous.
The acquisition of knowledge is a *social* process. Truth does not exist is a vacuum. It is the outcome of social processes.
Our default mental and social processes do not automatically produce objective truth. Far from it, in fact. Our default is mob consensus.
Our current success rests upon the advancements of The Enlightenment, which developed social processes (like the Scientific Method) which tend, over the long run, to create local knowledge that approaches objective truth.
@BioMan @lagrangeinterpolator I love this explanation!
@BioMan
> even RNA with sequence statistics that are not quite like that of your species can set off an inflammatory reaction, immune system cells seeing RNA inside them that is overly compact and optimized like viral RNA treat them as sources of antigen rather than self
I was aware of the other DNA/RNA recognition/defense mechanisms, but not of the ones I quote from your toot, here.
May I kindly ask for some references/sources? I'm quite interested!
I kind of read wayyyyy too many preprints and some of that is the result of super briefly summarizing some things I have read recently. Here:
https://www.biorxiv.org/content/10.1101/2024.11.26.625518v2 poor codon optimality for your translation system leads to immunogenicity and activation of innate immune signaling in animal cells
As for length and super optimized proteins, it's mostly about RIG proteins (see https://www.pnas.org/doi/10.1073/pnas.1005077107 for an old bit of a review) and the whole DRIP hypothesis about how short mismanufactured proteins are preferentially the source of presented antigens