AI reasoning works, but the chains of thought may be mumblings

The most honest summary of the AI reasoning debate comes from Santa Fe Institute researcher Melanie Mitchell, who has spent decades in the field. In a wide-ranging Quanta Magazine feature published July 31, she distills what we actually know about large reasoning models into three claims. Number one: they work, improving accuracy on reasoning tasks over plain LLMs. Number two: the chain of thought text they emit is not necessarily faithful to what happens inside the model. Number three: a lot of that text is not even useful. You can take it out.

That third point is the one the industry would rather not confront. A 2025 paper from Northeastern University and UC Berkeley examined frontier open-source LRMs and found that between 30% and 60% of their “thinking steps” had minimal causal impact on the answers produced for benchmark math questions. Chop half of them out, and performance barely suffers. The study’s author Weiyan Shi put it carefully: those chain-of-thought prompts may not be linked to the final output.

The evidence against chains of thought as faithful records keeps piling up. Arizona State University researcher Subbarao Kambhampati’s lab showed in 2025 that replacing a model’s correct traces with incorrect or irrelevant ones did not degrade performance on a formal reasoning task. A 2024 paper from New York University found that meaningless filler tokens, literally strings of dots, could function in place of human-readable reasoning. William Merrill, an author on that paper and now at the Toyota Technological Institute at Chicago, states the conclusion flatly: there is no guarantee the chain of thought has to be meaningful in any sense.

This creates a strange superposition. OpenAI’s general-purpose reasoning model solved a famous open mathematical research problem, the unit distance problem, in one shot in May 2026. Google DeepMind and mathematician Terence Tao used AI to improve solutions to 67 problems across analysis, combinatorics, geometry, and number theory. LRMs have won gold medals at the International Mathematical Olympiad. Yet the mechanisms that supposedly produce these results, the reasoning traces, appear to be at least partly decorative.

Kambhampati’s research group titled their position paper, presented at the 2026 International Conference on Machine Learning, “Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!” The title reads like exasperation. Kambhampati, a former president of the Association for the Advancement of Artificial Intelligence, does not deny the models work when they work. He objects to the explanations. “There’s this general mindset that says, ‘Let’s go ahead and claim certain abilities, because eventually that might become true anyway,’” he told Quanta. “And my sense is: That’s not science. That is investment.”

The counterargument comes from OpenAI’s Sébastien Bubeck, who dismissed the earlier Apple critique of AI reasoning as “wrong,” blaming a training quirk in obsolete models. He invited researchers to revisit those results with modern models starting with GPT-5.5, which he says do not suffer from the issue. Apple did not make its researchers available for interviews.

The deeper problem is that the field cannot easily adjudicate this dispute. Every researcher Quanta spoke to acknowledged that negative findings on smaller open-source LRMs may not generalize to the latest products. The inner workings of frontier models remain trade secrets. OpenAI has not publicly revealed raw chains of thought since 2024, a policy adopted by Google DeepMind and Anthropic. When Bubeck was asked whether the unit distance proof used Lean, the automated theorem-proving tool, to verify results, he seemed almost offended. “The whole point is that the model is reasoning like a human would,” he said. “And when humans reason, we don’t use Lean.”

Technically, OpenAI released a rewritten summary of the model’s chain of thought, produced by two human experts using Codex. That is not the same as releasing the raw trace. The distinction matters. If the chain of thought is the evidence that reasoning happened, and the chain of thought is edited by humans before release, the evidentiary value is compromised.

Kambhampati points to one straightforward reason state-of-the-art LRMs work: they are often surrounded by normal software that guides and verifies their outputs. Agentic AI systems, which have transformed software engineering since fall 2025, work this way. So does Google DeepMind’s AlphaProof Nexus, which relies on Lean. The stand-alone reasoning model, the “think” part, is the harder case.

Pavel Izmailov, an NYU researcher who works for Anthropic and was part of its original reasoning-model team, doubts that reinforcement learning even incentivizes faithful chains of thought. “I mean, maybe it will,” he told Quanta. “But I would say the chances are not very high.”

The stakes here go beyond academic squabbling. If chains of thought are not faithful records of model internals, then every audit, every safety review, every regulatory filing that relies on reading a model’s reasoning trace is reading tea leaves. The European AI Act and similar frameworks increasingly demand explainability. A model that emits plausible-sounding but causally irrelevant text is not explainable in any meaningful sense; it is performative.

There is a middle path, and it is the one the evidence actually supports. Reasoning models work. They produce correct answers on hard problems at a rate that exceeds their predecessors. But the chain of thought is best understood as a byproduct of training, not a window into cognition. The pinball machine runs on coins, and the words “In God We Trust” are incidental to the mechanism.

The industry has a choice. It can keep selling chains of thought as auditable reasoning, which invites the next round of “complete accuracy collapse” findings and the whiplash cycle Quanta documents. Or it can treat reasoning traces as what they appear to be: useful scaffolding that sometimes helps, sometimes does nothing, and occasionally misleads. The honest framing would acknowledge that LRMs are powerful pattern machines that sometimes produce correct results for reasons we do not fully understand.

Kambhampati is not negative, he insists. “I just sound negative because everybody else is way too positive.” That asymmetry is the real story. The field has spent two years celebrating results while treating the mechanism as settled. The science is not settled, and the researchers doing the unglamorous work of testing the mechanism keep finding the same thing: the traces are not the reasoning.