Chain-of-Thought Reasoning in the Wild Is Not Always Faithful (2025)
arxiv.org
August 19, 2026
2 min read
44/100
Summary
A study submitted to arXiv on March 11, 2025 finds that language models can produce chain-of-thought explanations that do not faithfully reflect the processes behind their answers, even when prompts are naturally worded and contain no added bias. The latest version was revised on June 16, 2026. When models answered separately phrased comparisons such as “Is X bigger than Y?” and “Is Y bigger than X?”, they sometimes gave coherent-sounding justifications for answering yes to both questions or no to both, despite the logical contradiction. The researchers call the apparent tendency to rationalize an implicit preference for yes or no “Implicit Post-Hoc Rationalization.” They recorded unfaithful reasoning rates of up to 13% among production models. Frontier systems performed better but were not fully faithful: DeepSeek R1 had a reported rate of 0.37%, while Sonnet 3.7 with thinking had a reported rate of 0.04%. The researchers also identify “Unfaithful Illogical Shortcuts,” in which subtly invalid reasoning makes speculative solutions to difficult mathematics problems appear rigorous. They conclude that chain-of-thought output can help assess model answers but does not fully represent the internal process that generated them, creating risks for agentic and safety-critical uses.
Key Takeaways
- Language models can generate plausible chain-of-thought explanations while giving mutually contradictory answers to reversed comparison questions.
- The researchers attribute some contradictory responses to implicit yes-or-no biases and call the behavior Implicit Post-Hoc Rationalization.
- The study reports unfaithful chain-of-thought rates of up to 13% in production models; DeepSeek R1 and Sonnet 3.7 with thinking were reported at 0.37% and 0.04%, respectively.
- Subtly illogical reasoning can make speculative answers to difficult math problems appear rigorously proven, according to the researchers.
- Chain-of-thought output is not a complete account of the internal process producing a model answer and warrants caution in agentic or safety-critical settings.
What the discussion said
The thread mostly treated the paper as confirmation of an uncomfortable fact already visible in everyday use of reasoning-enabled chatbots: a model can produce a convincing multi-step explanation, visibly brush against the correct diagnosis, and still land on an obviously wrong final answer. Several readers welcomed the study precisely because it turns that familiar failure mode into a testable question about whether equivalent prompts trigger stable conclusions. The strongest consensus was that displayed chain-of-thought is an output channel, not privileged access to the computation that selected the answer. Readers objected especially to language that makes intermediate tokens sound like human thought, arguing that this framing encourages people to infer too much from fluent prose. At the same time, the discussion did not settle on the simpler claim that traces are useless theater. Some argued that generated reasoning can causally steer the answer, remain coherent under training, and provide practical debugging clues even if its literal narrative is partly post-hoc. The paper’s author reinforced the concern by noting that plausible after-the-fact rationales arise even on easy tasks, while another reader noted that humans also rationalize after reaching conclusions. Overall, commenters saw the work as valuable empirical scrutiny of an AI behavior that should never have been trusted at face value.
Where opinion split
The sharp dispute is whether visible reasoning traces are merely retrospective rationalizations or meaningful parts of model inference. Skeptics say fluent intermediate text has no guaranteed relationship to the hidden mechanisms choosing an answer, so treating it as thought is a category error. The opposing view is that traces can both mislead and influence final outputs, making them imperfect but still experimentally and operationally useful.
Community Sentiment
Positives
- The paper gives a rigorous name to a common chatbot failure: an apparently sound derivation can recognize the crucial issue yet still emit the wrong binary conclusion.
- Testing semantically equivalent question forms could expose whether a model has stable underlying competence rather than brittle template-driven answer habits.
- Even unfaithful reasoning traces may remain useful debugging signals, because changing or omitting them can alter the model’s final response.
Concerns
- Polished chain-of-thought can be post-hoc justification, letting a model sound as if it reasoned its way to an answer already favored by prompt-pattern bias.
- A correct-looking derivation followed by a plainly wrong final answer makes exposed reasoning dangerous as evidence of reliability in high-stakes AI use.
- Calling intermediate tokens thought or reasoning invites anthropomorphism and disguises the gap between readable text and the model’s actual computation.
- The phenomenon appears common enough, including on easy tasks, that chain-of-thought visibility alone should not be marketed as interpretability.
Related Articles
VibeThinker: 3B param model that beats Opus 4.5 on reasoning with novel SFT+GRPO
Jun 23, 2026
Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning
Jul 16, 2026
Frontier AI agents violate ethical constraints 30–50% of time, pressured by KPIs
Feb 10, 2026
Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence (2025)
Aug 5, 2026
Psychometric Jailbreaks Reveal Internal Conflict in Frontier Models
Feb 5, 2026