--- title: "A new reasoning paper cuts tokens by 62.6% and raises accuracy. Neither number comes with a variance." description: "State of Thought reports a 62.6% token cut with an accuracy gain, and no seeds, run counts or error bars on either. Four questions worth asking of any paper in this band." publishedAt: 2026-09-21 author: Ena Pragma url: https://enapragma.co/field-notes/state-of-thought-cuts-tokens-reports-no-variance tags: ["reasoning", "research", "evaluation"] --- State of Thought (arXiv `2609.16055`, submitted 13 September 2026, NTU Singapore and KTH Sweden) proposes that a language model already carries a usable representation of *where it is* in a reasoning problem, and that you can read the control signal out of the model's own internal geometry instead of imposing it from outside. A 582-parameter controller does the reading. The reported result is a 62.6% reduction in reasoning tokens with a 3.8 point accuracy gain on the vision-language benchmark. The paper is eight days old and nobody has reproduced it. This note is not a verdict on whether it works. It is about the question a reader should ask of it, which is cheaper than a verdict and stays useful after one arrives. ## Why the efficiency number is the important one, and not for the reason you would expect Efficiency results in this literature are normally read as a convenience benefit: same answers, smaller bill. Here it is doing harder work than that. The standing null hypothesis for reasoning gains, stated plainly in the 2026 reassessment literature, is that most apparent improvements are better explained by generic extra serial compute than by any special representational object the method claims to have found. More thinking, more tokens, better answer. Nothing interesting inside. If accuracy rises *while* token count falls by 62.6%, that explanation stops working. You cannot attribute the gain to extra compute when the method used substantially less of it. So the efficiency figure is not a nice-to-have alongside the accuracy figure. It is the load-bearing evidence that something mechanistic is happening at all. That is our reading rather than the paper's, and it is worth stating as a reading. But it inverts how these results are usually skimmed, and it raises the stakes on the numbers considerably. ## The gap That argument holds only if the accuracy gains are real. And the paper reports **no seeds, no run counts, no standard deviations and no error bars on any main accuracy or efficiency result.** Exactly one dispersion figure appears anywhere in it: a selector AUC of 0.779 plus or minus 0.005 across five problem-held-out refits. That is a real measurement of one auxiliary component. It is not a measurement of the headline claims. This matters because the field has already published the test. The 2025 reassessment "A Sober Look at Progress in Language Model Reasoning" demonstrated that reported gains in this exact literature frequently fail to survive a change of random seed. The finding is not that reasoning papers are wrong. It is that the variance is often larger than the improvement, and a single run cannot tell you which you are looking at. So the question a reader should carry to State of Thought is not whether 62.6% is impressive. It is: measured against what, how many times, and with what spread. ## Two things the paper gets right, which a fair reading has to say It would be easy to file this as another sloppy benchmark paper. It is not one, and two specifics say so. **It names thirteen baselines**, spanning four different families: prompting (Chain-of-Thought, Plan-and-Solve, Self-Refine, Self-Consistency), search (Best-of-N, Constrained Beam, Monte Carlo Tree Search), KV-cache compression (H2O, SnapKV, StreamingLLM), and latent reasoning (COCONUT, GRPO-Soft Prompt). That is a broader comparison set than this literature usually carries. **It evaluates on sixteen standard datasets**, and AIME is not among them. That detail matters more than it looks. The sharpest overfitting finding in the Sober Look reassessment is specific to very small benchmarks like AIME'24, where a handful of problems can swing a headline. Whatever else is true here, that particular criticism does not transfer. A reporting gap and a sloppy paper are different things. This is the first. ## The citation problem Two papers are missing from the references, and both are load-bearing. **State Machine of Thoughts** (arXiv `2312.17445`, December 2023) records past reasoning trajectories, successes and failures alike, in a state machine, and selects optimal sub-solutions from it. Reuse of past trajectories selected by state therefore predates State of Thought by roughly three years. A full-text search of the new paper returns no citation to it, to its authors, or to the SMoT acronym. That absence is sharper than a missing reference usually is, because State of Thought positions itself specifically against *externally imposed* control. A hand-written state machine is precisely that. SMoT is not a competitor the paper overlooked; it is the version of the idea the paper defines itself in opposition to, and it is not discussed. **Sketch-of-Thought** (arXiv `2503.05179`, March 2025) is also uncited, and it shares the new paper's acronym, its efficiency goal, and its architecture of a small auxiliary selector operating at test time. It reported up to 84% token reduction eighteen months earlier. On the acronym: the Sketch-of-Thought collision we read first-hand. Other "SoT" expansions are reported in search results and we did not verify them, so treat the collision as one confirmed and others unchecked. None of this means the new work is unoriginal. The genuinely novel element, as best we can tell from comparing the abstracts, is narrow and real: the *origin* of the control signal, read from internal model geometry rather than supplied by a router, a prompting paradigm, or an author-written state machine. That reading is ours and no source asserts it. But "narrow and real" is a much smaller claim than a paper with no prior art, and the reader deserves to know which one they are being offered. ## The band Five methods since March 2025 report reasoning-token reductions between 57% and 84%. State of Thought's 62.6% sits in the middle of that range. That sentence is arithmetic and each figure is the paper's own headline claim. What it is not is a ranking. These methods use different backbones, different datasets, different baselines and different accuracy measures, and no like-for-like comparison between State of Thought and either of the two nearest neighbours exists, because neither is cited. Nobody has run it. It is the obvious next experiment and it is unclaimed. So the percentage tells you less than it appears to. Which is the point. ## The checklist If the field's own reassessment is right that gains here frequently evaporate under a seed change, then the useful skill is not judging papers. It is knowing what to ask of one. Four questions, none of which require you to reproduce anything: 1. **Measured against what?** Named baselines, across more than one family of method. 2. **On what data?** Standard benchmarks large enough that a handful of items cannot move the headline. 3. **How many runs, and what was the spread?** Seeds, run counts, error bars. A single number with no variance is a sample of one. 4. **What is the nearest prior work, and is it cited?** Both a shared mechanism and a shared name are worth checking. State of Thought answers the first two well, does not answer the third, and misses two papers on the fourth. That third question is the one we keep coming back to: we have written before about why [accuracy is the wrong bar](/field-notes/accuracy-is-the-wrong-bar) when the spread is never reported. That is a more useful description of it than good or bad, and you can produce it in an afternoon without a GPU. It also separates the five methods in that 57-to-84% band far better than their percentages do. ## Sources The paper under discussion: - [State of Thought Enables Endogenous Reasoning](https://arxiv.org/abs/2609.16055) (`2609.16055`, 13 Sep 2026, NTU Singapore and KTH) The two uncited papers: - [State Machine of Thoughts](https://arxiv.org/abs/2312.17445) (`2312.17445`, Dec 2023), the direct prior art for reuse of past trajectories selected by state - [Sketch-of-Thought](https://arxiv.org/abs/2503.05179) (`2503.05179`, Mar 2025), same acronym, same efficiency goal, up to 84% reduction The test the field has already published: - [A Sober Look at Progress in Language Model Reasoning](https://arxiv.org/abs/2504.07086) (`2504.07086`), gains in this literature frequently do not survive a seed change - [LLM Reasoning Is Latent, Not the Chain of Thought](https://arxiv.org/abs/2604.15726) (`2604.15726`), the serial-compute null hypothesis this result has to beat The rest of the efficiency band: - [A State-Transition Framework for Efficient LLM Reasoning](https://arxiv.org/abs/2602.01198) (`2602.01198`, Feb 2026) - [Think Less, Know More / STACK](https://arxiv.org/abs/2604.09150) (`2604.09150`, Apr 2026), 59.9% reduction, +4.8 points - [CopT](https://arxiv.org/abs/2605.20075) (`2605.20075`, May 2026), up to 57% reduction Related work on stateful and latent reasoning: - [Latent Reasoning with Supervised Thinking States](https://arxiv.org/abs/2602.08332) (`2602.08332`) - [Stateful Reasoning via Insight Replay](https://arxiv.org/abs/2605.14457) (`2605.14457`)