Report · bottlecapai.com
Gemma-4-26B-A4B with 39-69% less thinking at same accuracy
bottlecapai.com
Source published October 23, 2026
Summary: 1 min read
53/100
Summary
Bottlecap AI released ThinkingCap-gemma-4-26B-A4B-it, a token-efficient version of Google’s Gemma-4-26B-A4B-it reasoning model. Gemma-4-26B-A4B is a sparse mixture-of-experts model with 26 billion parameters and about 4 billion active parameters per token. Bottlecap says its model uses 38.9% fewer thinking tokens on average across 12 benchmarks while accuracy declined by 0.15 percentage points, from 75.18% to 75.03%, under identical vLLM serving and sampling settings. Thinking-token reductions ranged from 13.7% on IFBench to 69.1% on LongBench v2. The model used 54.4% fewer tokens across long-context and multimodal tasks, 44.4% fewer on knowledge and reasoning, 36.1% fewer on math, and 27.6% fewer on code. Accuracy increased on some tests, including IFBench and LongBench v2, but fell 4.00 percentage points on AA-LCR. Bottlecap measured looping in 0.76% of the optimized model’s responses, down from 3.64% for the base model. Bottlecap evaluated both models with thinking enabled, temperature 1.0, top-p 0.95, top-k 64, and a 65,536-token generation cap. The open model and GGUF builds are available under the bottlecapai/ThinkingCap-gemma-4-26B-A4B-it repository. Bottlecap also offers a commercial PRO version, which it says cuts thinking tokens by up to 87.6% and 62% on average.
Key Takeaways
- Bottlecap AI’s ThinkingCap-gemma-4-26B-A4B-it reduced average thinking-token use by 38.9% across 12 benchmarks, while its measured average accuracy was 0.15 percentage points below Google’s base model.
- LongBench v2 showed the largest reported reduction in thinking tokens at 69.1%, while IFBench showed the smallest at 13.7%.
- The optimized model’s looping rate was 0.76%, compared with 3.64% for the base model across the evaluation suite.
- Bottlecap evaluated both models with identical vLLM infrastructure, generation limits, and sampling parameters, including temperature 1.0 and top-p 0.95.
- Bottlecap says its commercial ThinkingCap PRO version uses up to 87.6% fewer thinking tokens and 62% fewer on average.
What the discussion said
The thread treated the claimed reduction in reasoning tokens less as a Gemma-specific breakthrough than as another test of whether compressed chain-of-thought can survive contact with real workloads. Several readers who used similar reduced-thinking variants of Qwen and Kimi found the trade-off visible: answers became less capable, models declared work complete without actually finishing it, and one lighter variant was abandoned almost immediately for being materially weaker. Benchmark parity therefore drew little automatic trust; commenters argued that private tasks expose failures an aggregate score can conceal. Still, the economics have clear appeal. For PR review and other high-volume tasks, reasoning tokens dominate the bill, so preserving accuracy with shorter deliberation would be genuinely valuable. A fast, reasonably competent 26B model could also fit ordinary chat and routine coding questions even if it is not the right tool for autonomous coding agents or peak-quality work. Readers repeatedly stressed end-to-end time over flashy tokens-per-second figures: a model that thinks at length can lose to one that reaches a correct answer quickly, while faster inference engines may eliminate much of the reason to cut reasoning tokens locally. The practical consensus was conditional: test it on your own workload, because savings are attractive only if trust and task completion remain intact.
Where opinion split
The central dispute is whether removing most thinking tokens preserves useful intelligence or merely hides a quality collapse behind favorable benchmarks. Supporters see lower reasoning spend as a major win when routine work keeps its accuracy; skeptics say their hands-on tests show weaker reasoning, hallucinated completion, and enough lost reliability to erase any token savings.
Coverage & discussions
Hacker News · bottlecapai.com
Community discussions
Community Sentiment
Positives
- Shorter reasoning at unchanged accuracy would cut PR-review inference bills where hidden deliberation, not the final response, consumes most of the budget.
- A quick 26B-class model can be a strong fit for general chat and commonplace coding, where agent-grade planning is unnecessary and responsiveness matters.
- Readers see genuinely impressive value in small models that deliver strong humanities output, suggesting efficient models are already punching beyond their parameter count.
- End-to-end latency can improve without sacrificing capability when a model reaches the answer directly instead of spending minutes visibly deliberating.
Concerns
- Hands-on users of comparable reduced-thinking models report noticeably weaker answers than the full reasoning versions, making benchmark-level parity hard to trust on bespoke tasks.
- One reduced-reasoning approach repeatedly hallucinated task completion, a failure mode that makes token savings meaningless for work requiring dependable execution.
- The most aggressively shortened variant was judged substantially dumber and was dropped quickly, suggesting the efficiency knob can cross a sharp quality cliff.
- Some compressed variants appear more restrictive than their base models, reducing usefulness for requests the original model would handle rather than simply improving efficiency.
