---
title: "ThinkingCap: Gemma-4-26B-A4B — up to 69% and on average 39% less thinking, at unchanged accuracy"
slug: thinkingcap-gemma-4-26b-a4b-up-to-69-and-on-average-39-less-thinking-at-unchanged-accuracy
url: https://listedarticles.com/articles/thinkingcap-gemma-4-26b-a4b-up-to-69-and-on-average-39-less-thinking-at-unchanged-accuracy
canonical_url: https://bottlecapai.com/post/thinkingcap-gemma-4-26b/
content_type: announcement
language: en
published_at: 2026-10-23T00:00:00.000Z
updated_at: 2026-10-23T17:09:48.436Z
author: "BottleCap AI"
authored_by: human
publisher: "BottleCap AI"
publisher_url: https://bottlecapai.com/
topics: ["AI", "LLMs", "Open Source"]
license: all-rights-reserved
word_count: 1385
reading_minutes: 6
citation: "BottleCap AI, BottleCap AI. \"ThinkingCap: Gemma-4-26B-A4B — up to 69% and on average 39% less thinking, at unchanged accuracy.\" 23 Oct 2026. https://bottlecapai.com/post/thinkingcap-gemma-4-26b/ (all-rights-reserved)"
# The full text follows. The web page shows an extract and sends readers
# to the source above; quote the citation and link the canonical URL.
---

# ThinkingCap: Gemma-4-26B-A4B — up to 69% and on average 39% less thinking, at unchanged accuracy

> BottleCap AI releases the fifth ThinkingCap model, a fine-tune of Google's gemma-4-26B-A4B-it that cuts thinking tokens by 38.9% on average (up to 69.1%) across twelve benchmarks for a 0.15pp accuracy cost, with looping reduced from 3.6% to 0.8%.

# ThinkingCap: Gemma-4-26B-A4B — up to 69% and on average 39% less thinking, at unchanged accuracy

![ThinkingCap Gemma-4-26B-A4B](https://bottlecapai.com/assets/thinkingcap-gemma-4-26b/press-thinkingcap-gemma-4-26b.jpg)

## TL;DR

This is the fifth model in the ThinkingCap series. We reduced how much Google's [gemma-4-26B-A4B-it](https://huggingface.co/google/gemma-4-26B-A4B-it) thinks, without compromising the quality of its answers.

- **Up to 69.1% reduction in thinking tokens, on average 38.9%** measured across twelve benchmarks, for **0.15pp of accuracy**
- The first ThinkingCap on a Google model, and the smallest accuracy cost of any ThinkingCap so far
- Looping drops from 3.6% of the original model's responses to 0.8% of ours
- Same sampling settings (thinking on, temperature 1.0, top\_p 0.95), same serving stack (vLLM), drop-in replacement


## Our fifth ThinkingCap

We have published four token-efficient models so far: [ThinkingCap-Qwen3.6-27B](https://bottlecapai.com/post/thinkingcap-qwen3-6-27b/), [ThinkingCap-Qwen3.8-27B](https://bottlecapai.com/post/thinkingcap-qwen3-8-27b/), [ThinkingCap-NVIDIA-Nemotron-3.5-Lightning-30B-A3B](https://bottlecapai.com/post/thinkingcap-nemotron-3-5-lightning/) and [ThinkingCap: Qwen3.8-Flash-Next](https://bottlecapai.com/post/thinkingcap-qwen3-8-flash-next/).

gemma-4-26B-A4B is Google's sparse mixture of experts — 26B parameters with about 4B active per token. It is the third model family we have applied this work to, after Qwen and NVIDIA. On this one we put extra effort into keeping the accuracy intact: thinking comes down 38.9% on average at a cost of only 0.15pp of accuracy, which is smaller than the run-to-run spread on most of the suite.

*Video comparison of the original model and the ThinkingCap model, prompted with the same input.  
Both models run on the same hardware and use the same inference settings.*

## The training objective

We start with [gemma-4-26B-A4B-it](https://huggingface.co/google/gemma-4-26B-A4B-it). The goal is to spend fewer tokens getting to the answer, and change nothing else: preserve the knowledge, reasoning ability, answer quality, instruction following and safety behaviour.

## Results

![Accuracy per task family, gemma-4-26B-A4B-it against the ThinkingCap model](https://bottlecapai.com/assets/thinkingcap-gemma-4-26b/accuracy.svg)

*Accuracy per task family, gemma-4-26B-A4B-it against the ThinkingCap model*

Accuracy barely moves in any family, and in two it moves up. Instruction following and agentic
gains 0.65pp and maths gains 0.39pp; code is flat at 0.02pp; knowledge and reasoning costs
0.51pp and long-context and multimodal 0.92pp. Averaged over all twelve benchmarks the change
is **−0.15pp**.

![Mean thinking tokens per task family, gemma-4-26B-A4B-it against the ThinkingCap model](https://bottlecapai.com/assets/thinkingcap-gemma-4-26b/mean-thinking-tokens.svg)

*Mean thinking tokens per task family, gemma-4-26B-A4B-it against the ThinkingCap model*

The thinking is where the change is, and every family gets shorter. Long-context and multimodal
reduces the most, 54.4%, then knowledge and reasoning at 44.4%, maths at 36.1% and code at
27.6%; instruction following and agentic reduces the least at 17.2%. Averaged over the twelve
benchmarks it is **38.9%**.

| Benchmark | Accuracy (base) | Accuracy (ours) | Δ accuracy | Mean thinking tokens | **Δ mean thinking** | Trunc. % | Looping % |
| --- | --- | --- | --- | --- | --- | --- | --- |
| **Math** | | | | | | | |
| `aime26` | 86.20% ±0.86 | 86.72% ±0.98 | +0.52pp | 14766 → 8963 | -39.3% | 5.2 → 0.5% | 5.5 → 0.9% |
| `hmmt_nov25` | 81.51% ±1.06 | 80.89% ±1.19 | -0.62pp | 16462 → 10542 | -36.0% | 9.0 → 1.2% | 8.6 → 1.2% |
| `hmmt_feb26` | 72.92% ±1.08 | 74.20% ±1.16 | +1.28pp | 22005 → 14717 | -33.1% | 18.6 → 4.6% | 17.0 → 4.5% |
| **Code** | | | | | | | |
| `livecodebench` | 76.98% ±0.56 | 76.96% ±0.67 | -0.02pp | 11560 → 8375 | -27.6% | 6.8 → 1.3% | 2.2 → 0.3% |
| **Knowledge & reasoning** | | | | | | | |
| `gpqa_diamond` | 79.92% ±0.57 | 78.96% ±0.67 | -0.96pp | 7699 → 4284 | -44.4% | 0.3 → 0.1% | 0.3 → 0.1% |
| `mmlu_pro` | 84.55% ±0.29 | 84.64% ±0.16 | +0.09pp | 3907 → 1989 | -49.1% | 0.0 → 0.0% | 0.1 → 0.0% |
| `mmmlu` | 84.09% ±0.26 | 83.43% ±0.27 | -0.66pp | 1212 → 731 | -39.7% | 0.0 → 0.0% | 0.0 → 0.0% |
| **Long-context & multimodal** | | | | | | | |
| `aa_lcr` | 64.62% ±2.49 | 60.62% ±2.89 | -4.00pp | 6519 → 2655 | -59.3% | 4.0 → 0.0% | 3.6 → 0.4% |
| `longbench_v2` | 59.43% ±1.14 | 60.88% ±1.29 | +1.45pp | 9486 → 2934 | -69.1% | 7.7 → 0.5% | 5.9 → 1.2% |
| `realworldqa` | 70.83% ±0.56 | 70.61% ±0.56 | -0.21pp | 819 → 533 | -34.9% | 0.0 → 0.0% | 0.0 → 0.0% |
| **Instruction following & agentic** | | | | | | | |
| `ifbench` | 76.83% ±0.80 | 78.67% ±0.73 | +1.83pp | 6470 → 5585 | -13.7% | 0.4 → 0.4% | 0.5 → 0.5% |
| `tau2` | 64.25% ±0.97 | 63.73% ±1.26 | -0.53pp | 4374 → 3466 | -20.8% | 0.0 → 0.0% | 0.0 → 0.0% |
| Average (12 benchmarks) | 75.18% | 75.03% | -0.15pp | 8773 → 5398 | -38.9% | 4.35 → 0.72% | 3.64 → 0.76% |

Every row is the original BF16 against the ThinkingCap model, run on the same
harness with the same settings on the same questions. Accuracy is the mean over seeds, ± the
95% confidence interval from the spread between them. Every cell in the last row is an average
across the twelve benchmarks, each counting once — so **Δ mean thinking** there is the mean of
the twelve reductions, not the ratio of the two token figures beside it, which is 38.5%. The
same distinction applies to the family bars above, which plot each family's mean tokens. On
τ²-bench the thinking tokens are per task, summed across the rollout's turns, rather than per
response. Column definitions are in [Evaluation methodology](#evaluation-methodology).

Every one of the twelve gets shorter. The deepest cuts are LongBench v2 (−69.1%), AA-LCR
(−59.3%) and MMLU-Pro (−49.1%); the shallowest IFBench (−13.7%) and τ²-bench (−20.8%). The two
shallow rows are the two whose work is least about thinking at length: following a list of
format constraints, and applying a fixed customer-service policy through the right tool calls.

On accuracy, most of the results are within noise: on eight of the twelve the gap is smaller
than the run-to-run spread, and five of the twelve come out *ahead* of the original model. The
four that move further than their intervals allow are IFBench (+1.83pp, in our favour), GPQA-Diamond
(−0.96pp), MMMLU (−0.66pp) and AA-LCR (−4.00pp). AA-LCR is the one real cost in the suite.

Looping improves across the suite. The original model loops on **3.64%** of its responses,
against **0.76%** for ours — and on the three benchmarks where
it loops most, AIME 2026, HMMT Nov 25 and HMMT Feb 26, the rate falls from 5.5%, 8.6% and 17.0%
to 0.9%, 1.2% and 4.5%.

![Accuracy against the thinking-token budget, both models](https://bottlecapai.com/assets/thinkingcap-gemma-4-26b/budget-curve.svg)

*Accuracy against the thinking-token budget, both models*

Read left to right, this is the question a deployment actually asks: for a given token budget
per question, how much accuracy do you get? ThinkingCap's curve sits above the original model's
through the whole middle of the range — the same accuracy for a smaller budget — and the two
converge once the budget is wide enough that neither model is constrained by it.

## Evaluation methodology

Reasoning models are noisy to evaluate. At the recommended temperature of 1.0 both the content
and the length of a response move run to run, so a single seed tells you very little. Everything
below is multiple seeds per benchmark, with accuracy reported as the mean and a 95% confidence
interval across seeds.

**Both sides are the same run.** The original model and ours were evaluated by the same harness,
with identical sampling and serving settings. The original model was measured by us, not quoted
from published figures. The configuration:

- **Harness** — our internal evaluation suite, running both models on vLLM.
- **Context** — generation capped at 65,536 tokens per response on every benchmark, and per turn on τ²-bench.
- **Sampling** — thinking on; temperature 1.0, top\_p 0.95, top\_k 64. Ours inherits Google's recommended preset unchanged, so no part of the comparison is a decoding change.
- **Accuracy** — the mean across seeds with a 95% t-interval, which says how far the answer moves when only the sampling seed changes.
- **Seeds** — 64 on AIME 2026, HMMT Nov 2025 and HMMT Feb 2026; 32 on GPQA-Diamond and LiveCodeBench; 16 on RealWorldQA, IFBench and τ²-bench; 8 on AA-LCR and LongBench v2; 4 on MMLU-Pro and MMMLU, whose question counts are large enough that question sampling is not the limiting noise.
- **Question sets** — AIME 2026 (30 problems), HMMT Nov 2025 (30), HMMT Feb 2026 (33), LiveCodeBench (175), GPQA-Diamond (198), MMLU-Pro (12,032), MMMLU (9,996), AA-LCR (100), LongBench v2 (370), RealWorldQA (765), IFBench (300) and τ²-bench (278 tasks across airline, retail and telecom).
- **Scoring** — exact match on the extracted answer, except LiveCodeBench, where the generated code is run against the problem's tests; IFBench, scored by its programmatic verifiers; and τ²-bench and AA-LCR, which are judged. τ²-bench also runs against a simulated user and is reported as the unweighted mean over its three domains.

### What the columns mean

- **Accuracy (base)** / **Accuracy (ours)** — as above; **Δ accuracy** is the difference between them, in percentage points.
- **Δ mean thinking** — the headline: the change in thinking tokens, totalled across the benchmark and divided by its question count, so a long question weighs more than a short one.
- **Mean thinking tokens** are the tokens before the reasoning closes, averaged over every question and seed of the benchmark, written `base → ours`.
- **Truncated %** — the share of responses whose thinking never closed, which score zero. **Looping %** — the share caught by a compression-based loop detector, which flags a response whose text compresses far beyond what ordinary prose does.

## Get the model

You can find the model here: [bottlecapai/ThinkingCap-gemma-4-26B-A4B-it](https://huggingface.co/bottlecapai/ThinkingCap-gemma-4-26B-A4B-it), with [GGUF builds](https://huggingface.co/bottlecapai/ThinkingCap-gemma-4-26B-A4B-it-GGUF) published alongside it.