Gemma 4 ships with reasoning turned on, and on short verifiable tasks that default cost me up to 48 accuracy points and 6x to 8x the money. One flag fixed it. The 12B and the 26B went from about 52% to 100% on my easy suite, and a right answer got up to 19x faster.
I measured 27 local models on a Strix Halo with a power meter running. Each one answered 190 small tasks with checkable answers, and I priced the electricity at my California summer rates: 34 cents per kilowatt-hour off-peak and 58 cents on-peak. I ran every task twice, under two different prompts.
Tell the model it can think
I ran every task once with the instruction to answer immediately with no explanation, and once with the instruction to work through the problem however it liked and put the answer on the last line.
| Model | Told to answer immediately | Told it can think | Gain |
|---|---|---|---|
| qwen3-30b-a3b-instruct | 56.8% | 97.9% | +41 points |
| mistral-small-24b | 57.9% | 98.9% | +41 |
| qwen3-coder-next | 61.1% | 97.9% | +37 |
| devstral-2-123b | 62.1% | 98.9% | +37 |
| qwen3-4b-instruct | 47.4% | 83.2% | +36 |
| gemma-3-27b | 62.1% | 97.9% | +36 |
| llama4-scout | 56.8% | 92.6% | +36 |
| gemma-3-12b | 53.7% | 86.3% | +33 |
| qwen3-next-80b-a3b | 61.1% | 91.6% | +31 |
| llama31-8b | 46.3% | 73.7% | +27 |
| gemma-3-4b | 48.4% | 71.6% | +23 |
| phi-4 | 62.1% | 84.2% | +22 |
The gain runs from 22 to 41 points, and the token counts stay in the low hundreds, so it barely moves the cost. qwen3-30b-a3b-instruct goes from a coin flip to 98% because I stopped telling it to be brief.
Letting a model think costs almost nothing. Running a model trained to think costs about 6x more per right answer. qwen3-30b-a3b-thinking costs 5.8x its own instruct sibling on the same tasks, and qwen3-4b-thinking costs 6.4x its sibling. On these tasks, that extra spend returned nine accuracy points at 4B and minus three at 30B.
One caution. Room to think pays off when a model uses the room and then stops. trinity-nano picked up nine points from the invitation but went from 61 tokens a task to 1,181 to get them, a 19x spend for a nine-point gain. phi-4-reasoning went down 26 points, because the free-form prompt let it spiral past the token cap on tasks it had answered correctly when forced to be brief.
Switch Gemma 4's reasoning off with one flag
Gemma 4 cannot be talked out of reasoning. Asked to answer in one line with no explanation, it emitted reasoning on 95 of 95 tasks in both prompt conditions. On my 4,096-token budget, the 12B and the 26B spent the whole thing thinking and handed back nothing on roughly half the questions.
You can switch it off. Use --reasoning off in llama.cpp, or enable_thinking: false through the chat template. Both worked on my build. An open upstream issue reports the opposite, though it is titled for Qwen3.5 and the Gemma 4 reports inside it contradict each other. Check yours rather than trusting any of us. If the flag took, tokens per answer fall off a cliff and you will see it in the first response.
| Model | Reasoning | Pass | Tokens/answer | $/1k | Seconds/answer |
|---|---|---|---|---|---|
| gemma-4-12b | on | 52.6% | 4,653 | 1.208 | 254.5 |
| gemma-4-12b | off | 100.0% | 256 | 0.185 | 15.6 |
| gemma-4-26b-a4b | on | 51.6% | 5,016 | 1.063 | 195.8 |
| gemma-4-26b-a4b | off | 100.0% | 245 | 0.130 | 10.3 |
| gemma-4-31b | on | 85.3% | 1,782 | 2.468 | 218.5 |
| gemma-4-31b | off | 100.0% | 199 | 0.394 | 28.7 |
One flag made Gemma 4 up to 48 accuracy points better, 6x to 8x cheaper, and up to 19x faster on short verifiable work. All three reach a perfect score. This is a lopsided gain: the 12B and 26B jump about 48 points each, and the 31B only 15, because it was less budget-starved to begin with. The prompting trick above was the best thing I'd found until this, and it tops out at 41 points.
Reasoning itself is worth having. Google publishes GPQA Diamond numbers for all three models, 78.8, 82.3, and 84.3, against the 61, 67, and 70 I measured with reasoning off. Some of that gap comes from my Q4_K_M weights and my 100-question subset rather than the flag, and I never ran GPQA with reasoning on to separate them.
The useful version isn’t "turn it off." The default suits one of two regimes. On short verifiable work, a model that reasons burns your budget solving something that wasn't hard. On genuinely hard work, you want it back on. Which side of that line your workload sits on decides the setting, so check it before you blame the model.
One trap sits next to this flag. --reasoning-budget 0 looks like it does the same job and doesn't. It empties the API's reasoning field while the model keeps reasoning in the ordinary content field, so anything counting reasoning tokens reports silence.
Also in this series: what a correct answer costs at each difficulty level, across all 27 models.
Check whether your model stops talking when told to
The terse prompt says to answer immediately, in one line, with no explanation. Twelve of the 27 models emitted reasoning anyway, and 11 of those did it on all 95 tasks without an exception. Here they are, sorted by what they spend:
| Model | Terse tasks with reasoning | Avg tokens per terse answer | In the reasoning field? |
|---|---|---|---|
| phi-4-reasoning | 0/95 | 2,811 | no |
| gemma-4-26b-a4b | 95/95 | 1,664 | yes |
| qwen3-4b-thinking | 95/95 | 1,411 | yes |
| gemma-4-12b | 95/95 | 1,403 | yes |
| qwen3.6-35b-a3b | 95/95 | 1,184 | yes |
| trinity-mini | 95/95 | 1,066 | yes |
| ornith-35b | 95/95 | 932 | yes |
| gemma-4-31b | 95/95 | 905 | yes |
| qwen3-30b-a3b-thinking | 95/95 | 884 | yes |
| qwen3-8b | 25/95 | 260 | yes |
| nemotron-3-super-120b | 95/95 | 245 | yes |
| gpt-oss-20b | 95/95 | 243 | yes |
| gpt-oss-120b | 95/95 | 192 | yes |
Look at the top row. phi-4-reasoning is the loudest model on the box, and it is not one of the 12, because it streams its thinking as ordinary output instead of into the API's separate reasoning field. A reasoning-field check calls it obedient while it writes 2,811 tokens in reply to "one line, no explanation." If I had trusted that one signal, I would have called the worst offender clean.
The models that obey answer in 14 to 15 tokens: llama4-scout, granite4-h-small, phi-4, and qwen3-next-80b-a3b. That is 14 tokens against phi-4-reasoning’s 2,811, on identical prompts. A model you cannot quiet has a cost floor you cannot design around, and the gap between the quietest and loudest here is 200x. If you pay per token in an agent loop or hold a latency budget, check this before you build on a model.
The gpt-oss pair and nemotron barely have this problem. They emit reasoning but spend a couple of hundred tokens on it. The models where it hurts combine stubbornness with volume.
Three ways expensive models fail
The models at the bottom of the cost table in the first post fail in three ways.
Verbosity. Every model in the bottom third of the cost table emits over 1,000 tokens per correct answer, and several emit over 2,000. They are mostly models trained to reason, and the reasoning draws power for every token.
Truncation. I capped generations at 4,096 tokens. phi-4-reasoning hit that cap on 78% of the free-mode tasks. It computed 4,000 tokens, drew power for all of them, and produced no usable answer. Those tokens cost exactly as much as productive ones, which is most of why it sits at $5.57 per thousand answers.
Spiraling. The worst case is a model that gets stuck. trinity-nano scored zero out of 14 on the constraint puzzles when I let it reason freely, because 12 of the 14 ran to the token cap producing text like "Let me think about this differently. Let's see if there's a way to arrange them..." and never reached an answer. It averaged 3,574 tokens on those 14 for nothing. The same model, told to answer immediately, averaged 10 tokens and got three of them right. A model that spirals generates tokens the whole time, so every throughput metric says it's working.
Leave the token budget alone
The obvious objection to the truncation numbers is that I set the cap too low and measured my own configuration. I re-ran the worst offender in the set, plus both Arcee models, with 4x the room.
| Model | Cap | Pass | Hit the cap | Tokens/answer | J/answer |
|---|---|---|---|---|---|
| trinity-mini | 4,096 | 90.5% | 3% | 1,181 | 788 |
| trinity-mini | 16,384 | 90.5% | 6% | 1,535 | 2,081 |
| trinity-nano | 4,096 | 38.9% | 25% | 3,031 | 1,004 |
| trinity-nano | 16,384 | 43.2% | 22% | 4,488 | 2,895 |
| phi-4-reasoning | 4,096 | 15.8% | 78% | 21,424 | 34,580 |
| phi-4-reasoning | 16,384 | 14.3% | 71% | 81,613 | 320,017 |
More room gained nothing and cost 2.5x to 9x as much. These models fill whatever space they are given. trinity-nano still hit a cap 4x larger on 22% of tasks instead of 25%, and picked up four tasks, which is the size of my run-to-run noise.
For trinity-mini, this is a tuning result. The tight budget was better on both axes: same accuracy for a third of the energy. If you run it, cap it. trinity-nano here is Trinity-Nano-Preview, a preview release measured against finished models, so weigh that before drawing conclusions about it. trinity-mini also scores the same whichever way you prompt it: 91.6% told to be brief against 90.5% told it could think. Nearly every conventional instruct model here swings 22 to 41 points on that one instruction, so if you can't control how every call gets phrased, that insensitivity is worth money. Both Arcee models also generate cheap tokens. trinity-nano has the lowest per-token cost I measured at 0.33 J, and trinity-mini is fifth of 27. What costs trinity-mini money is how many tokens it chooses to spend, and that is the configurable kind of problem.
The phi-4-reasoning row also exposed a llama.cpp trap that cost me a mistake. My first attempt at the larger budget asked for 16,384 tokens and got 7,903. The server's -c is the total context divided across --parallel slots, so a 32,768-token context with four slots gives each request 8,192, and the cap I asked for was never reachable. Every generation topped out at the same place, and I nearly published that as a measurement. If you set a generation cap, check it against your context divided by your slot count.
The next post in this series looks at why tokens per second tells you almost nothing about what an answer costs.
