A model that burns 13x more energy per token can be cheaper per right answer, and more than twice as accurate. On my Strix Halo, gemma-3-27b costs 729 joules per correct answer at 98% accuracy. trinity-nano, the most token-efficient model on the box, costs 1,004 joules at 39%.
I measured 27 local models with a power meter running, on 190 small tasks with checkable answers, each run under two prompts. Energy is priced at my California summer rates, 34 cents per kilowatt-hour off-peak and 58 cents on-peak. The usual efficiency numbers are tokens per second and energy per token. Neither predicts what an answer costs.
Knowing how efficiently a model produces tokens tells you almost nothing about what a right answer costs on this box. The rank correlation between energy per token and energy per correct answer across all 27 models is +0.27. A rank correlation of +1 means two lists are in the same order and 0 means one tells you nothing about the other, so +0.27 is close to nothing.
Rank models by tokens per answer
gemma-3-27b burns roughly 13x more energy per token than trinity-nano, and trinity-nano generates those tokens 9x faster. Both of the usual efficiency numbers say trinity-nano wins comfortably. The answer cost says otherwise, and the difference is the "tokens/answer" column. gemma-3-27b answers in 181 tokens. trinity-nano needs 3,031. You can generate tokens very efficiently and still spend a fortune if you generate thousands of them per useful answer.
What makes a model cheap here is how few tokens it spends before it is right: being terse and being correct together. Speed is a distant third.
One dollar of electricity at California summer on-peak rates buys this many correct answers on the easy tasks only. That is the number that misleads if you read it alone, because it says nothing about how a model behaves when the work gets harder. The first post in this series covers that.
| Model | Correct answers per $1 | How often it's right |
|---|---|---|
| gemma-3-4b | ~40,500 | 72% |
| qwen3-4b-instruct | ~32,400 | 83% |
| qwen3-30b-a3b-instruct | ~27,800 | 98% |
| gpt-oss-120b | ~17,500 | 100% |
| mistral-small-24b | ~9,000 | 99% |
| nemotron-3-super-120b | ~4,400 | 99% |
| devstral-2-123b | ~770 | 99% |
| phi-4-reasoning | ~179 | 16% |
That is a 225x spread between top and bottom, and you set it by typing a model name. phi-4-reasoning sits at the bottom because four of every five attempts burn their whole token budget and hand back nothing.
The full table follows, sorted by cost. "J/answer" is joules per correct answer, the raw measurement. The two dollar columns price that number at off-peak (34c/kWh) and on-peak (58c/kWh). Every task ran twice under two prompts, and this table uses the condition you would actually run, where the model is told it can work through the problem.
| Model | Pass | J/answer | $/1k off-peak | $/1k on-peak | Tokens/answer |
|---|---|---|---|---|---|
| gemma-3-4b | 71.6% | 153 | 0.014 | 0.025 | 241 |
| qwen3-4b-instruct | 83.2% | 192 | 0.018 | 0.031 | 347 |
| qwen3-30b-a3b-instruct | 97.9% | 224 | 0.021 | 0.036 | 242 |
| qwen3-next-80b-a3b | 91.6% | 296 | 0.028 | 0.048 | 216 |
| qwen3-coder-next | 97.9% | 301 | 0.028 | 0.048 | 217 |
| granite4-h-small | 66.3% | 320 | 0.030 | 0.052 | 66 |
| gpt-oss-20b | 94.7% | 342 | 0.032 | 0.055 | 360 |
| gpt-oss-120b | 100.0% | 354 | 0.033 | 0.057 | 225 |
| gemma-3-12b | 86.3% | 361 | 0.034 | 0.058 | 197 |
| llama31-8b | 73.7% | 414 | 0.039 | 0.067 | 441 |
| phi-4 | 84.2% | 421 | 0.040 | 0.068 | 252 |
| mistral-small-24b | 98.9% | 687 | 0.065 | 0.111 | 243 |
| gemma-3-27b | 97.9% | 729 | 0.069 | 0.117 | 181 |
| trinity-mini | 90.5% | 788 | 0.074 | 0.127 | 1,181 |
| llama4-scout | 92.6% | 836 | 0.079 | 0.135 | 211 |
| trinity-nano | 38.9% | 1,004 | 0.095 | 0.162 | 3,031 |
| qwen3-4b-thinking | 92.6% | 1,229 | 0.116 | 0.198 | 2,219 |
| qwen3-30b-a3b-thinking | 94.7% | 1,293 | 0.122 | 0.208 | 1,717 |
| nemotron-3-super-120b | 98.9% | 1,411 | 0.133 | 0.227 | 305 |
| qwen3.6-35b-a3b | 89.5% | 1,414 | 0.134 | 0.228 | 1,809 |
| qwen3-8b | 83.2% | 1,466 | 0.138 | 0.236 | 1,661 |
| ornith-35b | 96.8% | 2,098 | 0.198 | 0.338 | 1,410 |
| gemma-4-26b-a4b | 51.6% | 6,595 | 0.623 | 1.063 | 5,016 |
| gemma-4-12b | 52.6% | 7,500 | 0.708 | 1.208 | 4,653 |
| devstral-2-123b | 98.9% | 8,065 | 0.762 | 1.299 | 330 |
| gemma-4-31b | 85.3% | 15,321 | 1.447 | 2.468 | 1,782 |
| phi-4-reasoning | 15.8% | 34,580 | 3.266 | 5.571 | 21,424 |
Cost and accuracy do not line up anywhere in that table. The cheapest model is not the least accurate, and the most expensive is not the most accurate.
On this easy suite, 24 of the 27 are dominated: something else on the same box is no worse, once you allow for the four-task noise floor, and meaningfully cheaper. Three survive: gemma-3-4b, qwen3-4b-instruct, and qwen3-30b-a3b-instruct. Hold that lightly. "Dominated" means dominated at these questions, and the top three models on the hard questions all look dominated in this table.
Related in this series: What a correct answer costs on a Strix Halo.
Separate wrong answers from unfinished ones
A "pass" column lumps together two failures that mean opposite things. A model can finish and be wrong, which is a capability problem. Or it can run into my 4,096-token cap mid-generation, which is a budget problem, and it might well have been right.
| Model | Pass | Wrong | Unfinished |
|---|---|---|---|
| gpt-oss-120b | 100.0% | 0 | 0 |
| mistral-small-24b | 98.9% | 1 | 0 |
| qwen3-30b-a3b-instruct | 97.9% | 2 | 0 |
| gemma-3-27b | 97.9% | 2 | 0 |
| qwen3-next-80b-a3b | 91.6% | 8 | 0 |
| qwen3.6-35b-a3b | 89.5% | 0 | 10 |
| gemma-3-12b | 86.3% | 13 | 0 |
| gemma-4-31b | 85.3% | 0 | 14 |
| phi-4 | 84.2% | 15 | 0 |
| gemma-3-4b | 71.6% | 27 | 0 |
| granite4-h-small | 66.3% | 32 | 0 |
| gemma-4-12b | 52.6% | 0 | 45 |
| gemma-4-26b-a4b | 51.6% | 0 | 46 |
| trinity-nano | 38.9% | 34 | 24 |
| phi-4-reasoning | 15.8% | 21 | 74 |
Four models never gave a wrong answer across 95 tasks each. Every failure they had was me cutting them off.
Read granite4-h-small and gemma-4-26b-a4b together. Granite scores 66.3% and Gemma 51.6%, so the cost table ranks granite 15 points higher. Granite got 32 answers wrong. Gemma got none, and would not stop talking. A benchmark that ranks the frequently wrong model above the never-wrong one is measuring the wrong thing, and the models it penalizes are all reasoning-heavy, so the error is not spread evenly across the field.
For the three Gemma 4 rows, the pass column measures my cap. Their cost numbers hold, though. Being expensive because you won't stop talking is a property of the model.
Measure seconds per correct answer when you're waiting
If you sit in front of this box waiting, speed matters a lot, and a metric that ignores it is useless for choosing a model. The same question, asked about your time instead of your electricity bill, is not how fast a model emits tokens but how long until you have a right answer.
| Model | Pass | Tok/s per request | Seconds per correct answer | J/answer |
|---|---|---|---|---|
| gemma-3-4b | 71.6% | 46.1 | 5.6s | 153 |
| qwen3-4b-instruct | 83.2% | 53.0 | 6.9s | 192 |
| qwen3-30b-a3b-instruct | 97.9% | 30.0 | 8.0s | 224 |
| gpt-oss-20b | 94.7% | 30.8 | 11.6s | 342 |
| qwen3-coder-next | 97.9% | 18.5 | 12.1s | 301 |
| gpt-oss-120b | 100.0% | 15.9 | 14.5s | 354 |
| gemma-3-27b | 97.9% | 7.8 | 23.2s | 729 |
| mistral-small-24b | 98.9% | 11.0 | 23.3s | 687 |
| trinity-nano | 38.9% | 72.2 | 41.5s | 1,004 |
| qwen3.6-35b-a3b | 89.5% | 30.6 | 59.5s | 1,414 |
| nemotron-3-super-120b | 98.9% | 5.1 | 60.7s | 1,411 |
| devstral-2-123b | 98.9% | 2.1 | 164.5s | 8,065 |
| phi-4-reasoning | 15.8% | 17.6 | 1,241.2s | 34,580 |
Seconds per answer and joules per answer rank the models almost identically, at +0.98, so on this box the cheap models are also the fast ones. You have no trade to manage between your electricity bill and your patience.
Raw throughput misleads about waiting time the same way it misleads about cost. trinity-nano is the fastest model here at 72 tokens a second and twelfth from last on time to a right answer. qwen3-30b-a3b-instruct runs at less than half that rate and finishes in eight seconds. A token-rate benchmark would send you to the model you'd find unusable.
Seconds per answer also separates the seven models that tied on my easy suite. On waiting time, they are far apart: 8 seconds for qwen3-30b-a3b-instruct, 61 for nemotron-3-super-120b, and 165 for devstral-2-123b.
devstral-2-123b is the extreme case. At 2.1 tokens a second per request, it emits text slower than you read it. It is a 98.9%-accurate model that is not usable interactively on this hardware at any price, and no energy number would have told you that.
I measured those token rates with four requests running at once, so they show what one user sees while three others are being served. A single-user number would be higher. Seconds per answer is the column to trust either way, because it counts the whole job instead of a rate.
Price a token by its active parameters
Verbosity explains why some models spend a lot of tokens. Something else sets what each of those tokens costs, and on this box that is roughly how many parameters are active per token, not total model size.
I read the active count out of each GGUF instead of trusting model cards: total parameters across every shard, minus the routed expert weights, plus the fraction used per token.
| Model | Total params | Active params | J per token |
|---|---|---|---|
| trinity-nano | 6B | 1.02B | 0.33 |
| qwen3-30b-a3b-instruct | 31B | 3.35B | 0.96 |
| qwen3.6-35b-a3b | 35B | 3.45B | 0.78 |
| trinity-mini | 26B | 3.47B | 0.67 |
| qwen3-coder-next | 80B | 3.87B | 1.45 |
| qwen3-4b-instruct | 4B | 4.02B | 0.57 |
| gpt-oss-20b | 21B | 4.19B | 0.95 |
| gpt-oss-120b | 117B | 5.71B | 1.56 |
| llama31-8b | 8B | 8.03B | 0.97 |
| granite4-h-small | 32B | 8.80B | 5.06 |
| gemma-3-12b | 12B | 11.77B | 1.89 |
| nemotron-3-super-120b | 121B | 12.77B | 4.64 |
| phi-4 | 15B | 14.66B | 1.71 |
| mistral-small-24b | 24B | 23.57B | 2.93 |
| gemma-3-27b | 27B | 27.01B | 4.22 |
| devstral-2-123b | 125B | 125.03B | 25.22 |
Across the full set, that is a 123x range of active parameters against a 76x range in cost per token, with a rank correlation of +0.77. That is a strong trend and not a law, and the exceptions are large enough to list instead of picking four rows that line up neatly:
llama31-8bandgranite4-h-smallhave almost the same active count, 8.0B against 8.8B, and sit 5.2x apart in cost per token.gpt-oss-120bandnemotron-3-super-120bare the cleanest pair here: near-identical total size, 2.2x apart in active parameters, 3.0x apart in cost per token. That pair behaves.gpt-oss-120bagainstdevstral-2-123bis the same total size again and 16x apart, sparse against dense at its starkest.
I checked one way the table could be misleading. Every task carries a few hundred tokens of prompt whether the model writes 66 tokens or 3,000, so a terse model's joules per token is partly amortized prompt cost. granite4-h-small, which answers in 66 tokens, was the prime suspect. Fitting energy against tokens generated for each model separates the fixed cost of showing up from the marginal cost of one more token, and it moved the correlation from +0.77 to +0.78. Granite is genuinely expensive per token.
A 120B mixture-of-experts model with 5B active parameters is cheap to run on this box, a 123B dense model is not, and the number in the name tells you nothing useful about either. The mechanism is simple. This box is bandwidth-bound, cost per token tracks the parameters you read per token, and a sparse model reads fewer.
That claim is about price. It says nothing about quality. Sparse models do sit higher on my hard questions, and the next post covers why a controlled test says architecture isn't what is doing the work. Sparsity makes a model cheaper to run here, and nothing I measured shows it makes the model smarter.
The next post asks what does predict a model's score on hard questions.
