Of the 11 models I ran on both easy and hard questions, the cheapest on easy work costs 3.6 cents per thousand right answers on my Strix Halo. Give that same model hard questions and a right answer costs 37 times more. Two other models in the set only get twice as expensive over the same jump.
I am a big fan of local LLMs. I control the models and the harnesses, and I pay no subscription fee. That doesn't make local free. I live in California, the land of expensive electricity, and I had never worked out what an answer costs from my Strix Halo system. That question made me wonder how different models stack up, so I measured twenty-seven of them with a power meter running.
Picking the right model is worth more than anything else I tested, and "the right model" depends on how hard your questions are. The cheapest model for easy questions can cost 37 times more per right answer once the questions get hard. Match the model to the job. The numbers behind that are below.
Count correct answers instead of tokens
A "correct answer" here means one of 190 small tasks with checkable answers, split roughly evenly across seven kinds. One kind gives the model a running total starting at zero and an empty stack of numbers, explains five operations, and asks it to apply twenty of them in order:
POP, POP, RESET, DOUBLE, ADD 3, PUSH 8, ADD 7, PUSH 1, RESET, ADD 6, POP, DOUBLE, PUSH 2, ADD 9, DOUBLE, RESET, PUSH 7, PUSH 9, DOUBLE, RESET
PUSH puts a number on the stack, POP takes the top one off and into the running total, and ADD, DOUBLE, and RESET do what they say. The answer is the final total and how many numbers are left on the stack. No single step is hard. You have to hold two values straight for twenty of them without losing your place.
The other six kinds share that shape: follow a chain of arithmetic, transform a list, pull facts from a paragraph padded with distractors, satisfy constraints, write a function that passes tests, and produce output in an exact format. A script decides right or wrong, and prompts run a few hundred tokens.
Those tasks turned out to be too easy, so I also ran 11 of the models against GPQA Diamond: four-way multiple-choice physics, chemistry, and biology written by domain experts and deliberately Google-proof. I ran the first 100 of its 198 questions against each model. That is the "hard questions" column throughout.
Everything is Q4_K_M, with four concurrent requests unless I say otherwise. Energy is integrated from 1Hz wall-power telemetry. I'm in Southern California, so prices use my actual summer consumer rates, not a friendly national average: 34 cents per kilowatt-hour off-peak and 58 cents on-peak. Every dollar figure below is on-peak unless the column says otherwise.
Expect a right answer to cost more as questions get harder
Here is the reason you're reading. The last column is the ratio between the two: how much more a right answer costs once the questions stop being easy. I call it the difficulty tax.
| Model | Easy questions | Hard questions | Difficulty tax |
|---|---|---|---|
| gemma-4-26b-a4b (reasoning off) | $0.130 | $0.361 | 3x |
| gemma-4-12b (dense, reasoning off) | $0.185 | $0.515 | 3x |
| gpt-oss-120b | $0.057 | $0.702 | 12x |
| gemma-4-31b (dense, reasoning off) | $0.394 | $0.763 | 2x |
| ornith-35b | $0.338 | $0.795 | 2x |
| qwen3.6-35b-a3b | $0.228 | $0.849 | 4x |
| mistral-small-24b | $0.111 | $1.056 | 10x |
| qwen3-coder-next | $0.048 | $1.081 | 22x |
| qwen3-30b-a3b-instruct | $0.036 | $1.346 | 37x |
| gemma-3-27b | $0.117 | $1.794 | 15x |
| nemotron-3-super-120b | $0.227 | $14.656 | 64x |
All figures are dollars per thousand correct answers.
"Reasoning off" on three of those rows means what it says. Gemma 4 ships with a reasoning mode enabled, and I measured it both ways. Leaving it on costs up to 48 accuracy points and 6x to 8x the money on easy work. The second post in this series covers that flag.
Two rows need a caveat. qwen3-coder-next (46GB) ran at half the concurrency of the others and nemotron-3-super-120b (77GB) at a quarter, because that much weight plus a large KV allocation hangs this machine. Fewer slots mean longer wall time, and I derive hard-task energy from wall time, so both costs are inflated relative to the rest. Read them as upper bounds. For coder-next that matters a lot, because it scored 79% on the hard questions, joint best in the set.
Read only the first column and qwen3-30b-a3b-instruct is the obvious choice at 3.6 cents per thousand, cheaper than everything else here. Read the second column and it drops to third from last, costing 37x more per right answer than it did a column earlier. Across these 11 models, the difficulty tax runs from 2x to 64x.
Stop sending a 125B model to add 20 numbers
Seven of the 27 clear 97% on my easy suite. For that work, they are interchangeable: you get a right answer either way.
| Model | Easy pass | $/1k | Total params | Active params |
|---|---|---|---|---|
| qwen3-30b-a3b-instruct | 97.9% | 0.036 | 30.5B | 3.35B |
| qwen3-coder-next | 97.9% | 0.048 | 79.7B | 3.87B |
| gpt-oss-120b | 100.0% | 0.057 | 116.8B | 5.71B |
| mistral-small-24b | 98.9% | 0.111 | 23.6B | 23.57B |
| gemma-3-27b | 97.9% | 0.117 | 27.0B | 27.01B |
| nemotron-3-super-120b | 98.9% | 0.227 | 120.7B | 12.77B |
| devstral-2-123b | 98.9% | 1.299 | 125.0B | 125.03B |
devstral-2-123b is an excellent model, and here it is keeping a running total straight for 20 steps. It does that correctly, for 36x what the cheapest model charges to also do it correctly.
Sending a 125B dense model to add up 20 numbers costs 36x what the cheapest correct model charges for the same result. That is the half of "pick the right model for the job" that people already suspect. The other half needs the hard questions. The fact that seven models tie on my easy suite was my first clue that the suite was too easy, and the fourth post in this series covers what happened when I swapped in GPQA.
Match the model to your workload
There isn't one answer, and the difficulty tax is the reason.
For short, checkable work, which is most of what an agent loop does, use qwen3-30b-a3b-instruct, prompted to think. It costs 3.6 cents per thousand correct answers and takes eight seconds each, and nothing I measured is both cheaper and no worse. It is the best deal on this box by a distance, right up until the questions stop being easy.
For hard work, it is the wrong pick, and it isn't close. It scores 63% on GPQA, and a right answer costs $1.35 per thousand instead of 3.6 cents. Three models sit at the top of the hard questions within a point of each other: qwen3.6-35b-a3b and qwen3-coder-next at 79%, and ornith-35b at 78%.
On these numbers, qwen3-coder-next is the standout of the three. It matches the best hard-task score in the set and costs 4.8 cents per thousand on easy work, second-cheapest overall. A coding-tuned model taking the top score on graduate science questions is not a result I expected, and I re-read the samples twice because of it.
If you don't know which you have, gemma-4-26b-a4b with reasoning off is the hedge. It is the cheapest hard-task model I measured at 36 cents per thousand; it scores 100% on easy work, and its difficulty tax is 3x. It gives up the top of the accuracy range to be predictable, which for a machine serving mixed traffic is usually the right trade. It also costs 30% less per right answer than its dense sibling gemma-4-12b and scores no worse, on a third of the active parameters.
For simple work at volume where 72% accuracy is fine, gemma-3-4b costs 2.5 cents per thousand, still the cheapest thing here.
My own daily driver is none of those four. It is qwen3.6-35b-a3b, which sits in the expensive third of the easy-work cost table and looks dominated there. On hard questions it ties for first, using about an eighth of the tokens that qwen3-30b-a3b-instruct spends getting fewer of them right. A model can be cheap on easy work and expensive on hard work, and neither number settles the choice alone.
Before you pick, check two things that have nothing to do with the model name. Check whether reasoning is on by default and whether you want it, which was worth 48 points and 8x on Gemma 4. And check whether your model stops talking when told to, because twelve of mine wouldn't.
All of this comes from two tasks: a suite I wrote and a set of multiple-choice science questions. That is enough to show that a 125B dense model is a poor way to add up 20 numbers, and enough to show that the difficulty tax is real. It is nowhere near enough to pick your model for you. Take the tables as a shortlist and as a list of things worth measuring, then run your own work against the top few.
