Qwen3.8-27B Reasoning on the RTX 4090 GPU family

KEY FINDINGS

Five results to carry forward.

Turn the model's own reasoning on. It gets 94.4% of our questions right with reasoning off and 98.8% with it on — about seventeen more questions. Unchanged re-runs change by ± 3 to 4 questions, so a difference of 17 is real performance and not random noise.
Stop at low for this question set. Almost the whole gain arrives at the lowest reasoning setting. Every step above it buys 0.4 points or less because RV-380 is already 98.8% solved at low. Harder questions may move the best setting higher.
The default setting is the expensive end. If you never touch the dial you get xhigh, which writes 2.1× as much as low for a gain of 0.39 points.
Reasoning adds tokens rather than slowing token generation. Generation stays at 53–56 tokens per second across settings. The cost comes from longer outputs.
Do not combine it with speculative decoding. On this model, MTP at depth 3 lost 79% of its throughput at the highest reasoning setting, and it still cost accuracy. Treat MTP as model & workload-dependent, not a free speedup, and validate it against the target model and workload before deploying it. The technical note →

One caveat that matters more than the rest: "stop at low" is true of this question set. RV-380 is 98.8% solved by low, so there is almost nothing left for a higher setting to win. On harder questions the right stopping point moves further right — why that is, and what still transfers →

Plain-English glossary Every term this page uses, defined once. Click to open.

The reasoning dial

Native thinking: The model works a problem through before answering. That part is normally hidden by the client or the reasoning parser rather than genuinely unseen — on this server it arrives inline in the response — and it is generated text either way, so you pay for it.
Reasoning effort: The dial that controls how much of that private work happens: off, low, medium, xhigh.
Prompted reasoning: Asking the model in the prompt to show its working. A different mechanism, and it does not perform alike.
Token: A word or piece of a word. Models read and write in these, and they are what reasoning costs.
Checkpoint build: One downloadable copy of the model, quantized a particular way.

What it costs

Tokens per second (tok/s): How fast the model writes once it has started.
Time to first token: How long before anything at all appears on screen.
Time to answer: How long before the answer appears — later than the first token, because the private reasoning comes out first.
Concurrency: How many people are asking at the same moment.
Preemption: The server evicting a half-finished request to make room, then redoing it later.
Autoregressive decoding: The ordinary way a model writes: one token generated at a time, each chosen after seeing the last. The baseline for everything on this page.
Speculative decoding (MTP): Guessing several tokens ahead and keeping the guesses that pass verification. Its effect depends on the model and workload, so this study measures performance and answer behavior together.
Speculative depth: The number of tokens the predictor guesses ahead in a single step, on top of the autoregressive token the model produces anyway. Depth 3 means one autoregressive token plus up to three guesses, kept only if verification accepts them — so depth is the ceiling per step, not the yield. Greater depth means a larger speed win when the guesses are right, and more work discarded when they are not.

How we measure

RV-380: Our fixed set of 380 questions, marked right or wrong by code.
Round: One full pass through all 380 questions, on a freshly restarted server.
Greedy decoding: The model always takes the most likely next token, so sampling randomness is removed. It does not make GPU inference bit-for-bit deterministic — identical re-runs still move 3–4 answers per 380 — but it removes the dice, which is what lets a difference between runs be attributed to the setting we changed.
Paired comparison: Comparing two settings question by question rather than score against score. Only the questions they disagree on carry information.
p-value: If the two settings were genuinely equally good, the probability of seeing a split at least this lopsided. Small means the result is hard to explain by chance. Above 0.05 we report that this experiment did not separate the two — which is not the same as showing they are identical.
Noise floor: How much results move between two identical runs: 3–4 questions in 380. Differences in that range need more evidence before they can be attributed to a setting.

The headline

A step change,
not a marginal gain.

Averaged over four checkpoints, turning native thinking on moves RV-380 from the low-nineties to the high-nineties. Against a measured noise floor of 3–4 question flips per run, this is roughly seventeen questions — four to five times the noise.

THINKING OFF 94.44% mean of 4 checkpoints · averaging 129 generated tokens per answer (median 43)

THINKING ON — LOW EFFORT 98.82% mean of 4 checkpoints · averaging 189 generated tokens per answer (median 134)

RV-380 ACCURACY BY REASONING EFFORT · 380-QUESTION BASIS

AutoRound AWQ GPTQ RedHat xhigh measured on two checkpoints only

The full grid

Four checkpoints,
up to four settings.

Accuracy on the 380-question basis, with the mean number of generated tokens each setting costs. AutoRound comes from the 2026-09-08 run; the other three were measured on 2026-09-09.

FIRST — WHAT "OFF", "LOW", "MEDIUM" AND "XHIGH" MEAN

These are not four different models, and nothing about the model's weights changes between them. They are four positions on a single dial in the chat template called reasoning_effort. It is not a compute budget: reasoning_effort selects chat-template behavior, and the template injects a different instruction at each setting.

offAnswers immediately. No private reasoning at all. This is the regime every earlier RadianVector quality number was measured in.

lowThe template injects an instruction to keep the reasoning brief and move directly to the conclusion.

mediumNo reasoning instruction is injected at all — the neutral setting.

xhighThe template injects an instruction to reason carefully, validate assumptions and weigh alternatives. This is the template's default if you never set the dial.

The private reasoning is generated text, so it costs tokens and time even though the user never sees it. That is the entire cost of turning this dial up.

AND WHAT THE TOKEN COLUMNS ARE COUNTING

Every "tokens/answer" figure on this page is an average across individual answers, not a total. The scale behind it:

QUESTIONS ASKED10,640380 × 28 matched grid runs

TOKENS GENERATED2.31M2,306,626 across those 28 runs

AT THINKING OFF129average tokens per answer

Generated tokens per answer, by setting · 2 rounds
Setting	answers graded	total tokens	average per answer	median per answer	90th percentile
off	3,040	392,510	129	43	283
low	3,040	575,673	189	134	321
medium	3,040	738,652	243	175	481
xhigh	1,520	599,791	395	306	761

The average overstates the typical answer, and badly at "off". The median answer with thinking off is 43 tokens — the mean is 129 because a small number of answers run away to thousands. One question type is responsible for most of it: date_reason is 20 of the 380 questions, 5.3% of the set, and it produces 39% of every token generated at "off". It is also the one question that hits the 4,096-token output cap. So "129 tokens" is a real average over real answers, but most answers are far shorter than it, and the tail is concentrated in one place.

xhigh has half as many answers because only two of the four checkpoints were measured at that setting.

RV-380 accuracy, and the AVERAGE generated tokens per answer, by reasoning effort
Checkpoint	off	tok/ans	low	tok/ans	medium	tok/ans	xhigh	tok/ans	off → best
AutoRound	94.47%	166	98.60%	192	99.21%	256	99.21%	429	+4.74
AWQ	95.13%	91	98.95%	188	98.95%	234	99.21%	362	+4.08
GPTQ	94.21%	138	98.68%	207	98.95%	248	—	—	+4.74
RedHat	93.95%	121	99.04%	171	99.12%	234	—	—	+5.17
Mean	94.44%	129	98.82%	190	99.06%	243	99.21%	396	+4.62

These four are downloadable builds, not four independent algorithms. Three labels name a quantization method; “Red Hat AI” names a publisher, and that build’s own recipe runs AWQModifier followed by GPTQModifier — a composition of two of the other arms, from the same toolchain and version as the AWQ build. Read the rows as four checkpoints you can download, not as a comparison of four separate techniques.
Source: results/thinkgrid_20260908-2016/ (AutoRound, 3 rounds at every setting) + results/thinkgrid2_20260909-0024/ (AWQ: 2 rounds at off/xhigh and 3 at low/medium; GPTQ and RedHat: 2 at off and 3 at low/medium, with xhigh not run). 380-question basis. Scores use the corrected numbers scorer throughout.

THE COST SIDE

01
off → low
+4.38 points for 47% more generated tokens. The only trade on this page that is obviously worth taking.
02
low → medium
+0.24 points for a further 28% more tokens. Below or near the noise floor.
03
medium → xhigh
+0.26 points on AWQ for 55% more tokens; nothing at all on AutoRound.

All token figures are averages per answer. Tokens are the cost: at a fixed generation rate they are the user's wait.

NOT JUST BETTER THAN "OFF"

01
Answer-only, no reasoning
94.47% · 166 tokens. The regime every earlier RadianVector quality number was measured in.
02
Prompted reasoning, capped
92.89% · 56 tokens. Asking the model in the prompt to show its working.
03
Native thinking, low
98.60% · 192 tokens. The template's own reasoning channel.

All three on AutoRound, three rounds each, same 380 questions. Native reasoning and prompted reasoning are two different mechanisms, and they do not perform alike.

Experimental design

How to read
these numbers.

The tables that follow are full of paired tests and p-values. Two design decisions are doing the work behind them, and neither is visible in the numbers themselves.

01 · GREEDY DECODING

The model never rolls a die.

Greedy means temperature 0. At each step the model takes the single highest-probability next token, instead of sampling among the several plausible ones. That removes the sampling randomness. It does not make the serving stack bit-for-bit deterministic — identical re-runs still move 3–4 answers in 380 — but it takes the dice out of the comparison.

That matters more here than on the configuration study, because on this page the thing being changed is how much the model reasons before answering — a change that acts on the model's own output, not on the server. If sampling were on, every effort setting would produce different text on every run, and a 4.38-point gap would be impossible to separate from luck. Greedy makes reasoning effort the only moving part.

It is worth being clear about what greedy does not remove. Even at temperature 0 this stack is not perfectly deterministic: two identical runs of one configuration still differ on 3–4 answers in 380. That residual is the noise floor every claim on this page is measured against, and it is why no result rests on a single round.

02 · PAIRED COMPARISON

Question by question, not score against score.

Each comparison of off and low walks all 380 questions and sorts them into four groups: both right, both wrong, only the first setting right and only the second right.

The questions where both settings agreed are then set aside. One they both answer correctly tells you nothing about which is better, and neither does one they both fail. Only the disagreements carry information, and the test asks whether one side won them far more often than a coin flip would explain.

The reason to bother is that RV-380 mixes easy questions with brutal ones, so comparing overall scores mostly measures which questions you happened to ask. Pairing cancels that completely — both settings face the identical question, the same logic as measuring each patient before and after a drug rather than comparing two different groups of people.

It cuts both ways, and on this page you can watch it do so. off versus low separates at p=2.7e−25 because the disagreements are lopsided nearly every time. But with thinking on, the four checkpoints disagree on only 0–8 questions out of 380 — so the same test finds nothing, not because they are proven equal but because it has almost no evidence left to weigh. That distinction is the subject of the closing section →

One consequence worth stating plainly: pairing only works if both arms answered the same questions in the same rounds. That is why every effort setting on this page was run against one server boot per checkpoint, why suite hashes are never pooled, and why the speculative-decoding section reports its own basis separately rather than merging it with the rest.

Is it real?

Paired tests,
question by question.

Every comparison below is paired: the same question, the same round, one setting against another. Nothing here is an average of two separate scores.

FIRST — WHERE THE NUMBER 760 COMES FROM

A model does not always give the same answer to the same question. Ask it twice and it can be right the first time and wrong the second. So measuring a setting once cannot tell a real improvement from a lucky run — which is why every setting here was measured more than once, on a separately restarted server each time. We call each full pass through the question set a round.

QUESTIONS380the fixed RV-380 set

ROUNDS2separate runs of all 380

COMPARISONS760what every row below is out of

Some settings were actually run three times rather than two. The two tables below use two rounds for every row, so that every row is out of the same 760 and no row silently gets more evidence than its neighbors. Nothing is lost by doing so: every verdict in those tables is identical whether the third round is included or not. One later section — the speculative-decoding comparison — runs on a different basis for a stated reason, and shows its own arithmetic where it does.

THEN — HOW TO READ THE TABLES

Take one question in one round and ask it under two different settings. There are four possible outcomes: both right, both wrong, only the first right, only the second right. The first two tell you nothing about which setting is better — so they are set aside, and the whole test rests on the last two.

only A rightTimes when the first setting answered correctly and the second did not.

only B rightTimes when the second answered correctly and the first did not. The reverse case.

they differed onThe two added together — every time the settings disagreed. This, not the 760, is the real evidence the test works from.

The p column asks a single question: if the two settings were genuinely equally good, how likely is a split this lopsided by luck alone? Small means the result is hard to explain away as chance. Anything above 0.05 we report as no difference.

WORKED EXAMPLE — THE REDHAT ROW, IN WORDS

380 questions, asked twice at each setting, is 760 comparisons. Thinking-off and thinking-low gave the same verdict on 721 of them — both right, or both wrong. They differed on 39. On all 39, the thinking-on answer was the correct one and the thinking-off answer was wrong. Not one went the other way.

A 39–0 split is very hard to explain by chance, and that is the 3.6e−12 in the table. Treat that exponent as indicative rather than exact: the 39 discordances are counted over question-rounds, and two rounds of the same question are repeat measurements of one item rather than independent observations. The direction and size of this effect are not in doubt; the decimal places are.

This is also why a small "differed on" number is a warning rather than a reassurance. Two settings that differ on 3 questions cannot be separated by any test — not because they are proven equal, but because there is almost nothing to weigh.

A · Does the reasoning setting matter?

Effort vs effort, within each checkpoint · 380 questions × 2 rounds = 760 comparisons per row
Checkpoint	Compared	out of	only the harder setting right	only the easier setting right	they differed this many times	p by chance?	Verdict
AWQ	low vs off	760	31	2	33	1.3e−07	LOW BETTER
GPTQ	low vs off	760	38	4	42	5.7e−08	LOW BETTER
RedHat	low vs off	760	39	0	39	3.6e−12	LOW BETTER
AWQ	medium vs low	760	0	0	0	1.00	NO DIFFERENCE
GPTQ	medium vs low	760	2	0	2	0.50	NO DIFFERENCE
RedHat	medium vs low	760	1	0	1	1.00	NO DIFFERENCE
AWQ	xhigh vs medium	760	2	0	2	0.50	NO DIFFERENCE

B · Does the checkpoint matter?

Checkpoint vs checkpoint at matched effort · 380 questions × 2 rounds = 760 comparisons per row
Setting	Compared (A vs B)	out of	only A right	only B right	they differed this many times	p by chance?	Verdict
off	AWQ vs GPTQ	760	23	16	39	0.34	NO DIFFERENCE
off	AWQ vs RedHat	760	20	11	31	0.15	NO DIFFERENCE
off	GPTQ vs RedHat	760	18	16	34	0.86	NO DIFFERENCE
low	AWQ vs GPTQ	760	2	0	2	0.50	NO DIFFERENCE
low	AWQ vs RedHat	760	1	2	3	1.00	NO DIFFERENCE
low	GPTQ vs RedHat	760	1	4	5	0.38	NO DIFFERENCE
medium	AWQ vs GPTQ	760	0	0	0	1.00	IDENTICAL
medium	AWQ vs RedHat	760	0	2	2	0.50	NO DIFFERENCE
medium	GPTQ vs RedHat	760	0	2	2	0.50	NO DIFFERENCE

READ THIS BEFORE CONCLUDING "ALL CHECKPOINTS ARE EQUAL"

No pair separates at any setting, but the evidence differs by row. With thinking off there are 31–39 disagreements per pair and the wins split roughly evenly. With thinking on there are only 0–8 disagreements because the checkpoints converge to near-identical behavior. "No difference" at medium therefore reflects limited headroom in this instrument.

This also does not overturn the earlier conviction of GPTQ at p=0.004 and p=1.8e−08. Those were measured on harder instruments where the model scored in the seventies, leaving room to discriminate. Here every arm sits between 94% and 99%. A negative result is only as strong as the instrument's remaining headroom — and on this page that headroom is small by construction.

One naming caveat: these are four downloadable checkpoint builds, not four independent quantization algorithms. “RedHat” names Red Hat AI’s llm-compressor build; its recipe combines AWQModifier and GPTQModifier.

The positive control uses the same method, questions and rounds and separates off from low at p=2.7e−25. This paired table finds no separable difference among the three jointly run checkpoint builds.

The other half of the trade

Accuracy is what
it buys. This is
what it costs.

Everything above this point measures whether thinking makes the model right. It does. But nobody deploys an accuracy number — they deploy a server that has to answer people. So we ran the whole effort dial again on a second instrument, measuring wall-clock wait and how many users still fit.

Six checkpoints, four effort settings each, nine concurrency levels from one user to thirty-two — 216 measured rungs in one overnight campaign. Unlike the throughput numbers in the configuration study, this one does not pin the output length. That matters more than it sounds, and it is the whole reason a separate instrument was needed.

WHY THE CONFIGURATION STUDY COULD NOT ANSWER THIS

Our standard throughput harness sends a fixed max_tokens and sets ignore_eos, so the model emits exactly as many tokens as it is told to and never stops early. That is deliberate, and it is the right way to measure raw decode speed: it removes the model's choices from the measurement.

It also makes that harness structurally blind to reasoning effort — because the only thing effort changes is how many tokens the model chooses to emit. Pin that, and thinking becomes invisible. Point the old instrument at this question and it reports "no difference," confidently and wrongly.

Thinking does not make tokens slower. It buys more of them.

This is the finding everything else follows from, and it is unusually clean. Across all sixteen checkpoint-and-effort combinations at one user, the decode rate sits between 53.2 and 56.1 tokens per second — a spread of 5%, with no trend across the effort dial. This is decode-phase throughput. The configuration study's 49.6 tok/s comes from the end-to-end run and includes both the prefill and the decode phases. The model does not think "harder" per token. It emits a longer answer.

So every cost on this page is a token-count cost, and it is predictable: mean tokens per answer go from 117 at off to 172 at low, 235 at medium and 417 at xhigh. A 3.6× longer answer at the same tokens per second is a 3.6× longer wait.

Those four numbers are close to, but not the same as, the 129 / 190 / 243 / 396 quoted in the accuracy sections above, and the difference is not an error. The accuracy campaign asks all 380 RV-380 questions; this one cycles a 48-question sample drawn evenly across all 17 kinds, because a latency ladder has to send the same prompts at every concurrency level to stay comparable. Different question mix, slightly different mean answer length. The shape is what carries over: answers grow roughly threefold from off to xhigh on either basis — 3.1× on the accuracy suite, 3.6× here.

MEDIAN WAIT FOR A COMPLETE ANSWER · ONE USER · BY REASONING EFFORT

AutoRound AWQ GPTQ RedHat median of 24 requests per point

The four checkpoints start apart and end together. AWQ is the fastest with thinking off (1.10 s) and GPTQ the slowest (2.55 s), a 2.3× spread; by xhigh they sit within 1.9 s of each other and the ordering has partly reversed. Once the answer is long enough, how the weights were quantized stops mattering to the clock — the token count dominates.

The ledger

This table places accuracy and cost on the same view. Throughput is quoted at twelve concurrent users, where all four effort settings reach their peak, so every row uses the same load.

RV-380 ACCURACY AGAINST MEASURED COST · MEAN OF THE FOUR QUANTIZED CHECKPOINTS
Effort	RV-380 accuracy	Tokens per answer	Wait, 1 user	Answers/min at 12 users	What the step bought
off	94.44%	117	1.86 s	75.0	baseline
low	98.82%	172	3.47 s	61.1	+4.38 pts for −18.6% throughput
medium	99.06%	235	4.50 s	53.0	+0.24 pts for −13.2%
xhigh	99.21%	417	6.93 s	35.4	+0.15 pts for −33.1%

THE TRADE IN THROUGHPUT

Turning thinking on costs about a fifth of your throughput and buys 4.38 accuracy points. Going the rest of the way to the template's default costs a further 42% of what is left, and buys 0.39 points. The cost side of the dial is roughly linear; the benefit side flattened two settings earlier.

COMPLETED ANSWERS PER MINUTE · BY OFFERED CONCURRENCY · MEAN OF FOUR CHECKPOINTS

off low medium xhigh one hue, light to dark — the dial is ordered

Two things to read here. The curves flatten after twelve users on every setting, because the server stops accepting more work — see the ceiling below. And the dip every curve takes at sixteen users is real and reproduces on all four checkpoints; we do not yet have an explanation for it, so it is drawn rather than smoothed.

Where the time actually goes

The tables above give the totals. This is the same data opened up: each bar is one request’s whole life, split into the part spent waiting for a turn and the part spent generating text. Every load level shows both settings, so the pairs can be read straight down.

ONE REQUEST’S LIFE UNDER LOAD · THINKING OFF vs XHIGH · MEAN OF FOUR QUANTIZED CHECKPOINTS

waiting for a turn (queue + reading the prompt) generating text upper bar of each pair is thinking off · hover for exact seconds

Two readings, and they point opposite ways. At one user, effort is the whole cost: xhigh takes 8.12 s against off’s 2.49 s, and 96% of that is genuinely generating — the user is waiting for reasoning, not for a queue. Under load, the queue swallows the difference: by thirty-two users both settings spend about three-quarters of the request waiting, off at 74% and xhigh at 75%. What thinking changes is the scale — 20.6 s becomes 47.8 s — because every request holds its slot for longer and everyone else’s wait grows with it. Generating time itself grows only 1.5× across the whole ladder at xhigh, while waiting grows 117×.

Thinking does not change how many users fit

We expected the capacity ceiling to fall as answers got longer, because a longer answer holds its share of cache for longer. It does not. The number of requests the server will actually run at once is a property of the checkpoint and its memory layout, and it is identical at every effort setting — we offered thirty-two concurrent users and watched what the scheduler actually admitted.

PEAK CONCURRENT REQUESTS ACTUALLY RUNNING, MEASURED FROM THE SERVER'S OWN TELEMETRY
Checkpoint	Ceiling at off	Ceiling at xhigh	Preemptions across the ladder	Reading
AWQ	10	10	591	unchanged by effort
AutoRound	10	10	674	unchanged by effort
GPTQ	10	10	613	unchanged by effort
RedHat	10	10	616	unchanged by effort
8k context + fp16 SSM cache	16	16	7	reached every level we offered
MTP speculative depth 3	4	4	699	capped at 4 regardless

A preemption is the server evicting a half-finished request to make room, then recomputing it later. It is invisible in vLLM's logs and only appears in its metrics endpoint, which is why we assert the counter rather than reasoning about headroom.

THE CONFIGURATION THAT CHANGES THE ANSWER

The fifth row is not a quantization choice — it is the same model served with a 16k context trimmed to 8k and the Mamba state cache in fp16. That one change lifts the ceiling from 10 concurrent to 16, and takes preemptions from roughly six hundred across the ladder to seven. At sixteen users with thinking off it answers in 0.42 s where AWQ takes 5.03 s.

It is on this page because it reframes the cost table above: the throughput that reasoning effort spends is throughput a serving-configuration change can partly give back. The two dials are independent, and the memory-layout dial is the cheaper of the two.

Speculative decoding and thinking do not mix

The configuration study records that, on this model, multi-token prediction costs accuracy in proportion to how long the answer is. Thinking makes answers three to four times longer, so the two interact — and the interaction is worse than either alone. At sixteen users, MTP at depth 3 goes from 96.8 answers per minute with thinking off to 20.2 at xhigh, a 79% collapse, with 195 preemptions in that single rung and a 95th-percentile wait of 57 seconds. Every other checkpoint degrades by about half. If you are running speculative decoding, treat high reasoning effort as incompatible with it until this is understood.

ONE MEASUREMENT THAT IS NOT FINISHED — AND A CORRECTION

There is a wait this campaign did not originally capture. Because no reasoning parser is configured on this server, the model's thinking arrives inline in the response, wrapped in a <think> block. A first token therefore arrives almost immediately at every effort setting — but if your interface hides reasoning, the user sees nothing until that block closes.

The initial detector waited for an opening <think> tag, but the chat template places that tag in the prompt and the model emits only the closing tag. The corrected AWQ measurement finds that the first token still arrives in 0.30 s at every setting, while the answer starts at 3.03 s at low and 3.69 s at medium. At sixteen users, those values rise to 5.3 s and 6.4 s.

Three of the four effort settings have been re-measured on one checkpoint. The remaining ladders are planned, and no other result on this page uses this time-to-answer measurement.

results/thinkperf_20260910-0141/ · 216 rungs, six checkpoints × four efforts × nine concurrency levels, one server boot per checkpoint so all four efforts share a KV pool. Corrected time-to-answer from results/thinkperf_20260910-1112/.

The practical answer

Turn it on.
Then stop at low.

The template's default effort is xhigh. On this evidence that default is the wasteful end of a curve that has already flattened two settings earlier. On this suite — and that qualification matters →

OFF94.44%129 tokens · the old default

LOW98.82%190 tokens · +4.38 pts · recommended

MEDIUM99.06%243 tokens · +0.24 pts

XHIGH99.21%396 tokens · +0.15 pts · template default

THE TRADE IN TOKENS

Going from off to low buys 4.38 points for 61 extra tokens. Going from low all the way to xhigh buys a further 0.39 points for 206 extra tokens — five times the token cost for a ninth of the gain, at or below the run-to-run noise floor.

THE SAME CONCLUSION, REACHED A SECOND WAY

That paragraph is an argument about tokens. The cost measurements reach the same place from an independent direction: going from off to low costs 18.6% of your throughput and buys 4.38 points, while going from low to the template's default costs a further 42% and buys 0.39. Two instruments, two different failure modes, one answer — stop at low.

MEASUREMENT CORRECTION

The earlier AutoRound run found medium and xhigh to be exactly identical — zero disagreements across 1,140 question-rounds. That does not quite reproduce. On AWQ there are 2 disagreements across 760 question-rounds, and both are the same single question, which xhigh gets right in both rounds. So "the curve saturates" holds; "medium and xhigh are identical" does not. The gap is one question in 380.

The limit of the result

Thinking does not
rescue speculation.

Speculative decoding (MTP) cost this model accuracy, and the damage scaled with how much text it generated. Native thinking generates more text — so the open question was whether it would land at the safe end of that curve or make things worse.

CONTROL 98.89% native thinking, medium · 273 tokens

+ MTP DEPTH 3 97.68% identical config but speculation · 273 tokens

THE PAIRED TEST · 330 QUESTIONS × 3 ROUNDS = 990 COMPARISONS

01
−1.21 points
Control wins 14, MTP wins 2, of 16 disagreements across three matched rounds. p = 0.0042. On two rounds it is 12 against 2, p = 0.013 — same verdict either way.
02
19× less self-consistent
Run the same config twice: the control changes its mind on 0 and 1 questions. MTP changes its mind on 10 and 9.

Under greedy decoding, speculative decoding should give almost the same output as running without it — but the guarantee stops at rounding error, and vLLM does not promise two runs will match. Repeating itself 19× less reliably than the control is well beyond that. Technical note →

WHAT IT ANSWERS

−1.21 points is almost exactly where the earlier length ladder put speculation at this output length (−1.2), against −11.9 for short answers and −8.9 when capped. So the damage tracks how much the model writes, not why it writes it. Native thinking does not fix MTP on this model; it just puts you at the forgiving end of the same curve.

MTP TECHNICAL NOTE

MTP surfaced a deployment caveat worth investigating separately.

In our tested MTP configuration for this model, MTP was not behavior-preserving. With native thinking on, the measured difference was −1.21 points; in the configuration study it ranged from −11.9 accuracy points on short answers to −1.2 points under full reasoning. Both sit well above our repeat-run variation.

The operational conclusion

Validate MTP against the actual model and workload rather than treating it as a free speedup. These results characterize Qwen3.8-27B on this stack, not MTP in general; long, reasoning-heavy output is the forgiving end of the curve, not a safe default.

Why this is notable

Speculative decoding is intended to preserve the target model’s output, subject to numerical and runtime effects. vLLM qualifies its losslessness guarantees around hardware numerics and does not promise bit-for-bit identical output across runs. Here, two identical MTP runs changed 10 and 9 answers, while the non-speculative control changed 0 and 1.

The mechanism investigation is published separately: the vLLM MTP bug investigation traces this loss to a defect in vLLM's V1 model runner and tests a fix.

Scope · Qwen3.8-27B W4A16 · vLLM 0.28 · WSL2 · on the RTX 4090 GPU family · greedy, temperature 0. The MTP comparison on this page runs on 330 questions rather than 380, for the context-window reason noted above.

What this does not say

Limits, stated
on the page.

Four things a careful reader should know before reusing these numbers.

01 · THE SPECULATION ARMS USE A SMALLER BASIS

The two MTP-comparison configurations run with an 8,192-token context. With 4,096 tokens reserved for output, 50 long-context questions do not fit and the server refuses them outright with an HTTP error. A refused request is not a wrong answer, so those questions are scored on neither arm and that comparison runs on 330 questions, not 380. The refusals are identical on both arms, so the paired test and the −1.21 points are unaffected — but the absolute percentages in that section are not comparable to the rest of this page.

The configuration study counts these differently. There, a refused request is counted as a wrong answer and the basis stays at 380, which lowers those arms' headline percentages by about 13 points without changing any paired comparison. The two conventions are not interchangeable, so absolute percentages should not be compared across the pages. The number of refusals also differs by suite — 15 there and 50 here — because this page reserves a larger output budget, pushing more long-context questions past the limit.

02 · THE 20 EXCLUDED QUESTIONS ARE NOT HIDING ANYTHING

RV-380 excludes one 20-question task type whose prompt is ill-posed. Every checkpoint answers all 20 correctly at every effort setting — 760 of 760 question-rounds. Including them would raise every arm by 0.04–0.30 points and change no comparison, so they are excluded as non-discriminating items.

03 · ONE QUESTION TRUNCATES, AND ONLY WITH THINKING OFF

A single date-reasoning question hits the output cap on the thinking-off arm and on no other. The effective basis for those runs is therefore 379 rather than 380 — worth 0.26 points, below the noise floor, but it means the denominator is not perfectly constant across the page.

04 · THE ACCURACY AND COST FIGURES COME FROM DIFFERENT RUNS

Every accuracy number on this page comes from a quality run, where the token column is generated tokens rather than measured time. The wait, throughput and capacity figures in what it costs come from a separate campaign on a separate instrument, with a different question mix and a different output budget. They are two measurements of the same effort dial, not two columns of one measurement — which is why the token counts differ slightly between them, and why an accuracy percentage and a latency figure on this page should never be combined into a single derived number without checking that both rest on the same basis.

Boundaries

Where the conclusions
stop.

The results are bounded by the model, runtime, hardware, workload and configurations tested.

READ THIS BEFORE REUSING "LOW"

The knee is a property of the workload, not of the model. Stopping at low is the right answer for RV-380. It is not a general law about reasoning effort, and that distinction matters more than any other sentence on this page.

WHY THE CURVE FLATTENED

By low, every checkpoint is already answering 98.82% of RV-380 correctly. That leaves about four and a half questions in 380 still to be won. No setting can buy more than the headroom that remains — so the curve had to flatten at low, whether or not medium and xhigh are genuinely reasoning harder.
What the saturation chart measures is the ceiling of this instrument, not the limit of the model's reasoning. The significance section says the same thing from the other side: with thinking on, the four checkpoints disagree on 0–8 questions out of 380. The test does not run out of significance because the settings are equivalent. It runs out of questions able to tell them apart.

WHERE THE ANSWER WOULD DIFFER

On a harder or more varied workload — multi-step reasoning, long chains of dependent inference, genuinely ambiguous questions, anything where this model sits at sixty or seventy percent rather than ninety-nine — there is real headroom left, and medium or xhigh may well repay their throughput cost.
This project has already seen the mechanism from the other direction. Suite v2 could barely convict the GPTQ checkpoint (p=0.004). Suite v3, deliberately concentrated in discriminating questions, convicted the same checkpoint at p=1.8e−08. Same model, same method, same weights — a harder instrument, and more than five orders of magnitude more evidence. Difficulty is what creates headroom, and headroom is what lets a setting prove itself.

WHAT ACTUALLY TRANSFERS

Two different things on this page generalise to different degrees.
The cost curve travels well. What each step of the dial costs — roughly 47% more tokens from off to low, 78% more from medium to xhigh, and the throughput that follows from them — is a property of the model and the server, not of the questions. Expect similar costs on your workload.
The benefit curve does not travel. Where it flattens depends entirely on how hard your questions are. So the transferable finding is not "low is where the universe is." It is: measure your own knee, because the knee moves with difficulty — and now you know what each step costs while you look for it.

01

RV-380 is a regression suite

It is built to catch a configuration change that breaks something, and it is good at that. It is not a general intelligence benchmark, and its ceiling under thinking is the direct consequence. Treat 98.82% as "this suite is close to exhausted", not as "the model is nearly perfect".

02

Four settings, not a continuum

The dial exposes four named values. We cannot say where between off and low the gain actually arrives, only that almost all of it has arrived by low.

03

The cost figures are one traffic shape

Short questions, a bounded output budget, a queue drained by a fixed number of workers. A workload of long documents or streaming chat would hold cache differently and move the ceilings.

04

The search space keeps moving

One model family, one 24 GiB GPU, one vLLM version. Quantization, KV precision, batching, speculative decoding, kernels and memory policy all interact, and they change with every release. A result is a measurement at a date, not a constant.

The LLM inference configuration space changes with models, runtimes, kernels and hardware. RadianVector searches and narrows that space against the application’s workload, then reports the measured trade-offs for deployment. This study does not identify a universally optimal setting; its conclusions apply to the hardware, versions of the inference stack, workloads and configurations tested here.

In one line

Switch native thinking on. On a suite like this one, leave it at low — and pick your checkpoint on license and download size, not on this score.

The four words doing the work there are on a suite like this one. Low is where RV-380 stops paying, because RV-380 has almost nothing left to win by low. On harder questions the knee sits further right. Why that is, and what still transfers →

44 v7 quality runs archived · 4 checkpoint builds · up to 4 effort settings · RV-380, greedy decoding · Qwen3.8-27B W4A16 on the RTX 4090 GPU family, vLLM 0.28 under WSL2.
Every setting was run at least twice on separately restarted servers; some were run a third time. Checkpoint and effort comparisons use two rounds — 760 question-round comparisons per row. The MTP section additionally reports its matched third round (990 comparisons) and gives the two-round result; both produce the same verdict.
Sources: results/thinkgrid_20260908-2016/ and results/thinkgrid2_20260909-0024/. Suite: scripts/tasks_v7.py, sha256 7c48ce4f.

Thinking on,thinking off.