19 min read6 sections

Lab 08 · for chapter 06-02

Speculative decoding acceptance rate

Measure acceptance length across workloads and find the point where speculation becomes a net loss.

Download lab script · CPU reference checks · Environment preflight.

Speculative decoding has exactly one input you cannot compute on paper — the acceptance rate — and exactly one output that decides whether to ship it: the batch size at which the speedup crosses 1.0. This lab measures the first across workload types, then finds the second, and keeps the two apart, because they fail for completely different reasons.

Hardware

Part A — acceptance across workloads — runs on one 24 GB card with an 8B target and the n-gram proposer, which needs no draft weights at all. Part B — the break-even batch — needs one 80 GB card, because the crossing is predicted near 150 concurrent sequences and Llama-3-8B at 2k context costs 128 KiB of KV per token (§2.1): 150 sequences at 2,304 tokens is 44 GB of cache before you count weights. On 24 GB you will run out of KV before you reach the crossing, and the curve will bend for the wrong reason. Part C substitutes a synthetic acceptance rate for a real draft, and needs whatever Part B needed.

Not executed here

run.py was written against the config fields, metric names and harness output cited below, and its argument handling and arithmetic were exercised, but it has not been run against a live engine — no GPU was available while writing. Every acceptance figure on this page is either a swept parameter or derived arithmetic from the model in §6.2. None is measured. If your measured α disagrees with anything here, your number is the real one.

§1

What you measure

Three quantities, and the relationship between them:

Mean acceptance length $E$ — tokens emitted per verify step, bonus token included. This is what both engines actually report. Measure it per workload type.
The per-position acceptance vector $r_i$ — the unconditional probability that the first $i{+}1$ drafts all survive. The i.i.d. model says $r_i = \alpha^{i+1}$ for zero-based position $i$. Measure whether it fits.
The break-even batch $B^{*}$ — where speculation stops paying, measured by sweeping concurrency with speculation on and off and finding the crossing.

§6.2 derives the arithmetic that connects them. With $\kappa$ drafted tokens per step and per-token acceptance $\alpha$,

E = \frac{1-\alpha^{\kappa+1}}{1-\alpha}, \qquad B^{*} = \frac{T^{*}\big(E - \kappa c\big)}{\kappa+1}

where $T^{*}$ is the roofline ridge in token positions (295 for bf16 on an H100 SXM, §0.4) and $c$ is the draft step's cost as a fraction of the target's. For a Llama-3.2-1B draft against a Llama-3-8B target, §6.2 derives $c = 0.165$; for the n-gram proposer there is no draft forward pass at all, so $c \approx 0$ and every number in the table below moves right.

What your measured α would imply, at $\kappa=3$, $T^{*}=295$ — derived, arithmetic from the §6.2 model. α is a swept parameter here; measuring it is Part A.

α	$E$	speedup, 1B draft ($c$=0.165)	break-even $B^{*}$	speedup, n-gram ($c\approx0$)	break-even $B^{*}$
0.4	1.624	1.09×	83	1.62×	120
0.5	1.875	1.25×	102	1.88×	138
0.6	2.176	1.46×	124	2.18×	160
0.7	2.533	1.69×	150	2.53×	187
0.8	2.952	1.97×	181	2.95×	218
0.9	3.439	2.30×	217	3.44×	254

Read the break-even columns as a ceiling, not a ladder. $B^{*}$ is linear in $E$ and $E$ is capped at $\kappa+1$, so $B^{*} \le T^{*}$ always: no draft, however good, pushes the crossing past the roofline ridge. A perfect draft at $\kappa = 3$ would give $E = 4$ and $B^{*} = 295$, and that is the end of the road. What actually moves the crossing is a smaller $\kappa$ — at $\alpha = 0.7$, $\kappa=1$ breaks even at 226 concurrent sequences against $\kappa=3$'s 150, because the verify step carries fewer wasted positions. That asymmetry is why both engines adapt $\kappa$ at runtime rather than switching speculation on and off, and it is a place where they picked different signals. vLLM schedules $\kappa$ by batch size:

vllm/config/speculative.py:L180-L186 vLLM

    # dynamic speculative decoding control
    num_speculative_tokens_per_batch_size: list[tuple[int, int, int]] | None = None
    """Batch-size schedule used to dynamically choose speculative-token count.

    Each entry is ``(range_start, range_end, num_speculative_tokens)`` with an
    inclusive batch-size range.
    """

SGLang adapts on the measured acceptance rate instead:

python/sglang/srt/server_args.py:L2278-L2287 SGLang

    speculative_adaptive: A[
        bool,
        "Enable adaptive speculative decoding that dynamically adjusts num_steps based on acceptance rate.",
        NS("spec"),
    ] = False
    speculative_adaptive_config: A[
        Optional[str],
        "Path to a JSON config file for adaptive speculative decoding tuning knobs.",
        NS("spec"),
    ] = None

Both are defensible and they fail differently: batch size is known before the step, acceptance only after it. vLLM's schedule reacts instantly to load and not at all to a change of workload; SGLang's reacts to the workload and lags load by however long its controller averages over. Which you want depends on whether your traffic mix or your traffic volume is the thing that moves.

§2

Where the acceptance number lives

Do not compute acceptance from output token counts. Both engines already track it, both include the bonus token in the length and exclude it from the rate, and getting that convention wrong is a consistent one-token offset that looks exactly like a better draft.

vLLM: three counters, and a harness that already diffs them

The Prometheus counters are declared here, and the class docstring above them is the definition of every derived quantity:

vllm/v1/spec_decode/metrics.py:L177-L196 vLLM

class SpecDecodingProm:
    """Record spec decoding metrics in Prometheus.

    The acceptance rate can be calculated using a PromQL query:

      rate(vllm:spec_decode_num_accepted_tokens_total[$interval]) /
      rate(vllm:spec_decode_num_draft_tokens_total[$interval])

    The mean acceptance length (conventionally including bonus tokens)
    can be calculated using:

      1 + (
      rate(vllm:spec_decode_num_accepted_tokens_total[$interval]) /
      rate(vllm:spec_decode_num_drafts[$interval]))

    A per-position acceptance rate vector can be computed using

      vllm:spec_decode_num_accepted_tokens_per_pos[$interval] /
      vllm:spec_decode_num_drafts[$interval]
    """

vllm/v1/spec_decode/metrics.py:L227-L232 vLLM

        else:
            counter_specs = [
                ("vllm:spec_decode_num_drafts", "Number of spec decoding drafts."),
                ("vllm:spec_decode_num_draft_tokens", "Number of draft tokens."),
                ("vllm:spec_decode_num_accepted_tokens", "Number of accepted tokens."),
            ]

You do not have to write that PromQL. vllm bench serve scrapes /metrics before and after the measured window and reports the delta, which is the correct thing to do — a counter read once tells you about the server's whole lifetime, not about your run:

vllm/benchmarks/serve.py:L1107-L1119 vLLM

        if delta_draft_tokens > 0:
            acceptance_rate = (delta_accepted / delta_draft_tokens) * 100
            acceptance_length = (
                1 + delta_accepted / delta_drafts if delta_drafts > 0 else 0.0
            )
            spec_decode_stats = {
                "num_drafts": delta_drafts,
                "draft_tokens": delta_draft_tokens,
                "accepted_tokens": delta_accepted,
                "acceptance_rate": acceptance_rate,
                "acceptance_length": acceptance_length,
                "per_position_acceptance_rates": per_pos_rates,
            }

It prints an Acceptance rate (%) / Acceptance length block, and a per-position vector, at the end of the run (vllm/benchmarks/serve.py:L1381-L1408), and writes the same three fields into the result JSON as spec_decode_acceptance_rate, spec_decode_acceptance_length and spec_decode_per_position_acceptance_rates (L1300-L1308). That per-position vector is the $r_i$ you need for exercise 2; nothing else in either project exposes it.

For per-request acceptance — which is what you want if you are bucketing a mixed workload rather than running one category at a time — vLLM has an opt-in response field:

vllm/config/observability.py:L48-L53 vLLM

    per_request_spec_decode_metrics: Literal["none", "summary", "detailed"] = "none"
    """Include per-request speculative-decoding acceptance metrics in the
    response under `metrics.speculative_decoding`. `none` disables; `summary` adds mean
    acceptance length, draft acceptance rate, and the step-by-draft-length
    histogram; `detailed` additionally records the ordered per-step
    accepted/proposed arrays (one entry per verify step). Only reported for

vllm/entrypoints/openai/engine/protocol.py:L132-L151 vLLM

class SpeculativeDecodingMetrics(OpenAIBaseModel):
    """Per-request speculative-decoding acceptance metrics.

    Experimental, subject to change. Only populated for single-sequence requests
    (`n == 1`); `null` for `n > 1`, mirroring the timing metrics.
    """

    mean_acceptance_length: float
    draft_acceptance_rate: float
    # Dense histogram: index j holds the number of verify steps that accepted
    # exactly j draft tokens (length num_spec_tokens + 1). Excludes the
    # always-accepted bonus token.
    acceptance_histogram: list[int]
    num_spec_steps: int
    num_accepted_draft_tokens: int
    num_draft_tokens: int
    num_spec_tokens: int
    # Ordered per-verify-step arrays; populated only at the `detailed` level.
    per_step_accepted: list[int] | None = None
    per_step_drafted: list[int] | None = None

SGLang: a per-request field and a decode-log line

SGLang reports the same statistic per request, under return_spec_tokens_details on the request body:

python/sglang/srt/entrypoints/openai/protocol.py:L418-L427 SGLang

class SpecTokensDetails(BaseModel):
    """Per-request speculative decoding statistics."""

    spec_accept_rate: float = 0.0
    spec_accept_length: float = 0.0
    spec_cap_length: float = 0.0
    spec_block_accept_length: float = 0.0
    spec_num_correct_drafts: int = 0
    spec_num_proposed_drafts: int = 0
    spec_verify_ct: int = 0

and prints a running figure into the decode log every interval. Read the arithmetic carefully, because it is not the same denominator as vLLM's:

python/sglang/srt/managers/scheduler_components/metrics_reporter.py:L818-L830 SGLang

            spec_cap_length = 0
            spec_block_accept_length = 0
        else:
            spec_accept_length = self.spec_num_accept_tokens / self.spec_num_forward_ct
            num_correct_drafts = self.spec_num_accept_tokens - self.spec_num_forward_ct
            if get_spec().speculative_num_draft_tokens:
                draft_per_round = get_spec().speculative_num_draft_tokens - 1
            else:
                draft_per_round = get_spec().speculative_num_steps or 0
            total_draft_tokens = self.spec_num_forward_ct * draft_per_round
            spec_accept_rate = (
                num_correct_drafts / total_draft_tokens if total_draft_tokens > 0 else 0
            )

spec_accept_length is accepted-plus-bonus tokens over forward passes — the same $E$ vLLM calls acceptance_length. spec_accept_rate divides correct drafts by proposed drafts, where "proposed" is speculative_num_draft_tokens - 1 per round rather than $\kappa$ — so under a tree draft the two engines' "acceptance rate" are not the same ratio and must not be tabulated together. The lengths are comparable. Compare $E$; ignore the rates across engines.

Aggregate

SGLang also folds a run-lifetime figure into /get_server_info as avg_spec_accept_length (python/sglang/srt/managers/scheduler.py:L4404-L4411). It is a lifetime average and does not reset between your runs, so diff it the way vLLM's harness diffs its counters, or flush and restart.

§3

Running it

Part A: hold the model and $\kappa$ fixed and vary only the workload. Both projects ship a dataset built for this. vLLM carries Spec-Bench, whose rows are tagged by category, and its own docs enumerate them:

docs/benchmarking/cli.md:L371-L371 vLLM

Available categories include `[writing, roleplay, reasoning, math, coding, extraction, stem, humanities, translation, summarization, qa, math_reasoning, rag]`.

SGLang carries SPEED-Bench, whose categories are graded by output entropy directly — which is a workload descriptor, not acceptance itself. Acceptance is $1-D_{\mathrm{TV}}(p,q)$: a draft with $q=p$ has acceptance one at any entropy.

python/sglang/benchmark/serving.py:L2266-L2272 SGLang

    parser.add_argument(
        "--speed-bench-category",
        type=str,
        default=None,
        choices=["low_entropy", "mixed", "high_entropy"],
        help="Category filter for the speed-bench dataset.",
    )

shell — acceptance, one category at a time shell

# server: n-gram proposer, no draft weights, kappa = 3
vllm serve meta-llama/Meta-Llama-3-8B-Instruct \
    --speculative-config '{"method": "ngram", "num_speculative_tokens": 3,
                           "prompt_lookup_max": 5, "prompt_lookup_min": 2}'

# one run per category; the acceptance block is printed at the end of each
for CAT in summarization translation qa math_reasoning rag writing; do
  vllm bench serve --model meta-llama/Meta-Llama-3-8B-Instruct \
      --dataset-name spec_bench --dataset-path ./question.jsonl \
      --spec-bench-category "$CAT" --num-prompts 200 --request-rate 4 \
      --save-result --result-filename "accept-$CAT.json"
done

# or let run.py drive the loop and print one table
$ python3 run.py accept --categories summarization translation qa writing

Part B: the break-even sweep. Hold the workload fixed, sweep concurrency, and run the whole sweep twice — once with the speculative server and once with a plain one. The number that crosses 1.0 is the ratio of output-token throughput, not of latency; speculation improves latency at every batch size and that is not the question.

shell — the crossing shell

# two servers, identical in every other respect, one at a time
vllm serve meta-llama/Meta-Llama-3-8B-Instruct --max-num-seqs 320          # baseline
vllm serve meta-llama/Meta-Llama-3-8B-Instruct --max-num-seqs 320 \
    --speculative-config '{"method": "ngram", "num_speculative_tokens": 3,
                           "prompt_lookup_max": 5, "prompt_lookup_min": 2}'

for C in 8 16 32 64 96 128 160 192 256; do
  vllm bench serve --model meta-llama/Meta-Llama-3-8B-Instruct \
      --dataset-name random --random-input-len 2048 --random-output-len 256 \
      --max-concurrency $C --num-prompts $((C*8)) --seed 1 \
      --save-result --result-filename "spec-c$C.json"
done

$ python3 run.py sweep --concurrency 8 16 32 64 96 128 160 192 256

Read §10.3 first

That sweep uses --max-concurrency, which is a closed loop with the client-side clock started after the semaphore — every latency percentile it reports excludes client queueing. For finding a throughput crossing that is fine and it is the right tool, because you want the batch size pinned. It is not fine for any latency claim you make afterwards. §10.3 has the mechanism; flush the prefix cache between runs or the second half of your sweep is measuring the first half's leftovers.

Part C separates draft quality from speculation's cost. vLLM will fake the acceptance rate for you, which sounds like cheating and is in fact the cleanest experiment on this page: it holds $E$ exactly constant across the whole batch sweep, so whatever curvature you measure is the roofline and nothing else.

vllm/config/speculative.py:L219-L241 vLLM

    rejection_sample_method: RejectionSampleMethod = "standard"
    """The rejection sampling method to use. 'standard' uses probabilistic
    rejection sampling (with or without cached draft logits, controlled by
    draft_sample_method). 'synthetic' accepts draft tokens with a decaying
    probability calibrated to synthetic_acceptance_rate. 'block' uses block
    verification (Sun et al.), which jointly verifies the draft tokens as a
    block instead of one at a time."""

    synthetic_acceptance_rates: list[float] | None = None
    """Per-position *unconditional* acceptance rates for synthetic rejection
    sampling. Position i's entry is the marginal probability that the first
    i+1 draft tokens are all accepted; the list must have length
    num_speculative_tokens, each entry in [0, 1], and be monotonically
    non-increasing. Only valid when rejection_sample_method is 'synthetic'.
    Mutually exclusive with synthetic_acceptance_length."""

    synthetic_acceptance_length: float | None = None
    """Target mean acceptance length for synthetic rejection sampling, in
    [1, num_speculative_tokens + 1]. Resolved internally to
    synthetic_acceptance_rates. Only valid when rejection_sample_method is 'synthetic'.
    Mutually exclusive with synthetic_acceptance_rates."""

    enable_adaptive_verification: bool = False

shell — hold E fixed, vary only the batch shell

vllm serve meta-llama/Meta-Llama-3-8B-Instruct --max-num-seqs 320 \
    --speculative-config '{"method": "ngram", "num_speculative_tokens": 3,
                           "prompt_lookup_max": 5, "prompt_lookup_min": 2,
                           "rejection_sample_method": "synthetic",
                           "synthetic_acceptance_length": 2.533}'

Re-run the Part B sweep against that server. The measured crossing should land on the $B^{*} \approx 187$ zero-draft-cost row of the table above for $E = 2.533$, $\kappa = 3$ — and if it does not, the gap is everything the §6.2 model leaves out: the sampler's top-k/top-p sort, which speculation multiplies by $\kappa+1$ rows; the KV slots reserved per request per step; and the draft proposer's own cost, which the n-gram path makes small but not zero.

§4

What to expect

Acceptance is a property of the workload, not of the model

The n-gram proposer copies from the prompt, so its acceptance is a direct function of how much of the output is already present in the input. Summarization and RAG-style extraction quote heavily and should accept well; open-ended writing has nothing to copy and should collapse. A draft model behaves differently — its $\alpha = 1 - D_{\mathrm{TV}}(p, q)$ tracks how well the small model approximates the big one, which is worst exactly where the target is most uncertain. Both effects point the same way on high-entropy text and for different reasons, which is why SGLang's categories are named after entropy and vLLM's after task.

Temperature moves α and nothing else

Raising temperature changes $p$ and may also change $q$; TV distance and acceptance can increase or decrease. Run the same category at --temperature 0 and --temperature 1.0 and you have the cheapest possible sensitivity study. Note what does not change: the output distribution is still exactly the target's, at every temperature — §6.2 proves it. Speculation trades compute for latency and never quality, unless you touch --speculative-accept-threshold-acc, which voids the proof deliberately.

Test the geometric model rather than assuming it

The i.i.d. model says $r_i = \alpha^i$. Real acceptance can decay faster or slower than a geometric fit. One possible failure mechanism is that a draft that has already been wrong once is drafting from a context it partly invented. Fit $\alpha$ from $r_0$ and plot $\alpha^{i+1}$ against the measured $r_i$; the gap at $i=2$ and $i=3$ is the whole reason the optimal $\kappa$ is smaller than the geometric model suggests.

Where the crossing actually comes from

Why your measured crossing may sit left of the derived one. Every row costs throughput at high batch and none of them is in the $S(B)$ formula.

Cause	Direction	How to confirm it
KV slots reserved for drafting, per request, per step	large	Compare the engine's reported cache capacity and max concurrency with speculation on and off. SGLang reserves $2\max(\text{topk}\cdot\text{steps},\ \kappa{+}1)$ per request per step (§6.2); vLLM subtracts a drafting budget from the token budget every step. You lose concurrency before you lose throughput.
Sampler sort, multiplied by $\kappa+1$ rows	medium	Re-run with top-k and top-p disabled. If the gap shrinks, you were paying for `apply_top_k_top_p`, which §6.2 flags with the source's own warning.
CUDA-graph bucketing across the concurrency sweep	medium	Sweep on the capture ladder's rungs. A plateau is bucketing (§8.1), not the crossing.
Prefix-cache carryover between the two arms	either way	Flush between runs. The n-gram proposer is especially sensitive: a warm cache changes which prompts are even prefilled.
Long context keeps the step bandwidth-bound	right, not left	At 8k context the KV read dominates and is independent of $\kappa$, so drafted positions stay free far past $B^{*}$. Re-run at `--random-input-len 8192` and watch the crossing move right — this is the one effect that helps you.

Unverified

I could not find, at either SHA, any per-position acceptance vector on the SGLang side comparable to vLLM's vllm:spec_decode_num_accepted_tokens_per_pos. SGLang exposes spec_correct_drafts_histogram per request (python/sglang/srt/entrypoints/openai/protocol.py:L418-L427), which is a histogram over accepted counts rather than a rate per position; the two carry the same information for a chain draft but are not the same array, and I did not find a conversion in tree. I looked in python/sglang/srt/managers/scheduler_components/metrics_reporter.py and python/sglang/srt/entrypoints/openai/utils.py. Exercise 2 is therefore a vLLM exercise unless you derive $r_i$ from the histogram yourself.

§5

Exercises

Before running anything: you measure $E = 2.10$ with $\kappa = 3$ on the n-gram proposer. What $\alpha$ does the i.i.d. model imply, and what break-even batch does that predict? Now measure it. Which of the two numbers do you trust, and why does the disagreement matter more than its size?
Read the vector, not the scalar. Run one category with --spec-tokens 5 and record spec_decode_per_position_acceptance_rates. Fit $\alpha$ from position 0 alone, then plot $\alpha^{i+1}$ against the measured $r_i$. At which position does the geometric model first overstate reality by more than 20%, and what does that say about the $\kappa$ you should be running?
Predict, then verify. Run the same category at temperature 0 and at temperature 1.0. Predict the direction and rough magnitude of the change in $E$ before you look, from $\alpha = 1 - D_{\mathrm{TV}}(p, q)$. Then answer a second question: did the output distribution change? Cite the part of §6.2 that settles it.
Read the file. Open vllm/config/speculative.py and find synthetic_acceptance_rates. It must be "monotonically non-increasing". Say why that is a correctness requirement and not a style rule — what would a non-monotone vector mean about the early-terminating rejection loop? Then find the function in vllm/v1/spec_decode/utils.py that converts these unconditional rates into conditional ones and explain the division.
Run the Part B sweep twice: once with a real proposer, once with rejection_sample_method: "synthetic" pinned to the $E$ you measured. The two crossings should coincide. If the synthetic one sits to the right, what did the real proposer cost you that the model does not charge for? Name it, then measure it.

Answers

Solve $(1-\alpha^{4})/(1-\alpha) = 2.10$ numerically: $\alpha \approx 0.575$. With $c \approx 0$ for n-gram, $B^{*} = 295 \times 2.10/4 \approx 155$. The measured crossing will almost certainly land left of that, and the size of the gap matters less than its direction: the model charges only for extra token positions in the verify step, so anything that moves the crossing left is a cost the model does not know about — reserved KV, the sampler sort, the proposer's own time. Every one of those is in the table above and can be measured separately. (Derived; arithmetic.)
Plot measured survival against $\alpha^{i+1}$, using $\alpha=r_0$. There may be no position with a 20% deficit. If a deficit exists, the marginal yield is below the fitted prediction; whether to shorten the draft still depends on actual marginal verification and proposal costs. A geometric fit is an assumption, not a lower or upper bound.
There is no fixed sign for the effect on $E$: compare the target and draft distributions at each temperature. Exact speculation preserves the target distribution at that temperature; changing temperature changes the target itself. §6.2's proof is an identity chain with no assumption on $q$ whatsoever — every emitted token is drawn from exactly $p$ at whatever temperature $p$ was computed at. A bad draft costs throughput and nothing else.
$r_i$ is the unconditional probability that the first $i{+}1$ drafts are all accepted, so it is a survival function: $r_i \le r_{i-1}$ by construction, since the event at $i$ is a subset of the event at $i-1$. A non-monotone vector would make the conditional rate $c_i = r_i/r_{i-1}$ exceed 1, i.e. a probability greater than one, and the early-terminating loop would be sampling from something that is not a distribution. The converter is unconditional_to_conditional_rates (vllm/v1/spec_decode/utils.py:L598-L601); the division is exactly $c_i = p_i / p_{i-1}$ with $p_{-1} = 1$.
The real proposer costs draft-side time and draft-side KV that the synthetic path does not pay: n-gram costs a lookup and a scheduler round trip, a draft model costs $\kappa$ sequential forward passes at $c$ each. Measure it by taking the difference in output-token throughput between the two arms at a batch small enough to be well below either crossing, where the roofline term is identical and only the proposer differs.

§6

Key takeaways

Both engines already compute mean acceptance length with the bonus token included; compute it yourself from output token counts and you will be off by exactly one. vllm bench serve diffs the Prometheus counters around your measured window and prints the answer — a counter read once describes the server's whole lifetime, not your run.
Acceptance rate is not comparable across engines: vLLM divides accepted by drafted tokens, SGLang divides correct drafts by speculative_num_draft_tokens - 1 per round. Acceptance length is comparable. Tabulate lengths.
The break-even batch is nearly flat in α and steeply decreasing in $\kappa$: at $\alpha = 0.7$, $\kappa=1$ breaks even at 226 concurrent sequences and $\kappa=3$ at 150 (derived). A better draft buys latency; a shorter draft buys headroom. Both engines therefore adapt $\kappa$ at runtime rather than toggling speculation — vLLM on batch size, SGLang on the measured acceptance rate. No draft, however good, moves the crossing past the roofline ridge at all.
Pin $E$ with rejection_sample_method: "synthetic" and the batch sweep isolates the roofline from the draft. Anything left over is reserved KV, the $\kappa{+}1$× sampler sort, or the proposer — three separately measurable things that the textbook $S(B)$ formula charges nothing for.
Measure at your real context length. At 8k the step stays bandwidth-bound far longer and the crossing moves right; a break-even measured on 128-token prompts is a number about your benchmark, not about your service.