The V1 scheduler, in code

You have a scheduling bug. Maybe requests are starving, maybe the token budget is being overrun, maybe a preempted request came back with the wrong num_computed_tokens. So you open the file:

$ wc -l vllm/v1/core/sched/scheduler.py
3037 vllm/v1/core/sched/scheduler.py
$ grep -c "    def " vllm/v1/core/sched/scheduler.py
54

Fifty-four methods. The one you want, schedule(), starts at line 484 and ends at line 1327 — a single function longer than most files in the repo, with a five-deep nesting level at its worst point, two nested while loops, and an inner while True that mutates the very list the outer loop is indexing into. There is no unit of it you can read in isolation, because almost every branch adjusts a counter that a later branch reads.

Worse, a naive reading is actively misleading. You will miss that request.num_computed_tokens is advanced after SchedulerOutput is built, in a different method; that a preemption inside the running loop can rewind the loop cursor; and that the two budget counters in scope are not the same budget.

§1.4 owns the concepts — the state machine, admission, the preemption policy, recompute-vs-swap, starvation, the watermark. None of that is repeated here. This chapter owns the code: where things are, what calls what, what each block of schedule() actually does, and which asserts encode the contract.

Strip everything else away and schedule() is a single-pass greedy fill of a token budget across two ordered collections. First the self.running list, in list order, index by index. Then the waiting queues, front to back. Each request that fits takes num_new_tokens off the budget and gets KV blocks; each one that does not is either skipped (continue) or ends the pass (break), and which of the two it is encodes a policy decision.

Three properties make it readable:

Before walking the code, get the layout in your head. Every range below is from grep -n " def " vllm/v1/core/sched/scheduler.py at a556f3f; the end of each range is the line before the next def.

That last row matters more than it looks. Roughly a ninth of the file — 339 of 3,037 lines — is connector code: _connector_finished (L2699), _request_remaining_blocks (L2736), _inflight_prefill_reserved_blocks (L2750), _update_waiting_for_remote_kv (L2757), _update_from_kv_xfer_finished (L2836), _update_requests_with_invalid_blocks (L2865) and _handle_invalid_blocks (L2968), none of which executes unless vllm_config.kv_transfer_config is non-None. Do not write off the whole L2699-L3037 span, though: _try_promote_blocked_waiting_request (L2800-L2835) sits inside it and is called unconditionally from the waiting loop at L775-L777, because it also unblocks requests waiting on grammar compilation and on streaming input — plain single-node features.

The entry point declares the algorithm in a comment before it declares a variable:

    def schedule(self, throttle_prefills: bool = False) -> SchedulerOutput:
        self.current_step += 1
        # NOTE(woosuk) on the scheduling algorithm:
        # There's no "decoding phase" nor "prefill phase" in the scheduler.
        # Each request just has the num_computed_tokens and
        # num_tokens_with_spec. num_tokens_with_spec =
        # len(prompt_token_ids) + len(output_token_ids) + len(spec_token_ids).
        # At each step, the scheduler tries to assign tokens to the requests
        # so that each request's num_computed_tokens can catch up its
        # num_tokens_with_spec. This is general enough to cover
        # chunked prefills, prefix caching, speculative decoding,
        # and the "jump decoding" optimization in the future.

Take this literally. There is no prefill branch and no decode branch in the 844 lines. There is one quantity, num_tokens_with_spec - num_computed_tokens, and the method decides how much of that gap to close.

The two budgets

        req_to_new_blocks: dict[str, KVCacheBlocks] = {}
        num_scheduled_tokens: dict[str, int] = {}
        token_budget = self.max_num_scheduled_tokens
        spec = self.vllm_config.speculative_config
        draft_slots = spec.max_num_new_slots_for_drafting if spec is not None else 0
        input_budget = self.scheduler_config.max_num_batched_tokens
        if self._pause_state == PauseState.PAUSED_ALL:
            # Do not schedule any requests when paused.
            token_budget = 0

This is the first thing a casual reader gets wrong. There are two budgets, and they are decremented by different amounts. token_budget counts tokens the scheduler issues. input_budget counts rows the model runner's input buffers must hold — which is larger, because with speculative decoding the worker appends draft tokens the scheduler never issued. So every admission does:

            token_budget -= num_new_tokens
            input_budget -= num_new_tokens + draft_slots
            req_index += 1

and both loops carry a guard if input_budget <= draft_slots: break at their top, so the pass stops admitting once one more request could not have its drafts accommodated. max_num_scheduled_tokens defaults to max_num_batched_tokens when unset (vllm/config/scheduler.py:L56-L61), which is why the distinction is invisible without spec decode on. The class constant for both is 2,048 (vllm/config/scheduler.py:L42-L49), but the field's own docstring calls that value "mainly for convenience when testing" — a served engine never sees it. EngineArgs.get_batch_defaults (vllm/engine/arg_utils.py:L2580-L2638) resolves the budget by device and usage context, and on an H100 that is 8,192 for vllm serve and 16,384 for the in-process LLM class. For Llama-3-8B at 128 KiB of KV per token, an 8,192-token step writes 8192 × 128 KiB = 1 GiB of fresh KV in a single forward pass, which is the arithmetic that keeps the budget from going much higher.

Phase 1: the running loop

        # First, schedule the RUNNING requests.
        req_index = 0
        while req_index < len(self.running) and token_budget > 0:
            request = self.running[req_index]

Note the shape: a manual index into a mutable list, not for request in self.running. That is why the preemption branch below can splice an element out mid-iteration without corrupting the walk.

Between L536 and L564 sit three continue guards — async-scheduling exhaustion, the PP decode-cadence gate next_decode_eligible_step, and the DP prefill-throttle deferral. Each of them advances req_index and moves on, so a blocked request never blocks the requests behind it. Then the arithmetic:

            num_new_tokens = (
                request.num_tokens_with_spec
                + request.num_output_placeholders
                - request.num_computed_tokens
            )
            if 0 < self.scheduler_config.long_prefill_token_threshold < num_new_tokens:
                num_new_tokens = self.scheduler_config.long_prefill_token_threshold
            num_new_tokens = min(
                num_new_tokens, token_budget, input_budget - draft_slots
            )

That is the token-budget clamp of chunked prefill, which §1.5 owns. Four further clamps follow: max_model_len − num_computed_tokens − num_sampled_tokens_per_step (L579-L584), Mamba block alignment (L587-L590), the encoder compute budget via _try_schedule_encoder_inputs (L596-L608), and the multi-module-MTP prefill-lookahead reserve _reserve_prefill_lookahead (L612-L614). Each can only shrink num_new_tokens. If the result is zero:

                # NOTE(woosuk): Here, by doing `continue` instead of `break`,
                # we do not strictly follow the FCFS scheduling policy and
                # allow the lower-priority requests to be scheduled.
                req_index += 1
                continue

            # Schedule newly needed KV blocks for the request.

This is a documented departure from strict FCFS: a blocked earlier request can be skipped for a later eligible one. It can be completely deterministic given identical state. Nondeterminism instead requires varying arrivals, asynchronous results or other inputs; policy ordering and reproducibility are separate properties.

The allocation loop and what None means

            # Schedule newly needed KV blocks for the request.
            with record_function_or_nullcontext("schedule: allocate_slots"):
                while True:
                    new_blocks = self.kv_cache_manager.allocate_slots(
                        request,
                        num_new_tokens,
                        num_lookahead_tokens=self.num_lookahead_tokens,
                    )

                    if new_blocks is not None:
                        # The request can be scheduled.
                        break

allocate_slots returns KVCacheBlocks | None (vllm/v1/core/kv_cache_manager.py:L347-L360). None is not an error. It means "there were not enough free blocks under the watermark", and it is the sole signal that triggers preemption. It is also the sole failure mode: no exception, no log line, no counter incremented at this level. If you are looking for why a request did not get scheduled, this is the return value to breakpoint on. Note that the running-loop call passes only three arguments; the waiting-loop call at L1041-L1054 passes eleven, because admission has to reason about prefix-cache hits, external KV, and full-sequence reservation that a decode step does not.

On None, the inner while True preempts and retries. In FCFS mode the victim is the tail:

                    else:
                        preempted_req = self.running.pop()

                    self._preempt_request(
                        preempted_req,
                        scheduled_timestamp,
                        drop_stale_output=self.requires_kv_delivery,
                    )
                    preempted_reqs.append(preempted_req)
                    if preempted_req == request:
                        # No more request to preempt. Cannot schedule this request.
                        break

            if new_blocks is None:
                # Cannot schedule this request.
                break

Three things to read carefully here. First, the loop terminates because self.running shrinks by one per iteration and the request being scheduled is itself in that list — when it becomes its own victim, the loop stops. Second, in PRIORITY mode (L651-L684) the victim can be a request already scheduled earlier in this same pass, so the code pops it out of scheduled_running_reqs and refunds num_scheduled_tokens, token_budget, input_budget, req_to_new_blocks, scheduled_spec_decode_tokens, scheduled_encoder_inputs and encoder_compute_budget. It is not a complete rollback, and the gap is worth knowing: prefill_scheduled |= request.is_prefill_chunk (L704) is never un-set, and a stale True changes the spec-decode padding test at L949; request.spec_token_ids = [] (L728) is destructive; and the encoder-cache allocations are freed separately, by _preempt_request at L1364. If you add an accumulator, add it here too — and check whether the two above should have been. Third, the priority path adjusts req_index when the victim sat before the cursor — the comment calls out that omitting this would "silently omit" the next request.

The outer break at L698-L700 is a break, not a continue: once the cache is genuinely full, there is no point trying the rest of the running list.

Phase 2: the waiting loop

        # Next, schedule the WAITING requests.
        if not preempted_reqs and self._pause_state == PauseState.UNPAUSED:
            step_skipped_waiting = create_request_queue(self.policy)

            while (self.waiting or self.skipped_waiting) and token_budget > 0:
                if input_budget <= draft_slots:
                    break
                # Paused streaming sessions (WAITING_FOR_STREAMING_REQ) are not
                # in `running` but still hold a model-runner request slot.
                num_running = len(self.running) + self.num_waiting_for_streaming_input
                if num_running >= self.max_num_running_reqs:
                    break

                request_queue = self._select_waiting_queue_for_scheduling()
                assert request_queue is not None

The guard if not preempted_reqs is the anti-thrash rule: a step that had to evict anyone admits nobody. One line, and the most consequential policy statement in the method.

The body is over 400 lines but structurally it is: peek, filter, look up the prefix cache, size the chunk, allocate, and only then pop_request() at L1090. That ordering matters: the request stays at the head of its queue through every rejection path, so a break leaves the queue exactly as it started.

The filters between L771 and L806 all share a shape: pop_request() then step_skipped_waiting.prepend_request(request). Requests blocked on grammar compilation, on a remote KV transfer, on stale in-flight output, or on the max_loras cap are moved out of the way rather than blocking the queue — and re-queued into self.skipped_waiting at L1169-L1170 after the pass. That is why there are two waiting queues, and why get_request_counts() at L2323-L2325 sums both.

Admission itself is the tail of the loop:

                    scheduled_loras.add(request.lora_request.lora_int_id)
                req_to_new_blocks[request_id] = self.kv_cache_manager.get_blocks(
                    request_id
                )
                num_scheduled_tokens[request_id] = num_new_tokens
                token_budget -= num_new_tokens
                input_budget -= num_new_tokens + draft_slots
                request.status = RequestStatus.RUNNING
                request.num_computed_tokens = num_computed_tokens
                if pad_spec_decode:

Note the asymmetry with the running loop: there, req_to_new_blocks holds the blocks newly returned by allocate_slots; here it holds get_blocks(request_id), the request's entire block table. That distinction is what CachedRequestData.resumed_req_ids encodes downstream — for a resumed request the legacy runner replaces the block table, for a running one it appends. On the V2 path, which is what a dense model takes, that set is always empty: L1202-L1204 folds scheduled_resumed_reqs into scheduled_new_reqs and clears it before _make_cached_request_data runs, so a resumed request is re-sent whole as a NewRequestData with prefill_token_ids, and V2 drops the old state on preemption rather than patching it (vllm/v1/worker/gpu/model_runner.py:L944-L946, L963-L978).

RequestQueue and its two policies

vllm/v1/core/sched/request_queue.py is 208 lines and worth reading in full — it is the simplest file in the subsystem. An ABC at L20-L72 declares ten @abstractmethods; two implementations follow.

class FCFSRequestQueue(deque[Request], RequestQueue):
    """A first-come-first-served queue that supports deque operations."""

    def add_request(self, request: Request) -> None:
        """Add a request to the queue according to FCFS policy."""
        self.append(request)

    def pop_request(self) -> Request:
        """Pop a request from the queue according to FCFS policy."""
        return self.popleft()

FCFSRequestQueue subclasses deque, so prepend_request is appendleft — O(1), which is what makes preemption cheap: _preempt_request pushes the victim back to the front of the waiting queue at L1387. PriorityRequestQueue at L131-L198 wraps a heapq and orders by Request.__lt__ (vllm/v1/request.py:L350-L361): priority, then arrival time, then request id, then object identity. Two of its methods are traps — prepend_request and prepend_requests silently degrade to ordinary inserts, with docstrings that say so. Preemption under PRIORITY therefore does not give the victim any head-of-line advantage.

The Request fields schedule() actually reads

The RequestStatus enum at vllm/v1/request.py:L364-L380 has a load-bearing ordering trick: everything after PREEMPTED is a finished state, and is_finished() is an integer comparison. Insert a non-finished status after PREEMPTED and you break every finish check at once.

SchedulerOutput

vllm/v1/core/sched/output.py is 305 lines and holds five dataclasses — NewRequestData, CachedRequestData, ScheduledEncoderInputStats, SchedulerOutput and GrammarOutput. SchedulerOutput itself (L206-L297) has nine required fields and twelve optional ones. The delta encoding is real and is the reason there are two request lists:

class SchedulerOutput:
    # list of the requests that are scheduled for the first time.
    # We cache the request's data in each worker process, so that we don't
    # need to re-send it every scheduling step.
    scheduled_new_reqs: list[NewRequestData]
    # list of the requests that have been scheduled before.
    # Since the request's data is already cached in the worker processes,
    # we only send the diff to minimize the communication cost.
    scheduled_cached_reqs: CachedRequestData

    # req_id -> num_scheduled_tokens
    # Number of tokens scheduled for each request.
    num_scheduled_tokens: dict[str, int]
    # Total number of tokens scheduled for all requests.
    # Equal to sum(num_scheduled_tokens.values())
    total_num_scheduled_tokens: int

NewRequestData (L35-L75) carries the full prompt token ids, sampling params, LoRA request and block ids — everything the worker needs to build a cache entry. CachedRequestData (L129-L195) is column-oriented: parallel lists req_ids, new_block_ids, num_computed_tokens, num_output_tokens, plus a resumed_req_ids set. Its own comment at L132-L134 states the append-versus-replace rule for new_block_ids. new_token_ids is populated only under pipeline parallelism without async scheduling (scheduler.py:L1506-L1516) — under any other configuration it is an empty list, which is a common source of "why is this always empty" confusion.

On the way back, update_from_output returns dict[int, EngineCoreOutputs] keyed by client index. EngineCoreOutput (vllm/v1/engine/__init__.py:L196-L238) is a msgspec.Struct with array_like=True and omit_defaults=True — positional serialisation, which is why every comment about a new field says "appended last so positional serialization stays backward compatible". If you add a field anywhere but the end, you break rolling upgrades. EngineCoreOutputs (L253-L279) wraps the list plus scheduler_stats, finished_requests, and the DP wave signals.

grep -n "assert" vllm/v1/core/sched/scheduler.py returns 39 hits. They are the author's own statement of what must hold, and the densest cluster is the block that closes the scheduling pass:

        # Check if the scheduling constraints are satisfied.
        total_num_scheduled_tokens = sum(num_scheduled_tokens.values())
        assert total_num_scheduled_tokens <= self.max_num_scheduled_tokens

        assert token_budget >= 0
        assert input_budget >= 0
        assert len(self.running) <= self.max_num_running_reqs
        # Since some requests in the RUNNING queue may not be scheduled in
        # this step, the total number of scheduled requests can be smaller than
        # len(self.running).
        assert len(scheduled_new_reqs) + len(scheduled_resumed_reqs) + len(
            scheduled_running_reqs
        ) <= len(self.running)

Reclamation must respect ownership: an allocation retry may use only slots whose references and outstanding GPU writes are finished. The shown preemption helper calls the free path synchronously in the simple case, but async/connector paths can defer physical availability. If no usable memory was reclaimed, terminate or defer the retry safely rather than spinning. A function call named free is not itself an independent proof of safe reuse.

update_from_output spans L1744-L2168, and its centre is a single loop over scheduler_output.num_scheduled_tokens — the same dict the scheduler wrote. The author flags it as hot:

        # NOTE(woosuk): As len(num_scheduled_tokens) can be up to 1K or more,
        # the below loop can be a performance bottleneck. We should do our best
        # to avoid expensive operations inside the loop.
        stopped_running_reqs: set[Request] = set()
        stopped_preempted_reqs: set[Request] = set()
        for req_id, num_tokens_scheduled in num_scheduled_tokens.items():
            assert num_tokens_scheduled > 0
            request = self.requests.get(req_id)
            output_is_stale = False
            if request is not None:
                request.num_in_flight_tokens -= num_tokens_scheduled

That is the budget for this loop stated plainly: it runs once per forward pass, and for Llama-3-8B on an H100 the weights-only batch-1 decode floor is 4.48 ms (§0.4; §1.1's 6.1 ms is the same step with the KV read at $s = 2{,}048$ added). At the H100 serving default of max_num_seqs = 1024 (vllm/engine/arg_utils.py:L2619-L2628 — not the 128 in the SchedulerConfig class body) the loop body can run a thousand times inside that window; anything allocating per iteration shows up directly in inter-token latency.

The body, in order:

1. Drain in-flight accounting (L1813-L1818). num_in_flight_tokens comes down; if the request was preempted while this step was in flight, its stale share drains too.
2. Skip the dead (L1822-L1830). request is None or request.is_finished() means the request was aborted mid-flight. Continue silently.
3. Spec-decode reconciliation (L1841-L1877). num_accepted = max(len(generated_token_ids) - num_sampled, 0), and rejections roll num_computed_tokens back — the counter _update_after_schedule optimistically advanced.
4. Append and stop-check (L1898-L1901 calling _update_request_with_output at L2212-L2230), which appends each sampled token and calls check_stop from sched/utils.py:L94-L130. That function is where EOS, stop-token, max_tokens, max_model_len and repetition detection are all evaluated, and where request.status is set to its FINISHED_* value. Its signature is check_stop(request, max_model_len) -> bool, so it never sees the token list; the caller trims new_token_ids at the stop point, del new_token_ids[num_new:] at L2227.
5. Grammar advance (L1918-L1944). If a grammar rejects the sampled token the request becomes FINISHED_ERROR with a logged "Unexpected: grammar rejected tokens" line.
6. Retire:

            finish_reason = None
            if stopped:
                # Capture finish_reason BEFORE _handle_stopped_request, which may
                # reset the status to WAITING for streaming requests that continue.
                finish_reason = request.get_finished_reason()
                finished = self._handle_stopped_request(request)
                if finished:
                    kv_transfer_params, ec_transfer_params = self._free_request(request)

                if status_before_stop == RequestStatus.RUNNING:
                    stopped_running_reqs.add(request)
                else:
                    stopped_preempted_reqs.add(request)

Freeing is two-stage. _free_request (L2422-L2450) fires the connector hooks, frees the encoder cache, adds the id to self.finished_req_ids — which the next schedule() ships to the worker so it can drop its cached request state — and then calls _free_blocks, which returns the KV blocks and deletes the entry from self.requests. Removal from the running list happens once, in bulk, after the loop:

        # Remove the stopped requests from the running and waiting queues.
        if stopped_running_reqs:
            self.running = remove_all(self.running, stopped_running_reqs)
        if stopped_preempted_reqs:
            # This is a rare case and unlikely to impact performance.
            self.waiting.remove_requests(stopped_preempted_reqs)
            self.skipped_waiting.remove_requests(stopped_preempted_reqs)

remove_all (sched/utils.py:L62-L91) mutates in place for the single-element case and rebuilds the list otherwise — which is why the assignment self.running = remove_all(...) is mandatory and its docstring says so.

vllm/v1/core/sched/interface.py is the pluggable contract: SchedulerInterface(ABC) at L38 with seventeen @abstractmethods. A custom scheduler must implement __init__, schedule, get_grammar_bitmask, update_from_output, update_draft_token_ids, update_draft_token_ids_in_output, add_request, finish_requests, get_num_unfinished_requests, has_finished_requests, pause_state, set_pause_state, reset_prefix_cache, reset_encoder_cache, get_request_counts, make_stats and shutdown. Six methods are concrete defaults you can inherit: has_unfinished_requests (L173), has_requests (L193), get_kv_cache_usage (L238), and the three connector accessors get_kv_connector, get_ec_connector and get_kv_event_publisher_config (L255-L262), which all return None.

Selection happens in config, not in the engine:

    def get_scheduler_cls(self) -> type["SchedulerInterface"]:
        if self.scheduler_cls is None:
            if self.async_scheduling:
                from vllm.v1.core.sched.async_scheduler import AsyncScheduler

                return AsyncScheduler
            from vllm.v1.core.sched.scheduler import Scheduler

            return Scheduler

        # The first half of this warning can be removed once the Scheduler interface is
        # finalized and we can maintain support for scheduler classes that implement it
        logger.warning_once(
            "Using custom scheduler class %s. This scheduler interface is not public "
            "and compatibility may not be maintained. If you have subclassed Scheduler "
            "instead of AsyncScheduler, you will see degraded performance due to async "
            "scheduling being disabled.",
            self.scheduler_cls,  # type: ignore[arg-type]
        )

Two things to take from this. The interface is explicitly not public and its own warning says compatibility may not be maintained. And async_scheduling is on unless something disables it: the field defaults to None (vllm/config/scheduler.py:L148), and None resolves to True for an ordinary generation model on a supporting executor (vllm/config/vllm.py:L1279-L1328), so AsyncScheduler is what a stock server gets — so a plugin that subclasses Scheduler silently disables async scheduling, which the warning also tells you. Extend AsyncScheduler unless you have a reason not to. EngineCore.__init__ calls get_scheduler_cls() at vllm/v1/engine/core.py:L148 and types the field as SchedulerInterface at L161; §11.5 covers plugin registration generally.

A 900-token prompt arrives, generates 40 tokens, and finishes on EOS. No spec decode, no connector, and --no-async-scheduling so that the base Scheduler runs — a stock server gets AsyncScheduler instead (see §7), in which case steps 9 and 12 run the overrides at async_scheduler.py:L19 and L51 and step 14's num_new_tokens picks up num_output_placeholders. Set breakpoints in this order.

Step 19 is the one people miss: the id is announced to the worker one step after the request finished, and the reset at L1434-L1438 deliberately rebinds rather than calling clear(), because the previously-built SchedulerOutput still holds a reference to that same set object.

Reproduce the map, then watch the budget move. From a vLLM checkout at a556f3f:

# 1. The map, regenerated. Compare against Figure 1.
grep -n "^    def \|^    async def " vllm/v1/core/sched/scheduler.py

# 2. The invariants, as the author wrote them.
grep -n "assert" vllm/v1/core/sched/scheduler.py

# 3. Which branches are dead in your configuration -- and note that
#    use_v2_model_runner is normally ON, so its ELSE arms are the dead ones.
grep -n "self.connector is not None\|self.ec_connector is not None\|self.use_v2_model_runner\|need_mamba_block_aligned_split" vllm/v1/core/sched/scheduler.py

# 4. The exit points of schedule(): every break and continue, with context.
awk 'NR>=484 && NR<=1327 && /break|continue/ {print NR": "$0}' vllm/v1/core/sched/scheduler.py

Then instrument. The single most informative one-line patch to this file is to log the budget at the assert block — insert after L1179:

logger.info(
    "step=%d issued=%d/%d running=%d waiting=%d skipped=%d preempted=%d",
    self.current_step, total_num_scheduled_tokens,
    self.max_num_scheduled_tokens, len(self.running),
    len(self.waiting), len(self.skipped_waiting), len(preempted_reqs),
)

During pure decode, issued tokens approximately follow the number of scheduled requests; prefill-heavy phases can fill the token budget. Occasional preemption under changing lengths is not proof of thrashing. Diagnose repeated eviction/recomputation with poor forward progress, retained-prefix length and completed tokens per unit work.

• vLLM PR #9289 — the V1 core re-architecture that produced this scheduler shape, including the "no prefill phase, no decode phase" framing quoted at L486-L495.
• vLLM PR #19970 — the async-scheduling work behind num_output_placeholders and the AsyncScheduler subclass.
• vLLM V1 alpha announcement — the design rationale for the single-token-budget scheduler, written by the same author as the L486 comment.
• interface.py on main — track drift in the abstract method set; it is explicitly unstable.
• In this book: §1.4 for the concepts, §1.5 for the token-budget clamp, §2.2 for the block pool behind allocate_slots, §11.2 for the engine layering above, §11.4 for what consumes SchedulerOutput, and §12.2 for SGLang's very differently shaped answer.