Repo map, and what V1 changed

You clone vLLM to fix a bug in decode-path attention. You know from the papers that vLLM's signature contribution is a paged attention kernel. So you look for it:

$ ls vllm | grep -x attention     # the package that held every V0 backend
                                  # (no output)

$ ls csrc/attention/
attention_dtypes.h  attention_generic.cuh  dtype_bfloat16.cuh
dtype_float16.cuh   dtype_float32.cuh      dtype_fp8.cuh

Six header files. No .cu. The PagedAttention CUDA kernel was deleted outright in commit d715b3aa1e, "Delete PagedAttention (#47361)", dated 2026-07-02. The top-level attention package that once held every backend is gone too: the last backends under it were removed in PR #25351 (2025-09-21), and the residual utilities migrated out in PR #31916 (2026-01-09).

So you fall back to the model runner, which everyone says is where the tensors get built. There is one obvious candidate:

$ wc -l vllm/v1/worker/gpu_model_runner.py
8008 vllm/v1/worker/gpu_model_runner.py

$ wc -l vllm/v1/worker/gpu/model_runner.py
2024 vllm/v1/worker/gpu/model_runner.py

Two model runners. Both live. If your model is Llama-3-8B, the 8,008-line one is not the one executing your forward pass — and nothing in the file names tells you that. The second one ships with a four-line README that reads, in full:

# [Experimental] Model Runner V2

This directory contains the new model runner which is under active development.
Ping [Woosuk Kwon](https://github.com/WoosukKwon) for any changes.

That is the only architectural README anywhere under vllm/. The repository you are about to read is mid-rewrite, for the second time, and the tree records both events in its directory names. Learning to read those names is most of what this chapter teaches.

The organising principle is simple once you see it: vLLM's directory boundaries follow its process boundaries. A V1 deployment is not one program, it is three to four kinds of OS process talking over ZeroMQ, and each kind owns a distinct slice of the tree. Everything else — models, kernels, quantization, distributed primitives — is a shared library that the worker process links against.

Read the tree top-down through that picture and almost every directory places itself. HTTP and text live in the front; scheduling and block accounting live in the middle; tensors and kernels live in the back; config is the one thing everybody shares. The two rewrites this chapter tells are both about where the seam sits — V0→V1 moved the seam from "one Python process" to "a ZMQ boundary", and V1→V2 is moving the internals of the back ring from a monolith to a package.

Both rewrites are answers to the same arithmetic. Take the book's reference decode point: Llama-3-8B in bf16 on an H100 SXM (989.4 TFLOP/s bf16, 3.35 TB/s HBM, ridge point $I^{*} = 295$ FLOP/byte). At batch 1 the decode step is pure weight streaming, so the floor is

Here $W_{\text{bytes}}=15.0$ GB is a rounded streamed-weight assumption, not the complete resident parameter footprint. The stated 8.03 billion parameters at two bytes occupy 16.06 GB, or about 14.96 GiB; streaming all of them once at 3.35 TB/s gives 4.79 ms. Embedding lookup does not read every embedding row, so a smaller streamed set may be appropriate, but its membership must be specified. Both figures are bandwidth models, not measured step times.

Using the illustrative 15 GB streamed set, 4.48 ms is a lower-bound GPU term, not the entire allowable wall-clock budget. Adding 2 ms of non-overlapping CPU work gives 6.48 ms in that model. Real step time also depends on KV traffic, kernels, achieved bandwidth and batch shape; CPU work can become limiting without being guaranteed to grow exactly linearly.

Two structural fixes follow, and they are exactly the two rewrites:

Keep that ordering in mind. V1 was about process topology; V2 is about the interior of one process. They are not competing designs — V2 lives entirely inside V1's worker.

Here is the tree with responsibilities attached. Everything is relative to the repo root.

The engine: vllm/v1/

357 Python files, ~149,000 lines. The v1/ prefix is not a legacy marker — it is the live engine. Its subdirectories:

The model library: vllm/model_executor/ and vllm/models/

model_executor/ is the largest package in the repo — 768 files, ~342,600 lines: layers/ (linear, layernorm, rotary embedding, fused_moe/, quantization/, and attention/ — the nn.Module models actually call), model_loader/, kernels/, and models/ with 311 single-file model definitions.

Note the attention split carefully, because it is the single most common navigation mistake: model_executor/layers/attention/attention.py is the layer a model constructs; v1/attention/backends/*.py is the backend that builds per-step metadata and calls the kernel. Layer in the model library, backend in the engine.

vllm/models/ is new — first commit 2026-05-18, 195 files. It is not a duplicate of model_executor/models/. It is a per-model package layout for the handful of frontier architectures that outgrew one file: deepseek_v32/, deepseek_v4/, kimi_k3/, minimax_m3/, inkling/, dots3_note/, plus common/. Inside deepseek_v4/ you get attention.py, sparse_mla.py, compressor.py, quant_config.py and — tellingly — nvidia/, amd/ and xpu/ subdirectories. When a model needs its own kernels per vendor, one file stops working.

The recent additions

Four directories that did not exist a year before this SHA, and that older tutorials will not mention:

csrc/: mostly not where you think

309 files, but 187 of them are under csrc/libtorch_stable/ — a migration to PyTorch's stable ABI that began 2026-03-19 (PR #31509) and has since absorbed most active kernels: cache_kernels.cu, activation_kernels.cu, the fused DeepSeek-V4 and Kimi-K3 kernels, cutlass_extensions/, and the real MoE sources. What is left at the old paths is residue: csrc/moe/ contains exactly one file (dynamic_4bit_int_moe_cpu.cpp) and csrc/attention/ contains only dtype headers. csrc/cpu/ (71 files) and csrc/rocm/ (9) remain vendor-specific.

Contrast: how SGLang lays out the same concerns

SGLang at 7d89325 has 3,397 Python files under python/sglang/ and no version-named directory anywhere — find python/sglang -type d -name 'v[0-9]*' returns nothing. Its runtime sits in python/sglang/srt/, organised strictly by concern: managers/ (scheduler, tokenizer manager, detokenizer manager, TP worker), mem_cache/, model_executor/, layers/, speculative/, disaggregation/. Where vLLM forked v1/worker/gpu/ next to v1/worker/gpu_model_runner.py, SGLang decomposed in place: srt/model_executor/model_runner.py is 2,103 lines with the rest hoisted into model_runner_components/ (14 files), runner/ and runner_backend/. Same destination, opposite strategy — SGLang pays in churn on one file's history, vLLM in two implementations kept alive at once.

V0 was the original vLLM engine: one Python process running a synchronous LLMEngine.step() that tokenized, scheduled, ran the model, sampled, detokenized and returned — in sequence. The in-tree migration guide states the motivation plainly:

vLLM V0 successfully supported a wide range of models and hardware, but as new features were developed independently, the system grew increasingly complex. This complexity made it harder to integrate new capabilities and introduced technical debt, revealing the need for a more streamlined and unified design.

Building on V0's success, vLLM V1 retains the stable and proven components from V0
(such as the models, GPU kernels, and utilities). At the same time, it significantly
re-architects the core systems, covering the scheduler, KV cache manager, worker,
sampler, and API server, to provide a cohesive, maintainable framework that better
accommodates continued growth and innovation.

The three concrete costs V1 attacked, and its answers:

1. The synchronous step loop. CPU work sat between GPU steps, not beside them. V1's answer, per the announcement blog, was "an isolated EngineCore execution loop that focuses exclusively on the scheduler and model executor", enabling "greater overlap of CPU-intensive tasks—such as tokenization, multimodal input processing, de-tokenization, and request streaming—with the core execution loop."
2. Python overhead on the critical path. V0 rebuilt input tensors from scratch each step. V1 introduced the persistent batch: keep state tensors alive across steps and apply incremental diffs. The design doc for its successor describes the original rationale precisely — "Building these tensors from scratch each step is often very slow in Python, especially for large tensors like block tables" (docs/design/model_runner_v2.md:L17).
3. Scheduler/worker coupling. V1 caches request state worker-side and, per the blog, transmits "incremental updates (diffs) at each step", letting the scheduler and worker 0 run as separate processes with "a clean, symmetric architecture".

The performance claim, cited: the V1 alpha announcement (blog.vllm.ai, 2025-01-27) reports "up to 1.7x higher throughput compared to V0 (without multi-step scheduling)" for Llama 3.1 8B and Llama 3.3 70B on the ShareGPT dataset, and "consistently lower latency than V0 especially at high QPS". Larger speedups are claimed for Qwen2-VL on VisionArena, attributed to input-processing offload. No measurement in this book reproduces those numbers; treat them as the authors' published figures with the configuration as stated.

What remains of V0 in the tree is a set of tombstones. vllm/engine/ has five files, and two of them are aliases:

# SPDX-License-Identifier: Apache-2.0
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project

from vllm.v1.engine.llm_engine import LLMEngine as V1LLMEngine

LLMEngine = V1LLMEngine  # type: ignore
"""The `LLMEngine` class is an alias of [vllm.v1.engine.llm_engine.LLMEngine][]."""

async_llm_engine.py is the same seven lines pointing at vllm.v1.engine.async_llm.AsyncLLM. The real V0 LLMEngine was deleted in PR #25033 (2025-09-20). What is left in vllm/engine/ that is not a shim: arg_utils.py (2,909 lines — still the CLI argument surface, and still one of the hottest files in the repo) and protocol.py (281 lines).

The live equivalent of V0's step() now looks like this, inside the EngineCore process:

    def step(self) -> tuple[dict[int, EngineCoreOutputs], bool]:
        """Schedule, execute, and make output.

        Returns tuple of outputs and a flag indicating whether the model
        was executed.
        """

        # Check for any requests remaining in the scheduler - unfinished,
        # or finished and not yet removed from the batch.
        if not self.scheduler.has_requests():
            return {}, False
        scheduler_output = self.scheduler.schedule(self._should_throttle_prefills())
        future = self.model_executor.execute_model(scheduler_output, non_block=True)
        grammar_output = self.scheduler.get_grammar_bitmask(scheduler_output)
        with (
            self.capture_iteration_details(scheduler_output) as iteration_details,
            self.log_error_detail(scheduler_output),
        ):
            model_output = future.result()

Three things to notice, all of them V1's inheritance: execute_model(..., non_block=True) returns a future, so grammar bitmasks are computed while the GPU runs; there is no tokenization or detokenization anywhere in this function; and the whole thing is driven by a loop that only ever polls a queue.

    def run_busy_loop(self):
        """Core busy loop of the EngineCore."""
        while self._handle_shutdown():
            # 1) Poll the input queue until there is work to do.
            self._process_input_queue()
            # Publish request counts before and after GPU step to ensure freshness.
            self._maybe_publish_request_counts()
            # 2) Step the engine core and return the outputs.
            self._process_engine_step()
            self._maybe_publish_request_counts()

        raise SystemExit

That queue is fed by a ZMQ DEALER socket on a separate thread (vllm/v1/engine/core.py:L1204). The resulting topology is §5.5's subject; docs/design/arch_overview.md:L105-L113 gives the arithmetic — for A API servers, DP ranks and N GPUs, A + DP + N processes plus a DP coordinator when DP > 1.

The second rewrite is not history. It landed as PR #25266, "GPU Model Runner V2", on 2025-11-21, and by this SHA vllm/v1/worker/gpu/ has absorbed 409 commits. Its design document does not soften the assessment:

Since vLLM V1 was first implemented, we discovered several fundamental design mistakes and accumulated significant technical debt. Many features were bolted on that were not considered in the original design. We also gained valuable insights into sampling techniques (for example, Gumbel-max sampling), tools (for example, Triton), and CUDA features (for example, UVA). With this knowledge, we implemented Model Runner V2 (MRV2) from first principles to be cleaner, more efficient, and more modular.

In hindsight, many of V1's design choices were suboptimal. While MRV2 is not yet feature-complete, not rigorously tested, and still has open design decisions, we believe it is a substantial improvement over V1.

Why 2,024 lines can replace 8,008

It does not, in aggregate. The whole gpu/ tree is 19,467 lines across 60+ files — more code than the monolith. What changed is that no single file is the runner any more. The design doc calls this out directly: "Compared to V1's large, entangled gpu_model_runner.py, MRV2 splits feature logic across dedicated files" (docs/design/model_runner_v2.md:L167).

The measurable difference: the legacy runner defines 125 top-level methods on one class and inherits from three mixins; V2's defines 48 and inherits from one.

# ... legacy:
class GPUModelRunner(
    LoRAModelRunnerMixin, KVConnectorModelRunnerMixin, ECConnectorModelRunnerMixin
):
    def __init__(

# ... V2:
class GPUModelRunner(LoRAModelRunnerMixin):
    def __init__(self, vllm_config: VllmConfig, device: torch.device):

The two connector mixins became modules — gpu/kv_connector.py and gpu/ec_connector.py — imported explicitly. V2's runner has 30 from vllm.v1.worker.gpu.… import statements; the legacy runner has 13 from vllm.v1.worker in total. Composition replaced inheritance.

What exists only in V2

The cleanest proof that V2 is a fork rather than a refactor: features that have no legacy implementation at all. Block verification for speculative decoding is one. The config accepts three methods —

    rejection_sample_method: RejectionSampleMethod = "standard"
    """The rejection sampling method to use. 'standard' uses probabilistic
    rejection sampling (with or without cached draft logits, controlled by
    draft_sample_method). 'synthetic' accepts draft tokens with a decaying
    probability calibrated to synthetic_acceptance_rate. 'block' uses block
    verification (Sun et al.), which jointly verifies the draft tokens as a
    block instead of one at a time."""

— but grep -rn '"block"' vllm/v1/ returns exactly one hit, in the V2 tree:

        rejection_sample_method = spec_config.rejection_sample_method
        self.use_block_verification: bool = False
        self.synthetic_conditional_rates: torch.Tensor | None = None
        if rejection_sample_method == "synthetic":
            assert spec_config.synthetic_acceptance_rates is not None
            self.synthetic_conditional_rates = torch.tensor(
                unconditional_to_conditional_rates(
                    spec_config.synthetic_acceptance_rates
                ),
                dtype=torch.float32,
                device=device,
            )
        elif rejection_sample_method == "block":
            self.use_block_verification = True

The shown legacy sampler has the synthetic branch but no block-verification branch. This establishes a local implementation difference, not that every configuration carrying rejection_sample_method="block" starts and silently ignores it: full config validation, runner eligibility and later dispatch must also be checked at the pinned revision. Treat the DSpark, DFlash2, context-parallel and diffusion examples as feature-gate investigations, not interchangeable silent-no-op guarantees.

The state representation is where the design difference bites hardest. Legacy V1 keeps a Python dataclass per request as a backup copy — CachedRequestState in gpu_input_batch.py:L34-L54, with block_ids, output_token_ids and friends as Python lists. V2 has no such class (grep -rn CachedRequestState vllm/v1/worker/gpu/ is empty). Instead:

        # NOTE(woosuk): This tensor can be extremely large (e.g., several GBs)
        # depending on the configured max_num_reqs and max_model_len.
        # To save GPU memory, we use UVA instead of GPU for this tensor.
        self.all_token_ids = StagedWriteTensor(
            (self.max_num_reqs, self.max_model_len),
            dtype=torch.int32,
            device=device,
            uva_instead_of_gpu=True,
        )

Every request gets a permanent row for its lifetime; updates are staged as diffs and applied with one kernel; preemption is treated as completion. That is the "decouple persistent state from per-step inputs" argument from docs/design/model_runner_v2.md:L33-L39, made concrete. §11.4 takes both runners apart properly; here the point is only that they do not share state machinery.

The rewrite is not confined to the worker

This is the part people miss. use_v2_model_runner appears at 28 sites across 15 files outside vllm/config/ — grep -rn use_v2_model_runner vllm/ | grep -v vllm/config/ — four of them in the scheduler:

        # Construct the scheduler output.
        if self.use_v2_model_runner:
            scheduled_new_reqs.extend(scheduled_resumed_reqs)
            scheduled_resumed_reqs.clear()
            new_reqs_data = [
                NewRequestData.from_request(
                    req,
                    req_to_new_blocks[req.request_id].get_block_ids(),
                    req._all_token_ids,
                    uses_mrope=self.model_uses_mrope,
                )
                for req in scheduled_new_reqs
            ]
# ...
        # Record the request ids that were scheduled in this step (MRV1-only).
        if not self.use_v2_model_runner:
            self.prev_step_scheduled_req_ids.clear()
            self.prev_step_scheduled_req_ids.update(num_scheduled_tokens.keys())

Under V2 the scheduler stops distinguishing resumed requests from new ones — "treat preemption as completion" reaches back across the process boundary — and it stops tracking the previous step's scheduled IDs entirely. async_scheduler.py:L46-L49 adds a V2-only next_decode_eligible_step for pipeline microbatching, and even v1/attention/backends/flashinfer.py:L940 disables pinned memory under V2. The comment marker MRV1-only is worth grepping for.

Follow the selection from environment variable to constructor. Five hops.

Hop 1 — the flag is declared as tri-state. vllm/envs.py:L297 types it bool | None, defaulting to None, and the parser at vllm/envs.py:L2041-L2044 keeps the unset case distinguishable from 0:

    # Flag to control the v2 model runner. If unset, use config defaults.
    "VLLM_USE_V2_MODEL_RUNNER": lambda: maybe_convert_bool(
        os.getenv("VLLM_USE_V2_MODEL_RUNNER", None)
    ),

Hop 2 — VllmConfig.use_v2_model_runner resolves it. vllm/config/vllm.py:L648-L652: if the env var is set, that wins, full stop. Otherwise a policy runs.

Hop 3 — forced-on cases. vllm/config/vllm.py:L654-L680 returns True unconditionally for prefill-context parallelism, DSpark speculation, multi-KV-group DFlash drafts, DFlash2 drafts, and diffusion models — each with a comment explaining that the feature exists only in V2.

Hop 4 — the default policy. Two predicates. First, an allow-list of ten architectures:

DEFAULT_V2_MODEL_RUNNER_ARCHITECTURES = frozenset(
    {
        "DeepseekV2ForCausalLM",
        "DeepseekV32ForCausalLM",
        "DeepseekV4ForCausalLM",
        "GlmMoeDsaForCausalLM",
        "GraniteMoeForCausalLM",
        "InklingForCausalLM",
        "InklingForConditionalGeneration",
        "KimiK3ForConditionalGeneration",
        "LongcatFlashNgramForCausalLM",
        "Qwen2MoeForCausalLM",
    }
)

Then the predicate that actually decides most deployments — and the line to read carefully is the last one:

    def _is_default_v2_model_runner_model(self) -> bool:
        model_config = self.model_config
        if model_config is None:
            return False

        architectures = getattr(model_config, "architectures", [])
        default_architectures = default_v2_model_runner_architectures()
        is_default_v2_architecture = any(
            arch in default_architectures for arch in architectures
        )

        if getattr(model_config, "is_hybrid", False) and (
            not is_default_v2_architecture
        ):
            return False

        if getattr(model_config, "is_attention_free", False):
            return False
        return is_default_v2_architecture or not model_config.is_moe

or not model_config.is_moe. Eligible dense models default to the V2 runner after the preceding platform, hybrid, attention-free and feature guards. Llama-3-8B is LlamaForCausalLM — not on the allow-list, not hybrid, not attention-free, not MoE — so it takes V2. The 8,008-line file you opened first is, for the book's reference model, dead code.

Two escape hatches follow: if Triton is unavailable, vllm/config/vllm.py:L685-L689 logs "Model Runner V2 requires Triton; using the V1 model runner instead." and falls back; and _get_v2_model_runner_unsupported_features() (vllm/config/vllm.py:L2438-L2543) collects gaps — n-gram speculative decoding, sequence parallelism, stock torch.compile, PP under external_launcher — logging "Model Runner V2 does not yet support %s" before falling back.

Hop 5 — the worker constructs one. This is the only place either class is instantiated for CUDA:

        # Construct the model runner
        if self.use_v2_model_runner:
            from vllm.v1.worker.gpu.model_runner import (
                GPUModelRunner as GPUModelRunnerV2,
            )

            # HACK(woosuk): This is a temporary fix to avoid type errors.
            self.model_runner: GPUModelRunner = GPUModelRunnerV2(  # type: ignore
                self.vllm_config, self.device
            )
        else:
            from vllm.v1.worker.gpu_model_runner import (
                GPUModelRunner as GPUModelRunnerV1,
            )

            self.model_runner = GPUModelRunnerV1(self.vllm_config, self.device)

Both classes are named GPUModelRunner. Both are aliased at the import site. If you set a pdb breakpoint on GPUModelRunner.execute_model by name, you have a 50% chance of breaking in the wrong file. The same pattern repeats in vllm/v1/worker/xpu_worker.py:L130 and vllm/v1/worker/cpu_worker.py:L174 — V2 is not CUDA-only.

• vllm/entrypoints/openai/api_server.py is a 59-line shim. It re-exports from entrypoints/launchers/ and warns that it "is deprecated and will likely be unsupported in a future version" (vllm/entrypoints/openai/api_server.py:L24-L30). Routes live in entrypoints/launchers/api_server/routers.py; chat handling moved to entrypoints/openai/chat_completion/serving.py.
• Tokenizer imports moved. transformers_utils/ → tokenizers/ as of 2025-11-29.
• csrc/moe/ has one file. The MoE kernels live under csrc/libtorch_stable/moe/.
• Root-level benchmarks/*.py are shims. The implementations are in vllm/benchmarks/ (25 files, ~13,900 lines) and are reached via vllm bench.
• A v1 in a path never means "old". Besides the engine, vllm/distributed/kv_transfer/kv_connector/v1/ is the current connector API.

Confirm the instantiated runner. A print in the legacy file may not fire because the eligible reference configuration chose V2. Silence alone is not evidence: some ineligible paths return before fallback logging. Inspect the resolved flag and type(worker.model_runner).__module__ at the construction site; record the model, features and revision.

Second diagnostic: parsing a speculative flag proves neither successful engine construction nor execution of its feature. Trace validation, eligibility and the selected sampler separately. The local missing branch is a verification lead; without executing the full pinned path, do not promise startup, silence, or a particular acceptance-rate difference.

Answer "which runner am I on?" without a GPU, straight from config:

python -c "
from vllm.engine.arg_utils import EngineArgs
cfg = EngineArgs(model='meta-llama/Meta-Llama-3-8B-Instruct').create_engine_config()
print('architectures :', cfg.model_config.architectures)
print('is_moe        :', cfg.model_config.is_moe)
print('use_v2_runner :', cfg.use_v2_model_runner)
"

Flip it with VLLM_USE_V2_MODEL_RUNNER=0 and watch the decision change. This reads vllm/config/vllm.py:L648-L700 and needs no device. Three more one-liners reproduce every count in this chapter:

# the shape of the tree
find vllm -name '*.py' | wc -l                       # 2270
find vllm/v1 -name '*.py' | wc -l                    # 357
find vllm/v1/worker/gpu -name '*.py' -exec wc -l {} + | tail -1   # 19467 total

# which files are actually moving, last 180 days
git log --since='2026-02-22' --name-only --pretty=format: -- vllm/ \
  | grep -v '^$' | sort | uniq -c | sort -rn | head -10

# how fast the second rewrite is going
git log --oneline -- vllm/v1/worker/gpu/ | wc -l     # 409

Both runners in the top four is the whole chapter in one row: this is not a migration that has already happened, it is one being carried out under load.

Where to start reading, by task

• In-tree, read these first. docs/design/model_runner_v2.md (206 lines — the clearest architectural writing in the repo, and the source for §5 above); docs/design/arch_overview.md (process counts and the TP=4 / TP=2·DP=4 topology figures); docs/usage/v1_guide.md (the V0→V1 behavioural diff, including logprobs semantics and prefix-caching interactions); docs/design/vllm_ir.md (why vllm/ir/ exists); vllm/v1/worker/gpu/README.md (four lines, but the ones that matter).
• PRs that define the timeline. #9289 — "[V1] Implement vLLM V1 [1/N]", 2024-10-22, creates vllm/v1/. #25033 — "[V0 Deprecation] Remove LLMEngine", 2025-09-20. #25351 — "Remove V0 attention backends", 2025-09-21. #25266 — "GPU Model Runner V2", 2025-11-21, creates vllm/v1/worker/gpu/. #31916 — "[1/N][Attention] Restructure attention: move files", 2026-01-09. #47361 — "Delete PagedAttention", 2026-07-02.
• RFC #18571 — the V0 deprecation RFC, linked from docs/usage/v1_guide.md:L5.
• vLLM V1: A Major Upgrade to vLLM's Core Architecture (2025-01-27) — the source of the 1.7× ShareGPT throughput claim and of the EngineCore / ZMQ framing quoted above.
• Next in this part. §11.2 walks the AsyncLLM → EngineCore → Executor → Worker chain with the actual ZMQ hops; §11.3 reads schedule() line by line; §11.4 takes both model runners apart; §11.5 covers what you must implement twice while both runners live.