Sources

cd ~/Documents/other_git_repos

git clone https://github.com/vllm-project/vllm.git
git -C vllm checkout a556f3fccb701e5618d84d547ff454c56a1bfdfb
git -C vllm rev-parse HEAD   # must print a556f3fccb701e5618d84d547ff454c56a1bfdfb

git clone https://github.com/sgl-project/sglang.git
git -C sglang checkout 7d893255c359bb8ab74d2870c8ac865fb57230d6
git -C sglang rev-parse HEAD # must print 7d893255c359bb8ab74d2870c8ac865fb57230d6

Clone location assumed by every citation and lab in this book: ~/Documents/other_git_repos/{vllm,sglang}. Paths inside citations are relative to each repository root — vllm/v1/core/sched/scheduler.py resolves inside ~/Documents/other_git_repos/vllm/, and python/sglang/srt/managers/scheduler.py resolves inside ~/Documents/other_git_repos/sglang/.

This table records the original import's incomplete external-dependency provenance. Later lessons inspect engine wrappers and some vendored kernels, but that does not pin or verify every installed external library. No complete executable dependency lock or GPU-tested engine environment has been established in this checkout.

Record FlashInfer, FlashAttention, XGrammar, CUTLASS and transport library versions or commits, compiler/build options, GPU driver, model/tokenizer revision and retrieval date for every reproduced run. Keep wrapper inspection separate from external kernel inspection. The primary engine SHAs above remain the edition pins; do not replace them silently with current releases.

Papers, blogs, and talks the book draws on. This list grows as chapters are drafted; every entry should be traced to its owning lesson; chapter-level further-reading links remain the detailed bibliography. This imported list is not an automatically complete citation inventory. New references must include a stable primary URL and an owning lesson. A source path existing does not prove that a quotation or behavioral claim is correct.

Systems papers

• Efficient Memory Management for Large Language Model Serving with PagedAttention — Kwon et al., SOSP 2023. The vLLM paper. arXiv:2309.06180. Used in Part 2.
• SGLang: Efficient Execution of Structured Language Model Programs — Zheng et al., NeurIPS 2024. RadixAttention and the frontend language. arXiv:2312.07104. Used in Parts 2, 6, 12.
• Orca: A Distributed Serving System for Transformer-Based Generative Models — Yu et al., OSDI 2022. Iteration-level (continuous) batching. USENIX. Used in Part 1.

Kernels and numerics

• FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness — Dao et al., NeurIPS 2022. arXiv:2205.14135.
• FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning — Dao, 2023. arXiv:2307.08691.
• FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision — Shah et al., 2024. arXiv:2407.08608.
• Online normalizer calculation for softmax — Milakov & Gimelshein, 2018. arXiv:1805.02867. The recurrence Part 3 derives.
• Roofline: An Insightful Visual Performance Model — Williams et al., CACM 2009. Used in chapter 00-04.

Decoding

• Fast Inference from Transformers via Speculative Decoding — Leviathan et al., ICML 2023. arXiv:2211.17192. The correctness proof reproduced in chapter 06-02.
• Accelerating Large Language Model Decoding with Speculative Sampling — Chen et al., 2023. arXiv:2302.01318.
• EAGLE / EAGLE-2 / EAGLE-3 — Li et al., 2024–2025. Feature-level drafting and dynamic draft trees. Used in chapter 06-04.
• Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads — Cai et al., 2024. arXiv:2401.10774.

Attention variants and architectures

• Fast Transformer Decoding: One Write-Head is All You Need — Shazeer, 2019. MQA. arXiv:1911.02150.
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints — Ainslie et al., 2023. arXiv:2305.13245.
• DeepSeek-V2 / V3 technical reports — Multi-head Latent Attention and the MoE design analysed in chapters 03-05 and 07-02.
• RoFormer: Enhanced Transformer with Rotary Position Embedding — Su et al., 2021. arXiv:2104.09864.
• YaRN: Efficient Context Window Extension of Large Language Models — Peng et al., 2023. arXiv:2309.00071.
• Efficient Streaming Language Models with Attention Sinks — Xiao et al., 2023. arXiv:2309.17453.

Quantization

• GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers — Frantar et al., 2022. arXiv:2210.17323.
• AWQ: Activation-aware Weight Quantization — Lin et al., 2023. arXiv:2306.00978.
• SmoothQuant — Xiao et al., 2022. arXiv:2211.10438.

Every number in this book is one of three kinds, and is always labelled:

There is no fourth kind. If a table in this book has a number in it, it is one of these three, and the caption says which.

The maintained checker is site/check_inference_citations.py. It inventories explicit path-and-line references in code and source headers. Without external checkouts it reports them as unchecked, not verified. With clean checkouts at the stated SHAs it checks path existence and line bounds and records the evidence used to resolve shortened references. Ambiguous or unsupported qualifications remain unresolved. It does not compare complete quotation text, certify interpretations, or claim a complete bibliography.

Measured audit, September 2026

Both exact upstream revisions were checked out cleanly for this audit. All 3,528 explicit references passed file-existence and line-bound checks, with zero unresolved path attributions. Of these, 2,941 name an engine through their full path or source-block label; the other 587 have recorded resolution evidence. Counts are occurrences, not unique files or unique quotations. The strict pinned-checkout command now exits successfully.

Resolution uses three distinct evidence levels. 129 written multi-component paths exist exactly in only one of the two pinned trees; this is exact path matching, not a basename search. 96 shortened references match a unique explicitly qualified citation in the same section or lesson whose interval contains the written range. 362 occurrences were qualified from their surrounding prose, table or call trace after inspecting the actual pinned source. Their HTML retains the readable abbreviation and records the full path, engine and a source symbol or literal that the checker revalidates at the cited lines. References inside mixed-engine code listings instead use full paths. A symbol check supports file identification, not every assertion in the adjacent paragraph.

The source review corrected several concrete mismatches. Two FlashInfer error references now include their actual exception branches; the automatic backend-selection reference includes the assignment of an import failure to the rejection reasons. The SGLang discovery-poll default and extension-API-version quotation now point to their actual lines rather than nearby settings or a heading. The P/D discussion distinguishes DFlash's make_draft_input_v2 from DSpark's make_next_draft_input, which receives the same two arguments. The kernel chapter now distinguishes native instruction layouts from block-scaled recipes and documents the linear-backend selection filter. These findings demonstrate why passing line bounds alone must not be described as complete quotation verification.

Download the pinned range audit (JSON). It records every written reference, resolved full path and pinned URL where available, qualification evidence, exact commits, UTC check time, counts by page and unresolved path, and a fingerprint of the inventory and authored source metadata. The build regression detects changed citation inputs so this snapshot cannot silently describe a different inventory. It does not clone the repositories or repeat the upstream check in CI.

The abbreviated-path backlog is closed for this inventory, including duplicate names such as base.py, config.py, __init__.py and hardware-specific backends. The JSON's unresolved_by_path is empty. This does not close the broader evidence backlog: full quotation and claim review, separately pinned external dependencies, build-configuration gates, and real-engine correctness or performance experiments still require their own evidence. The checker inventories only explicit file-and-line references, not every bare line number, source mention, bibliography entry or behavioral assertion.

python site/check_inference_citations.py
python site/check_inference_citations.py --vllm-root /path/to/vllm --sglang-root /path/to/sglang --summary --output citation-summary.json
python site/check_inference_citations.py --vllm-root /path/to/vllm --sglang-root /path/to/sglang --strict

The last command passes with the two clean pinned checkouts and the current inventory. Without those checkouts it still reports unchecked references and exits nonzero under --strict. Omit --summary to obtain every individual record and pinned source URL. A future nonzero result flags missing evidence or an invalid reference, not an engine runtime failure: this checker never starts either engine.

How to finish a source review

1. For new or changed references, establish the owning repository and full path using the surrounding lesson, imports and call graph. Do not auto-select the first basename match. Rerun the pinned audit whenever citation text or authored qualification metadata changes.
2. At the pinned commit, compare the complete displayed quotation with the actual function, including the surrounding branch, configuration gate and caller. A valid line range and a matching symbol do not establish every neighboring claim.
3. Record whether the claim concerns an engine wrapper, a vendored implementation or an external dependency. Pin external dependencies separately before claiming their behavior was inspected.
4. For a runtime claim, record the model revision, configuration, device, dependency versions, command and observed output. Source inspection alone cannot establish latency, memory safety under concurrency or kernel compatibility.

Use the generated pinned URLs for manual review. A passing range check is necessary but insufficient evidence of factual correctness.