NLP
NLP changed more between 2017 and 2026 than any other area of machine learning, and the change was not incremental. Task-specific architectures gave way to pretrained transformers; pretrained transformers gave way to general models prompted in natural language. What survived is the underlying set of problems — tokenization, representation, ambiguity, evaluation — which is why these pages are organised by problem rather than by model.
Topics
| Topic | Level | What it covers |
|---|---|---|
| Text Preprocessing | beginner | Unicode, the tokenization problem, BPE/WordPiece/SentencePiece with a worked example, what modern pipelines should not do |
| Text Representation | intermediate | TF-IDF and BM25, word2vec and GloVe derived, fastText, contextual and sentence embeddings, vector search |
| Language Models | intermediate | n-grams and smoothing, neural LMs, perplexity done properly, scaling laws, emergence, hallucination |
| Text Classification | beginner | the approach ladder from TF-IDF up, fine-tuning, LLM classification, imbalance, error analysis |
| Sequence Labeling | intermediate | BIO schemes, HMMs and CRFs with Viterbi, transformer taggers, subword alignment, nested entities |
| Machine Translation | advanced | SMT to seq2seq to transformer, back-translation, multilingual models, BLEU and COMET |
| Text Generation & Decoding | intermediate | greedy/beam/top-p/min-p, repetition control, constrained decoding, speculative decoding |
| Pretrained Model Families | intermediate | BERT, GPT, T5 and their descendants; the modern decoder recipe; choosing a checkpoint |
| LLM Prompting & Alignment | advanced | SFT, RLHF, DPO, GRPO; prompting that works; chain of thought; tool use; failure modes |
| RAG & Retrieval | advanced | chunking, hybrid search, reranking, context assembly, verification, stage-wise evaluation |
| Speech & Audio | advanced | spectrograms, CTC and RNN-T, Whisper, TTS, speaker tasks, audio LLMs, WER |
| NLP Evaluation | intermediate | metrics per task, BLEU/ROUGE/BERTScore/COMET, LLM judges and their biases, contamination |
How they fit together
Ctrl/Cmd + wheel to zoom · drag to pan · double-click to fit · ⛶ full size
Suggested order
- Text Preprocessing — tokenization decisions leak into everything.
- Text Representation — and note that TF-IDF is still a baseline worth beating.
- Language Models — the object everything else is built on.
- Text Classification — the most deployed task, and the one where the cheap baseline is most often skipped.
- Pretrained Model Families — how to choose a checkpoint.
- Text Generation & Decoding, then LLM Prompting & Alignment.
- RAG & Retrieval — the standard production architecture.
- NLP Evaluation — revisit in depth here; read its split/metric protocol before tuning any earlier model or prompt.
- Sequence Labeling, Machine Translation, and Speech & Audio as needed.
Related courses
- Transformers Deep Dive — 17 modules deriving the architecture from first principles.
- The Inference Engineering Course — 14 chapters on serving these models efficiently.
- LLM Fine-Tuning Deep Dive — 21 modules on adapting these models after pretraining.
- PyTorch from Tensors to LLMs — 38 modules on the framework underneath, ending with a GPT built and trained from scratch.
The short version
- Tokenization is not a detail. It decides cost, context, arithmetic ability, and cross-language equity.
- Run the cheap baseline. TF-IDF plus a linear model, and BM25 for retrieval, are still hard to beat and take minutes.
- Fine-tuning and retrieval solve overlapping problems. Weights can learn facts; retrieval often makes changing information and attribution easier.
- Likelihood is not a truth guarantee. Next-token training alone neither guarantees factuality nor proves a theorem that every possible model must hallucinate.
- Evaluate relevant data with enough evidence. Private task fixtures complement public benchmarks; fifty examples may be useful for debugging but underpowered for small gains or rare subgroup failures.
Shared task contract
Before choosing a checkpoint, record input unit, output ontology, annotation ambiguities, source/license permissions, privacy handling, and document/user/time split boundaries. Entry skills are array shapes, conditional probability, and train/development/test isolation. Exit skills include explaining a tokenizer's offsets, fitting a leakage-safe baseline, distinguishing score from decision metrics, and tracing a generated claim to authorized evidence or abstaining.
A small capstone can reuse a versioned, redistributable support-document corpus for topic classification, entity extraction, retrieval and grounded answers. Maintain one annotation guide and provenance manifest, plus unanswerable queries, near duplicates, minority-language slices, and malformed input cases. Freeze the evaluation protocol before repeated prompt/model selection. Record quality, uncertainty, latency and cost together; a polished output on one example is not an evaluation dashboard.
Start the executable portion with the text-classifier artifact capstone. It trains and evaluates a sparse baseline, saves its data/split/schema manifest, reloads the full pipeline, and checks served probabilities against a fixed prediction fixture. The corpus is intentionally synthetic and offline. Extraction, retrieval, grounded generation, and real multilingual benchmark slices are not claimed by that narrower acceptance test.