Visualization: Seeing What the Numbers Hide

Anscombe's quartet is four datasets with identical means, variances, correlations, and regression lines that look completely different when plotted. That is not a curiosity — it is the argument for this entire page. Summary statistics compress, and compression loses exactly the structure you most need to see.

The library landscape

The practical policy: Seaborn for statistical exploration, Matplotlib when you need to control something Seaborn will not expose, Plotly when someone will interact with the figure.

Matplotlib: learn the object model, not the plt shortcuts

Matplotlib has two interfaces. The stateful plt.plot() API mutates a hidden "current figure", which is fine in a scratch notebook and terrible in a function that must not depend on global state. The object-oriented API is explicit:

import matplotlib.pyplot as plt

fig, axes = plt.subplots(2, 3, figsize=(15, 8), sharex=True,
                         constrained_layout=True)
ax = axes[0, 1]
ax.plot(x, y, lw=2, label="train")
ax.set(xlabel="epoch", ylabel="loss", title="Training loss", yscale="log")
ax.legend(frameon=False)
fig.savefig("loss.png", dpi=200, bbox_inches="tight")

The vocabulary that unlocks the documentation:

plt.rcParams.update({
    "figure.dpi": 120, "savefig.dpi": 200,
    "font.size": 11, "axes.titlesize": 13,
    "axes.spines.top": False, "axes.spines.right": False,
    "axes.grid": True, "grid.alpha": 0.3,
    "figure.constrained_layout.use": True,
})

Setting rcParams once at the top of a notebook does more for figure quality than styling each plot individually. constrained_layout solves the overlapping labels problem that tight_layout handles less reliably.

Close your figures in loops. plt.close(fig) — Matplotlib keeps every figure alive until closed, and a training loop that plots each epoch will exhaust memory.

Seaborn: statistical plots as one-liners

Seaborn's figure-level functions take a tidy (long-format) dataframe and produce a whole grid.

import seaborn as sns
sns.set_theme(style="whitegrid", context="notebook", palette="colorblind")

sns.relplot(data=df, x="epoch", y="loss", hue="model", col="dataset",
            kind="line", errorbar=("ci", 95), height=4)

sns.displot(data=df, x="score", hue="label", kind="kde",
            common_norm=False, fill=True)

sns.catplot(data=df, x="model", y="auc", kind="box", col="split")

sns.pairplot(df[num_cols + ["target"]], hue="target", corner=True, diag_kind="kde")

sns.heatmap(df[num_cols].corr(), annot=True, fmt=".2f",
            cmap="RdBu_r", center=0, vmin=-1, vmax=1, square=True)

Mixing them up is the usual Seaborn frustration: figure-level functions cannot be drawn into an existing subplot. If you are composing a multi-panel figure by hand, use the axes-level functions with ax=.

common_norm=False on displot/kdeplot with hue is worth remembering: by default Seaborn normalises across all hue groups together, so a rare class appears as a flat line. Setting it to False normalises each group separately, which is almost always what you meant.

Choosing the right plot

The ECDF is under-used. Unlike a histogram it has no bin-width parameter, unlike a KDE it invents nothing, and it makes quantiles readable directly: sns.ecdfplot(data=df, x="latency", hue="version") answers "what fraction is under 200 ms?" at a glance.

Bar charts of means with error bars are the worst common choice: they hide sample size, distribution shape, and outliers. Prefer a box or violin with the raw points overlaid (sns.stripplot on top of sns.boxplot) whenever \(n\) is small enough.

The plots that matter in ML

Learning curves — is it bias or variance?

from sklearn.model_selection import LearningCurveDisplay
LearningCurveDisplay.from_estimator(model, X, y, cv=5, n_jobs=-1,
                                    score_type="both", std_display_style="fill_between")

LearningCurveDisplay varies training sample count, not epoch. An improving validation score at larger sample counts is evidence for testing more data, not a guaranteed return on collection cost.

Training curves — is it converging?

Plot loss against an explicitly stated resource: updates, examples/tokens, epochs, or elapsed time. Equal steps do not imply equal examples or compute with different batch sizes. Log axes require positive values; use a linear or appropriate signed scale otherwise. Increasing validation loss during continued fitting may indicate overfitting; increasing validation accuracy instead indicates improvement.

Also plot: learning rate (confirms the schedule fired), gradient norm (spikes precede divergence), and per-layer parameter-update ratio \(\|\Delta w\|/\|w\|\), interpreted by layer, optimizer, parameterization, and training phase rather than a universal \(10^{-3}\) threshold.

Confusion matrix — normalised, always

from sklearn.metrics import ConfusionMatrixDisplay
ConfusionMatrixDisplay.from_estimator(model, X_test, y_test,
                                      normalize="true", values_format=".2f",
                                      cmap="Blues", xticks_rotation=45)

normalize="true" gives per-class recall on the diagonal; without it, a dominant class visually swamps everything and a rare class's total failure is invisible. For many classes, sort the classes by frequency and look for off-diagonal blocks — they reveal systematically confusable groups, which is a possible taxonomy, representation, model, or annotation problems. Inspect actual errors before choosing a cause; show raw counts alongside normalized rates.

ROC and precision–recall

fig, (a1, a2) = plt.subplots(1, 2, figsize=(11, 4.5))
RocCurveDisplay.from_estimator(model, X_test, y_test, ax=a1)
a1.plot([0, 1], [0, 1], "k--", lw=1, label="chance")
PrecisionRecallDisplay.from_estimator(model, X_test, y_test, ax=a2)
a2.axhline(y_test.mean(), ls="--", c="k", lw=1, label="base rate")

Always draw the baselines. The ROC diagonal and the PR base-rate line are what turn "0.85" into "0.85 against a chance level of 0.5" or "0.30 against a base rate of 0.02". On imbalanced problems the PR curve is the informative one.

Calibration

from sklearn.calibration import CalibrationDisplay
CalibrationDisplay.from_estimator(model, X_test, y_test, n_bins=15, strategy="quantile")

Plot predicted probability against observed frequency. A perfectly calibrated model sits on the diagonal in population. Below means overprediction of the positive-class probability; above means underprediction. This is not universally predicted-class overconfidence: predicted positive probability 0.1 with observed frequency 0.05 underestimates confidence in the predicted negative class. Use strategy="quantile" to target comparable bin counts — uniform-width bins on a skewed score distribution produce meaningless endpoints with two samples in them.

Add a histogram of predicted probabilities underneath: a model whose probabilities cluster near the base rate need not be calibrated. A truly constant prevalence predictor can be calibrated but nondiscriminative, which is why both calibration and ranking matter.

Residual analysis for regression

fig, axes = plt.subplots(1, 3, figsize=(15, 4))
axes[0].scatter(y_pred, y_true - y_pred, s=8, alpha=0.3); axes[0].axhline(0, c="k")
axes[0].set(xlabel="predicted", ylabel="residual", title="Residuals vs fitted")
stats.probplot(y_true - y_pred, plot=axes[1])          # Q-Q plot
axes[2].scatter(y_true, y_pred, s=8, alpha=0.3)
axes[2].plot([y_true.min(), y_true.max()], [y_true.min(), y_true.max()], "k--")
axes[2].set(xlabel="actual", ylabel="predicted", title="Predicted vs actual")

Treat curvature, funnels, heavy tails and prediction compression as diagnostic clues, not unique diagnoses. Check subgroup mixtures, label errors, target transformations, model misspecification and expected regression-to-the-mean before changing the loss or regularizer.

Embedding visualisation

import umap
emb = umap.UMAP(n_neighbors=30, min_dist=0.1, metric="cosine",
                random_state=0).fit_transform(features)
sns.scatterplot(x=emb[:, 0], y=emb[:, 1], hue=labels, s=6, alpha=0.6,
                palette="tab20", linewidth=0)

Read these plots with real caution. In both t-SNE and UMAP:

• Cluster sizes and densities can be substantially distorted.
• Distances between clusters are largely meaningless.
• Apparent clusters can appear in pure noise, especially with small perplexity or n_neighbors.
• The result changes with the random seed and with every hyperparameter.

They are useful for generating hypotheses — "these two classes overlap entirely, maybe the labels are ambiguous" — and for spotting duplicates or mislabelled points. They are not evidence. PCA, being linear, is less pretty and more trustworthy: its axes have meaning and the explained-variance ratio is interpretable.

Drift monitoring

from scipy import stats
fig, axes = plt.subplots(1, 3, figsize=(15, 4))
for ax, col in zip(axes, watch_cols):
    reference = train[col].dropna()
    current = live[col].dropna()
    if min(len(reference), len(current)) < 2:
        ax.set_title(f"{col}: insufficient nonmissing observations")
        continue
    sns.ecdfplot(reference, ax=ax, label="train")
    sns.ecdfplot(current, ax=ax, label="live")
    ks = stats.ks_2samp(reference, current)
    ax.set_title(f"{col}  KS={ks.statistic:.3f}  p={ks.pvalue:.1e}")

Overlay the training and production distributions per feature, and pair the plot with a quantitative test (KS for continuous, chi-square or population stability index for categorical). A PSI above 0.2 is the usual "investigate" threshold.

Large data

Scatter plots stop working around 10⁵ points — everything overplots into a solid blob, and the SVG file becomes unusable.

ax.hexbin(x, y, gridsize=80, bins="log", cmap="viridis")   # density, not points
ax.scatter(x, y, s=1, alpha=0.05, rasterized=True)         # keeps vector text, raster points

rasterized=True is the trick for publication figures: the marker layer becomes a bitmap inside the PDF while axes and text stay vector.

Colour, and getting it right

Rules that are not stylistic preferences:

• Never use jet/rainbow for continuous data. It has false luminance boundaries that create visual features where the data has none, and it is unreadable in greyscale and to colourblind viewers.
• Diverging maps need an explicit centre. center=0, vmin=-1, vmax=1 on a correlation heatmap; otherwise the colour scale drifts with the data and two heatmaps are not comparable.
• ~8% of men have red–green colour deficiency. Use colorblind palettes, and encode with shape or line style in addition to colour.
• Do not encode a variable twice (colour and size for the same quantity) unless it is deliberate redundancy for accessibility.

Interactive plots

import plotly.express as px

fig = px.scatter(df, x="pc1", y="pc2", color="label",
                 hover_data=["id", "text", "confidence"],
                 opacity=0.7, width=900, height=650)
fig.update_traces(marker=dict(size=5))
fig.write_html("embeddings.html")           # self-contained, shareable

Interactivity earns its place when hovering reveals the identity of a point — which is exactly the case for embedding plots, error analysis, and anything where "what is that outlier?" is the question. For a static figure in a report, it adds weight and no information.

Anti-patterns

A reusable diagnostic panel

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import (RocCurveDisplay, PrecisionRecallDisplay,
    ConfusionMatrixDisplay, f1_score, precision_score, recall_score)
from sklearn.calibration import CalibrationDisplay

def evaluate(model, X, y, name="", positive_label=1, threshold=0.5):
    classes = np.asarray(model.classes_)
    if len(classes) != 2 or positive_label not in classes:
        raise ValueError("This panel requires two classes and an explicit positive label")
    y = (np.asarray(y) == positive_label).astype(int)
    if len(np.unique(y)) != 2:
        raise ValueError("ROC AUC requires both labels in the evaluated cohort")
    p = model.predict_proba(X)[:, np.flatnonzero(classes == positive_label).item()]
    if not np.isfinite(p).all():
        raise ValueError("nonfinite probabilities")
    fig, ax = plt.subplots(2, 3, figsize=(16, 9), constrained_layout=True)
    RocCurveDisplay.from_predictions(y, p, ax=ax[0, 0])
    ax[0, 0].plot([0, 1], [0, 1], "k--", lw=1)
    PrecisionRecallDisplay.from_predictions(y, p, ax=ax[0, 1])
    ax[0, 1].axhline(y.mean(), ls="--", c="k", lw=1)
    CalibrationDisplay.from_predictions(y, p, n_bins=15, strategy="quantile", ax=ax[0, 2])
    sns.histplot(x=p, hue=y, bins=50, stat="density", common_norm=False, ax=ax[1, 0])
    ConfusionMatrixDisplay.from_predictions(y, p >= threshold, normalize="true", ax=ax[1, 1])
    thr = np.linspace(0.01, 0.99, 99)
    ax[1, 2].plot(thr, [f1_score(y, p > t) for t in thr], label="F1")
    ax[1, 2].plot(thr, [precision_score(y, p > t, zero_division=0) for t in thr], label="precision")
    ax[1, 2].plot(thr, [recall_score(y, p > t) for t in thr], label="recall")
    ax[1, 2].legend(); ax[1, 2].set_xlabel("threshold")
    fig.suptitle(name)
    return fig

Six panels, one call, and it answers: does it rank well, does it work at the operating point, are the probabilities meaningful, are the classes separable, what does it confuse, and how sensitive are metrics to the threshold. Choose the threshold on development data before final test evaluation; the test panel is not permission to tune against the test labels. Missingness rates should be plotted separately when nonmissing values are used for drift curves.

Self-check

Runnable noninteractive figure check

This creates an actual calibration counterexample in memory using Agg and closes the figure. Standard deviation describes variation, standard error estimates uncertainty in an average under assumptions, a confidence interval targets a population quantity, and a prediction interval targets a future observation. Do not relabel one as another. Repeated users need grouped resampling rather than treating their rows as independent bootstrap units.

from io import BytesIO
import numpy as np
import matplotlib
matplotlib.use("Agg")
import matplotlib.pyplot as plt
from sklearn.calibration import calibration_curve
y = np.array([1]*5 + [0]*95)
p = np.full(100, .1)
observed, predicted = calibration_curve(y, p, n_bins=5)
assert np.isclose(observed[0], .05) and np.isclose(predicted[0], .1)
assert 1-observed[0] > 1-predicted[0]
with plt.rc_context({"font.size": 10}):
    fig, ax = plt.subplots(figsize=(5, 4), constrained_layout=True)
    ax.plot([0, 1], [0, 1], linestyle="--", color="gray")
    ax.scatter(predicted, observed, label="100 cases; 5 positives")
    ax.set(xlabel="Predicted positive probability", ylabel="Observed positive fraction",
           title="Positive overprediction, negative underconfidence", xlim=(0, 1), ylim=(0, 1))
    ax.legend()
    buffer = BytesIO()
    fig.savefig(buffer, format="png", dpi=100)
    assert len(buffer.getvalue()) > 5000
    assert ax.get_xlabel() and ax.get_ylabel()
    plt.close(fig)
assert not plt.get_fignums()
print("calibration interpretation, rendered PNG, labels and figure cleanup passed")

Seaborn line plots often aggregate repeated x values by an estimator and interval; use units and estimator=None when individual trajectories are the intended object. State the bootstrap unit, binning/bandwidth and hue normalization. Primary contracts: LearningCurveDisplay and PairGrid.

1. What does Anscombe's quartet demonstrate, and what habit does it justify?
2. Why normalise a confusion matrix, and along which axis for per-class recall?
3. A learning curve shows validation still improving at the largest training size. What does that justify spending money on?
4. Give three things a t-SNE plot cannot tell you.
5. Why is jet a bad colormap? Give two independent reasons.
6. When is a PR curve more informative than an ROC curve, and why?
7. You must plot 5 million points. Name two approaches and what each sacrifices.

Where to go next

• Pandas — shaping data into the tidy form these plots expect.
• Scikit-learn — the Display classes used throughout.
• Math for ML notes — the uncertainty these plots should be showing.