Extension points

You wrote a new attention kernel. You point vLLM at it and get this:

        path = _ATTN_OVERRIDES.get(self, self.value)
        if not path:
            raise ValueError(
                f"Backend {self.name} must be registered before use. "
                f"Use register_backend(Backend.{self.name}, 'your.module.YourClass')"
            )

That error is the good outcome. It tells you the seam exists and names the function that opens it. The bad outcome is the one with no error at all: you fork the repo, edit gpu_model_runner.py — or gpu/model_runner.py, the V2 runner that is the default for eligible dense configurations after the platform and feature guards — and six weeks later you are hand-resolving a rebase across a file that has moved under you. Both were in the ten most-churned files in the 180 days before this SHA. §11.1 counted the two rewrites this codebase has already been through; the seams are what let out-of-tree code survive them.

So the contributor's first question is never "how do I make vLLM do X". It is: is there a registry for X, and if not, what is the smallest fork? The answer for most of X is a table below. The answer for the rest is honest and short.

This chapter does not cover adding a model — §8.4 takes that end to end, from WeightsMapper through EntryClass to the registry entry. Everything else lives here.

Every extension seam in vLLM sits in exactly one of the three process families §11.2 laid out: the API server front end, the EngineCore process, and the $N$ worker processes. Which family a seam lives in determines almost everything about it — when it loads, what state it can see, and whether your object gets constructed once or once per rank.

Two groups cross all three: vllm.general_plugins, loaded by an explicit load_general_plugins() call, and vllm.platform_plugins, loaded lazily in every process the first time vllm.platforms.current_platform is touched (vllm/plugins/__init__.py:L21-L23, vllm/platforms/__init__.py:L229). Everything else is a class you name in config, or a decorator you run at import time inside a plugin one of those two mechanisms loaded.

A registration mechanism for a multi-process engine has to solve three things: where does the code come from, which processes run it, and when relative to config. vLLM answers the first with Python entry points. The groups are constants:

# Default plugins group will be loaded in all processes(process0, engine core
# process and worker processes)
DEFAULT_PLUGINS_GROUP = "vllm.general_plugins"
# IO processor plugins group will be loaded in process0 only
IO_PROCESSOR_PLUGINS_GROUP = "vllm.io_processor_plugins"
# Platform plugins group will be loaded in all processes when
# `vllm.platforms.current_platform` is called and the value not initialized,
PLATFORM_PLUGINS_GROUP = "vllm.platform_plugins"
# Stat logger plugins group will be loaded in process0 only when serve vLLM with
# async mode.
STAT_LOGGER_PLUGINS_GROUP = "vllm.stat_logger_plugins"
# Endpoint plugins group is loaded in the API server front end process only.
# Each entry point resolves to a factory returning an `EndpointPlugin`
# (see `vllm/plugins/endpoint_plugins/interface.py`).
ENDPOINT_PLUGINS_GROUP = "vllm.endpoint_plugins"

# make sure one process only loads plugins once
plugins_loaded = False

Note that the comments themselves are the process contract: all processes, process0 only, front end only. The loader is thin — discover, filter by the VLLM_PLUGINS allowlist, call .load(), and catch discovery/import failures per plugin. The subsequent registration callback can still raise and abort startup; plugin code is trusted executable code:

    plugins = dict[str, Callable[[], Any]]()
    for plugin in discovered_plugins:
        if allowed_plugins is None or plugin.name in allowed_plugins:
            if allowed_plugins is not None:
                log_level("Loading plugin %s", plugin.name)

            try:
                func = plugin.load()
                plugins[plugin.name] = func
            except Exception:
                logger.exception("Failed to load plugin %s", plugin.name)

    return plugins

The allowlist semantics are unusual and worth memorising: unset means load everything, and empty string means load nothing, because the env parser splits on commas without special-casing the empty case (vllm/envs.py:L1149-L1156). The one group that inverts this default is vllm.endpoint_plugins, which opens HTTP routes and therefore refuses to load unless explicitly named (vllm/plugins/__init__.py:L120-L131).

The second problem — which processes — is answered by re-entrancy plus a module-level flag. load_general_plugins() is idempotent within a process and called from five places — EngineArgs.__post_init__ (vllm/engine/arg_utils.py:L811), AsyncEngineArgs.add_cli_args (:L2888), vllm/v1/engine/core.py:L118, vllm/v1/worker/worker_base.py:L251 and the model-inspection subprocess entry point vllm/model_executor/models/registry.py:L1531 — so any process that needs plugins gets them regardless of how it was started:

def load_general_plugins():
    """WARNING: plugins can be loaded for multiple times in different
    processes. They should be designed in a way that they can be loaded
    multiple times without causing issues.
    """
    global plugins_loaded
    if plugins_loaded:
        return
    plugins_loaded = True

    plugins = load_plugins_by_group(group=DEFAULT_PLUGINS_GROUP)
    # general plugins, we only need to execute the loaded functions
    for func in plugins.values():
        func()

Work the count for a concrete deployment. Llama-3-8B on one H100 SXM node at TP=8, served with vllm serve: one API server process, one EngineCore process, eight worker processes — ten processes, and your entry-point function runs ten times, in ten separate interpreters, with ten separate copies of every module-level registry dict. That is what "re-entrant" is protecting. A plugin that appends to a global list is fine; a plugin that writes a file, opens a socket, or increments a shared counter is not.

The third problem — timing — is the subtle one, and the clearest evidence is in the argument parser:

    def add_cli_args(
        parser: FlexibleArgumentParser, async_args_only: bool = False
    ) -> FlexibleArgumentParser:
        # Initialize plugin to update the parser, for example, The plugin may
        # add a new kind of quantization method to --quantization argument or
        # a new device to --device argument.
        load_general_plugins()
        if not async_args_only:
            parser = EngineArgs.add_cli_args(parser)

Plugins load before config is built, because ModelConfig._verify_quantization rejects any --quantization value not in QUANTIZATION_METHODS (vllm/config/model.py:L1246, L1326-L1331). Note where the rejection is not: argparse does not constrain the flag at all. ModelConfig.quantization is typed QuantizationMethods | str | None, and literal_to_kwargs emits a metavar rather than choices whenever the union contains str (vllm/engine/arg_utils.py:L190-L204) — and even on the choices path it reads the static Literal, never the mutable list. Get the ordering wrong and your method dies in config validation, not at parse time.

The platform group is the exception to the "call it explicitly" pattern: it fires lazily on the first access to current_platform (vllm/platforms/__init__.py:L229), and the resolver refuses to continue if two out-of-tree platform plugins both claim the host — "Only one platform plugin can be activated".

Finally, what a plugin cannot do. It cannot change a config that is already frozen, it cannot reach across a process boundary, and — for endpoint plugins — the module docstring draws the line explicitly:

An endpoint plugin adds HTTP routes to the OpenAI compatible API server.
Its scope is HTTP surface only. It registers routes and optionally
per app state used by those routes. It must not open new paths into the
engine by reaching the engine the same way an in-tree serving handler does
via `EngineClient` (e.g. `engine_client.collective_rpc(...)`).

If a plugin also needs engine side behavior (a new worker side RPC method,
a custom stat, etc.) pair this entry point with one registered under
`vllm.general_plugins` (see `vllm/plugins/__init__.py`). The
`general_plugins` entry installs the engine side method and the
`endpoint_plugins` entry exposes it over HTTP. The two are registered and
loaded independently where neither implies the other.

A registration line looks like this, from the in-tree test fixture — one package, two groups, one function each:

setup(
    name="vllm_add_dummy_platform",
    version="0.1",
    packages=["vllm_add_dummy_platform"],
    entry_points={
        "vllm.platform_plugins": [
            "dummy_platform_plugin = vllm_add_dummy_platform:dummy_platform_plugin"  # noqa
        ],
        "vllm.general_plugins": [
            "dummy_custom_ops = vllm_add_dummy_platform:register_ops"
        ],
    },
)

§3.4 owns the AttentionBackend ABC and the metadata-builder split; read it for why the interface is shaped that way. Here is what registration costs.

The ABC demands four static methods with no default: get_name, get_impl_cls, get_builder_cls, and get_kv_cache_shape(num_blocks, block_size, num_kv_heads, head_size, cache_dtype_str) (vllm/v1/attention/backend.py:L72-L96). Everything else — head sizes, dtypes, sliding window, sinks, MLA, sparsity — is a classmethod predicate with a permissive default, so a minimal backend overrides four things and inherits thirty-one capability answers.

The builder is where the real work is. Its one abstract entry point returns the per-step metadata object your kernel consumes:

    def build(
        self,
        common_prefix_len: int,
        common_attn_metadata: CommonAttentionMetadata,
        fast_build: bool = False,
    ) -> M:
        """
        Central method that builds attention metadata.
        Some builders (MLA) require reorder_batch to be called prior to build.

        Args:
            common_prefix_len: The length of the common prefix of the batch.
            common_attn_metadata: The common attention metadata.
            fast_build: The meta-data will prioritize speed of building over
                then speed at execution. Can be used for spec-decode where the
                result of a build call may only be used for few layers/iters.
        """
        raise NotImplementedError

Alongside it you declare five class variables (the fifth, supports_draft_decode_metadata_update, is at L687), and the first is the one reviewers check hardest:

class AttentionMetadataBuilder(ABC, Generic[M]):
    # Does this backend/builder support CUDA Graphs for attention (default: no).
    # Do not access directly. Call get_cudagraph_support() instead.
    _cudagraph_support: ClassVar[AttentionCGSupport] = AttentionCGSupport.NEVER
    # Does this backend/builder reorder the batch?
    # If not, set this to None. Otherwise set it to the query
    # length that will be pulled into the front of the batch.
    reorder_batch_threshold: int | None = None
    # Does this backend/builder support updating the block table in existing
    # metadata
    supports_update_block_table: bool = False
    # Whether the builder constructor requires the block-table width.
    requires_block_table_width: ClassVar[bool] = False

AttentionCGSupport is a claim about what your metadata can survive: the four levels run NEVER, UNIFORM_SINGLE_TOKEN_DECODE, UNIFORM_BATCH, ALWAYS (vllm/v1/attention/backend.py:L655-L669), and a backend's level is a hard cap on the engine's graph mode — §3.4 works through why. Declaring ALWAYS when your builder allocates a shape-dependent buffer produces a silently wrong replay, not a crash. Start at NEVER, get correctness, then raise it.

Registration is a decorator that writes into an override dict keyed by an enum member:

    def decorator(cls: type) -> type:
        if is_mamba:
            _MAMBA_ATTN_OVERRIDES[backend] = f"{cls.__module__}.{cls.__qualname__}"  # type: ignore[index]
        else:
            _ATTN_OVERRIDES[backend] = f"{cls.__module__}.{cls.__qualname__}"  # type: ignore[index]
        return cls

    if class_path is not None:
        if is_mamba:
            _MAMBA_ATTN_OVERRIDES[backend] = class_path  # type: ignore[index]
        else:
            _ATTN_OVERRIDES[backend] = class_path  # type: ignore[index]
        return lambda x: x

    return decorator

Out-of-tree code targets one specific member:

    TURBOQUANT = "vllm.v1.attention.backends.turboquant_attn.TurboQuantAttentionBackend"
    # Placeholder for third-party/custom backends - must be registered before use
    # set to None to avoid alias with other backend, whose value is an empty string
    CUSTOM = None

There is exactly one CUSTOM slot per enum: AttentionBackendEnum.CUSTOM (L130) and MambaAttentionBackendEnum.CUSTOM (L193), which write to different override dicts, so two out-of-tree backends can coexist only if one of them is a Mamba backend. Two out-of-tree backends installed in the same environment will silently clobber each other, because register_backend overwrites without complaint. The alternative is to override an existing member by name, which is worse. If you need two, one of them is going in-tree.

§4.1 traces config.json to a kernel; the registry itself is a decorator that does three things at once:

    def _wrapper(quant_config_cls):
        if quantization in QUANTIZATION_METHODS:
            logger.debug(
                "The quantization method '%s' already exists and will be "
                "overwritten by the quantization config %s.",
                quantization,
                quant_config_cls,
            )
        else:
            QUANTIZATION_METHODS.append(quantization)
            # Automatically assume the custom quantization config is supported
            if sq := current_platform.supported_quantization:
                sq.append(quantization)

        if not issubclass(quant_config_cls, QuantizationConfig):
            raise ValueError(
                "The quantization config must be a subclass of `QuantizationConfig`."
            )
        _CUSTOMIZED_METHOD_TO_QUANT_CONFIG[quantization] = quant_config_cls
        return quant_config_cls

It appends to the CLI choice list, tells the current platform it supports your method without asking, and stores the class. That "automatically assume" is a real hazard: your method will be accepted on any platform, and the error when the kernel is missing surfaces at weight-load time, deep inside a worker.

Lookup merges the customised dict over the in-tree one, so your registration wins on a name collision:

    # Update the `method_to_config` with customized quantization methods.
    method_to_config.update(_CUSTOMIZED_METHOD_TO_QUANT_CONFIG)

    return method_to_config[quantization]

Your QuantizationConfig subclass must implement get_name, get_supported_act_dtypes, get_min_capability, get_config_filenames, from_config, and the one that actually does work per layer:

    @abstractmethod
    def get_quant_method(
        self, layer: torch.nn.Module, prefix: str
    ) -> QuantizeMethodBase | None:
        """Get the quantize method to use for the quantized layer.

        Args:
            layer: The layer for the quant method.
            prefix: The full name of the layer in the state dict
        Returns:
            The quantize method. None if the given layer doesn't support quant
            method.
        """
        raise NotImplementedError

Returning None for a layer means "leave it unquantised" — that is how lm_head and per-model ignore lists work, and it is the correct answer for any layer whose shape your kernel cannot handle.

Below the config sits a second, finer seam: kernel selection. A MMLinearKernel answers two questions, hardware and configuration, and both return a reason string on failure:

    @abstractmethod
    def can_implement(cls, config: _ConfigT) -> tuple[bool, str | None]:
        """Check if this kernel can implement the given configuration.

        This method checks configuration-level compatibility (e.g., quantization
        scheme, group sizes, static vs dynamic quantization). It's called after
        is_supported() to determine if this kernel can handle the specific
        quantization configuration.

        Args:
            config: The kernel configuration to check

        Returns:
            A tuple of (can_implement, reason):
                - can_implement: True if this kernel supports the config
                - reason: If not supported, a string explaining why; otherwise None
            ```
        """
        raise NotImplementedError

The selector walks the platform's candidate list in order, collects every rejection string, and if nothing matches raises with all of them concatenated (vllm/model_executor/kernels/linear/__init__.py:L649-L663) — which is why a bad quantization config produces a wall of "CutlassFP8ScaledMMLinearKernel ..., MarlinLinearKernel ..." text rather than one clean error. Write good reason strings; they are the user-facing diagnostic. Note also VLLM_DISABLED_KERNELS, checked first (vllm/model_executor/kernels/linear/__init__.py:L577-L581): a comma-separated list of class names, the fastest way to bisect a kernel-selection bug.

Out-of-tree kernels append to the candidate lists:

    if kernel_type == "mp":
        if platform not in _POSSIBLE_KERNELS:
            _POSSIBLE_KERNELS[platform] = []
        _POSSIBLE_KERNELS[platform].append(kernel_class)

Append, not prepend. Your kernel is tried last, after every in-tree candidate. If an in-tree kernel already says yes to your config, yours never runs — and there is no priority argument on register_linear_kernel. The levers are VLLM_DISABLED_KERNELS, the --linear-backend filter the selector applies to the candidate list (vllm/model_executor/kernels/linear/__init__.py:L650, L353-L380; flag at vllm/engine/arg_utils.py:L1626), and the force_kernel argument in-tree callers pass, which is tried ahead of the platform list (:L606, L634-L645).

Say you want a knob called target_token. The naive path is to add a field to SamplingParams, a field to the OpenAI protocol model, a tensor to InputBatch, a column to SamplingMetadata, and a branch in Sampler.forward — five files in core, and a rebase liability forever. The supported path touches zero core files.

The transport already exists at both ends. On the request side, every OpenAI-compatible protocol model carries a free-form bag:

    vllm_xargs: dict[str, str | int | float | list[str | int | float]] | None = Field(
        default=None,
        description=(
            "Additional request parameters with (list of) string or "
            "numeric values, used by custom extensions."
        ),
    )

which lands in SamplingParams as:

    extra_args: dict[str, Any] | None = None
    """Arbitrary additional args, that can be used by custom sampling
    implementations, plugins, etc. Not used by any in-tree sampling
    implementations."""

On the consumption side, a LogitsProcessor is a batch-level object with four methods. It is constructed once per engine, not once per request:

    @abstractmethod
    def __init__(
        self, vllm_config: "VllmConfig", device: torch.device, is_pin_memory: bool
    ) -> None:
        raise NotImplementedError

    @abstractmethod
    def apply(self, logits: torch.Tensor) -> torch.Tensor:
        """Apply LogitsProcessor to batch logits tensor.

        The updated tensor must be returned but may be
        modified in-place.
        """
        raise NotImplementedError

    @abstractmethod
    def is_argmax_invariant(self) -> bool:
        """True if logits processor has no impact on the
        argmax computation in greedy sampling.
        NOTE: may or may not have the same value for all
        instances of a given LogitsProcessor subclass,
        depending on subclass implementation.
        """
        raise NotImplementedError

    @abstractmethod
    def update_state(
        self,
        batch_update: "BatchUpdate | None",
    ) -> None:
        """Called when there are new output tokens, prior
        to each forward pass.

        Args:
            batch_update: Non-None iff there have been changes
                to the batch makeup.
        """
        raise NotImplementedError

The design consequence of "batch-level, constructed once" is that update_state is where your per-request state has to track the persistent batch's slot churn. BatchUpdate carries three lists — removed, added, moved — and the docstring pins the order they must be applied in (vllm/v1/sample/logits_processor/interface.py:L36-L57). Getting this wrong means request A's parameter applied to request B's logits, which shows up as intermittent garbage under load and never in a single-request test.

is_argmax_invariant() is not a hint; it selects which of two call sites runs your apply(). Non-invariant processors run before penalties and before the greedy/random split:

        # Apply logits processors which can impact greedy sampling.
        for processor in sampling_metadata.logitsprocs.non_argmax_invariant:
            logits = processor.apply(logits)

while invariant ones run later, after temperature, on the random-sampling path only (vllm/v1/sample/sampler.py:L281-L284). Declaring invariance when you are not means greedy requests silently ignore your parameter.

The full path, in order: vllm_xargs → SamplingParams.extra_args → InputBatch.add_request records a BatchUpdate → per step, InputBatch.refresh_metadata() drains it into every processor:

        batch_update = self.batch_update_builder.get_and_reset(self.num_reqs)
        if self.thinking_budget_state_holder is not None and batch_update:
            self.thinking_budget_state_holder.sync_batch(batch_update)
        for logit_proc in self.logitsprocs.all:
            logit_proc.update_state(batch_update)
        if batch_update:
            self.sampling_metadata = self._make_sampling_metadata()

→ the processors ride along on SamplingMetadata.logitsprocs (vllm/v1/worker/gpu_input_batch.py:L961) → Sampler.forward calls apply_logits_processors at vllm/v1/sample/sampler.py:L99-L101. §6.1 walks the rest of that batch through the sampler.

§11.3 enumerates the seventeen abstract methods of SchedulerInterface and quotes get_scheduler_cls(), including its warning that the interface "is not public and compatibility may not be maintained". What the engine guarantees about call order is the part that belongs here, and it is in the ABC's own docstrings:

        """Schedule the requests to process in this scheduling step.

        The scheduling decision is made at the iteration level. Each scheduling
        step corresponds to a single forward pass of the model. Therefore, this
        method is called repeatedly by a busy loop in the engine.

        Essentially, the scheduler produces a dictionary of {req_id: num_tokens}

and its partner:

        """Update the scheduler state based on the model runner output.

        This method is called after the model runner has processed the scheduled
        requests. The model runner output includes generated token ids, draft
        token ids for next step, etc. The scheduler uses this information to
        update its states, checks the finished requests, and returns the output
        for each request.

        Returns:

One schedule() per forward pass, one update_from_output() after it, both on the EngineCore thread — add_request() and finish_requests() are interleaved between them by the busy loop, never during. Construction happens once, after KV caches are profiled and sized: EngineCore.__init__ calls get_scheduler_cls() only at vllm/v1/engine/core.py:L148, after _initialize_kv_caches at L144, and passes the resolved kv_cache_config into the constructor. Your scheduler therefore knows the block count at __init__ time and can size its own structures.

The KV connector is the one seam with a half on each side of the process boundary, and the module docstring is the cleanest statement of the contract in the repo:

The class provides the following primitives:
    Scheduler-side: runs in the scheduler, binds metadata, which
    is used by the worker-side to load/save KV cache.
        get_num_new_matched_tokens() - get number of new tokens
            that exist in the remote KV cache. Might be called multiple
            times for a given request and should be side-effect free.
        update_state_after_alloc() - update KVConnector state after
            temporary buffer alloc by the CacheManager.
# ...
    Worker-side: runs in each worker, loads/saves KV cache to/from
    the Connector based on the metadata.
# ...
        start_load_kv() - starts loading all KVs (maybe async)
        wait_for_layer_load() - blocks until layer i load is done

        save_kv_layer() - starts saving KV for layer i (maybe async)
        wait_for_save() - blocks until all saves are done

"Should be side-effect free" is load-bearing: the scheduler may call get_num_new_matched_tokens() several times for the same request across scheduling attempts. The same class is instantiated twice, once per role, and the factory comment says why the split is enforced rather than merely documented (vllm/distributed/kv_transfer/kv_connector/factory.py:L67-L76). Registration is lazy by module path, so importing your connector's dependencies costs nothing unless it is selected:

    def register_connector(cls, name: str, module_path: str, class_name: str) -> None:
        """Register a connector with a lazy-loading module and class name."""
        if name in cls._registry:
            raise ValueError(f"Connector '{name}' is already registered.")

        def loader() -> type[KVConnectorBase]:
            module = importlib.import_module(module_path)
            return getattr(module, class_name)

        cls._registry[name] = loader

Unlike the attention and quantization registries, this one refuses duplicates rather than overwriting. You cannot shadow NixlConnector; you must pick a new name. §1.6 owns prefill/decode disaggregation and §2.6 owns offload — those are the two things connectors are for.

§8.3 covers writing the kernel. Making it reachable is CustomOp, which is an nn.Module with a per-platform dispatch table chosen once at construction:

        if current_platform.is_rocm():
            return self.forward_hip
        elif current_platform.is_cpu():
            return self.forward_cpu
        elif current_platform.is_tpu():
            return self.forward_tpu
        elif current_platform.is_xpu():
            return self.forward_xpu
        elif current_platform.is_out_of_tree():
            return self.forward_oot
        else:
            return self.forward_cuda

In-tree ops register a name; that name is what --compilation-config's custom_ops list toggles with +name / -name (vllm/model_executor/custom_op.py:L285-L293):

    # Decorator to register custom ops.
    @classmethod
    def register(
        cls,
        name: str,
        dynamic_arg_dims: dict[str, int | list[int]] | None = None,
    ):
        def decorator(op_cls):
            assert name not in op_registry, f"Duplicate op name: {name}"
            op_cls.name = name
            op_cls._dynamic_arg_dims = dynamic_arg_dims
            op_registry[name] = op_cls
            return op_cls

        return decorator

Out-of-tree code uses the sibling decorator, which populates a second dict:

    @classmethod
    def register_oot(cls, _decorated_op_cls=None, name: str | None = None):
        def decorator(op_cls):
            reg_name = name if name is not None else cls.__name__
            assert reg_name not in op_registry_oot, f"Duplicate op name: {reg_name}"
            op_cls.name = reg_name
            op_registry_oot[reg_name] = op_cls
            return op_cls

The substitution happens in CustomOp.__new__ (vllm/model_executor/custom_op.py:L109-L128): every construction of the in-tree class checks op_registry_oot and returns an instance of your class instead. That is why a plugin never has to edit a model file to swap a layer. The in-tree fixture is eleven lines:

# Register CustomRotaryEmbedding to CustomOP.
@RotaryEmbedding.register_oot
class DummyRotaryEmbedding(RotaryEmbedding):
    """Original rotary positional embedding."""

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.addition_config = True

    def forward_oot(self, *args, **kwargs) -> tuple[torch.Tensor, torch.Tensor]:
        return super().forward_oot(*args, **kwargs)

Note the target: register_oot is called on the class being replaced, and the registry key defaults to that class's name. PluggableLayer (vllm/model_executor/custom_op.py:L32-L101) is the same trick for whole layers that are not CustomOps.

The honest list. Each of these is a place where the answer is "fork, or land it upstream".

The published guarantee is narrower than the seam list suggests:

vLLM guarantees the interface of documented plugins, such as `ModelRegistry.register_model`, will always be available for plugins to register models. However, it is the responsibility of plugin developers to ensure their plugins are compatible with the version of vLLM they are targeting. For example, `"vllm_add_dummy_model.my_llava:MyLlava"` should be compatible with the version of vLLM that the plugin targets.

The interface for the model/module may change during vLLM's development. If you see any deprecation log info, please upgrade your plugin to the latest version.

One documented interface is guaranteed. Everything else in this chapter is a seam that exists because someone needed it, not a contract. The same doc's deprecation section is where you find out things like "_Backend in vllm.attention ... has been removed in v0.13.0" — read it before every version bump.

Three artefacts, one command. The processor below is the in-tree reference — it masks every token except target_token for requests that pass one, and leaves all other requests untouched.

The file excerpt. This listing shows state and validation hooks but omits the required apply implementation and supporting imports. It is not a complete installable plugin; use the linked test module and interface at the same pinned revision.

class DummyLogitsProcessor(LogitsProcessor):
    """Fake logit processor to support unit testing and examples"""

    @classmethod
    def validate_params(cls, params: SamplingParams):
        target_token: int | None = params.extra_args and params.extra_args.get(
            "target_token"
        )
        if target_token is not None and not isinstance(target_token, int):
            raise VLLMValidationError(
                f"target_token value {target_token} {type(target_token)} is not int"
            )

    def __init__(
        self, vllm_config: "VllmConfig", device: torch.device, is_pin_memory: bool
    ):
        self.req_info: dict[int, int] = {}

    def is_argmax_invariant(self) -> bool:
        """Never impacts greedy sampling"""
        return False

    def update_state(self, batch_update: BatchUpdate | None):
        def extract_extra_arg(params: SamplingParams) -> int | None:
            self.validate_params(params)
            return params.extra_args and params.extra_args.get("target_token")

        process_dict_updates(
            self.req_info,
            batch_update,
            lambda params, _, __: extract_extra_arg(params),
        )

Two things to copy. validate_params raises VLLMValidationError, which the engine boundary converts to an HTTP 400 rather than a 500 — the interface docstring says a bare ValueError is also converted, for backward compatibility (vllm/v1/sample/logits_processor/interface.py:L61-L67). And process_dict_updates (vllm/v1/sample/logits_processor/builtin.py:L289-L327) is the utility that handles removed/added/moved bookkeeping for sparse per-request state, so you never write that loop yourself. Returning None from the callback means "this request does not use me" and drops the entry.

The registration. Three routes, all converging on _load_custom_logitsprocs. The entry-point route is a two-line addition to pyproject.toml:

    [project.entry-points."vllm.logits_processors"]
    dummy_logits_processor = "your.module.path:DummyLogitsProcessor"

Unlike the general plugin group, this one is scanned directly by the sampler package — LOGITSPROCS_GROUP = "vllm.logits_processors" (vllm/v1/sample/logits_processor/__init__.py:L48) — and it does not honour VLLM_PLUGINS. The docs say so plainly: "vLLM will always load all logits processors which are exposed via entrypoints". A failure to load here is also fatal, not swallowed: the loader re-raises as RuntimeError(f"Failed to load LogitsProcessor plugin {entrypoint}") (vllm/v1/sample/logits_processor/__init__.py:L79-L84). The explicit routes are --logits-processors your.module:MyLP on the server, or LLM(..., logits_processors=[MyLP]) offline, both landing in ModelConfig.logits_processors (vllm/config/model.py:L362-L364).

The test. tests/v1/logits_processors/test_custom_offline.py covers all three routes against a reference LLM, with the entry-point route simulated by monkeypatching importlib.metadata.entry_points — the fixture is careful to keep other groups intact, which is itself asserted (tests/v1/logits_processors/test_custom_offline.py:L47-L70). Requests are constructed half with extra_args={DUMMY_LOGITPROC_ARG: …} — two of the four, at 128 and 67 — and half without, so the test proves the processor applies selectively (tests/v1/logits_processors/test_custom_offline.py:L30-L43). That selectivity assertion is the one a reviewer will look for; a test where every request uses the parameter cannot catch a broken update_state.

Running it.

export VLLM_WORKER_MULTIPROC_METHOD=spawn
pytest -v -s tests/v1/logits_processors
python examples/features/logits_processor/custom.py

The env var and the pytest invocation are copied from the CI job that gates this directory (.buildkite/test_areas/misc.yaml:L28-L30).

Where the tests live, per seam: plugins in tests/plugins/ (installable fixture packages) and tests/plugins_tests/ (the tests that install them); attention registry in tests/test_attention_backend_registry.py; logits processors in tests/v1/logits_processors/; schedulers in tests/plugins_tests/test_scheduler_plugins.py.

That last one is 36 lines and worth copying wholesale as a template — it proves a custom class is reached by making it throw:

class DummyV1Scheduler(Scheduler):
    def schedule(self, throttle_prefills: bool = False):
        raise Exception("Exception raised by DummyV1Scheduler")


def test_scheduler_plugins_v1(monkeypatch: pytest.MonkeyPatch):
    with monkeypatch.context() as m:
        # Explicitly turn off engine multiprocessing so
        # that the scheduler runs in this process
        m.setenv("VLLM_ENABLE_V1_MULTIPROCESSING", "0")

        with pytest.raises(Exception) as exception_info:
            engine_args = EngineArgs(
                model="facebook/opt-125m",
                enforce_eager=True,  # reduce test time
                scheduler_cls=DummyV1Scheduler,
            )

VLLM_ENABLE_V1_MULTIPROCESSING=0 is the trick that makes any engine-core extension testable: it collapses the process split from §11.2 so exceptions and breakpoints reach your test process.

CI runs plugin integration as one serial job that installs each fixture package, runs its test, and uninstalls it:

- label: ":nvidia: (L4) Plugin Integration"
  key: plugin-tests-2-gpus
  timeout_in_minutes: 35
  working_dir: "/vllm-workspace/tests"
  num_devices: 2
  source_file_dependencies:
  - vllm/plugins/
  - tests/plugins/
  commands:
  # begin platform plugin and general plugin tests, all the code in-between runs on dummy platform
  - pip install -e ./plugins/vllm_add_dummy_platform
  - pytest -v -s plugins_tests/test_platform_plugins.py
  - pip uninstall vllm_add_dummy_platform -y
  # end platform plugin tests

source_file_dependencies is the gate: your PR only triggers this job if it touches vllm/plugins/ or tests/plugins/. Adding a test outside those paths means it never runs in CI — a reviewer will catch it, but you should catch it first. The equivalent for sampling is the V1 Sample + Logits job in .buildkite/test_areas/misc.yaml:L5-L33, which watches vllm/sampling_params.py and vllm/v1/ among others.

Reviewer conventions, from docs/contributing/README.md: prefix the PR title with a category and use all applicable ones (L226-L241) — [Core] for scheduler and engine changes, [Kernel] for compute kernels, [Frontend] for API surface, [Misc] sparingly. Install pre-commit and let it run on commit (L83-L90); some hooks are CI-only and run locally with pre-commit run --hook-stage manual mypy-3.11 (L102-L105). Every commit needs a DCO sign-off (L182-L196). If your change is a kernel, the checklist at L258-L276 is specific: a schema, a meta-function registered in Python so dynamic dims work, and torch.library.opcheck() coverage.

• The empty allowlist. VLLM_PLUGINS="" parses to [""], not None, so it means "load nothing" — the docstring for load_endpoint_plugins calls this out explicitly (vllm/plugins/__init__.py:L106-L110). Setting it to empty to "reset" the variable disables every plugin instead.
• Failure handling depends on the phase. The shown discovery/import handler logs Failed to load plugin and continues, but the subsequent registration func() call is outside that handler and can fail startup. Other groups have their own policies. All imported plugins can execute arbitrary process code; allowlists and pinned trusted packages are security controls, not an exception sandbox.
• Registering too late. Anything that mutates a config choice list must run inside a vllm.general_plugins entry point, not at import of your own module in the user's script. Import order in the worker processes is not yours to control.
• Documentation drift. Both examples/features/logits_processor/README.md:L35 and examples/features/logits_processor/custom.py:L7-L8 point you at a test_utils.py at the top of the vllm package for the reference implementation. That file does not exist at a556f3f; the class lives in tests/v1/logits_processors/utils.py and is duplicated inline in custom.py:L50. When a doc points at a path, check it exists before you trust the rest of the page.
• Subclassing the wrong scheduler. The warning at vllm/config/scheduler.py:L181-L187 spells it out: subclass Scheduler while async_scheduling is on and you silently lose async scheduling. Subclass AsyncScheduler.

# 1. See every entry-point group vLLM will scan, and every registration line in-tree.
grep -n "PLUGINS_GROUP\|LOGITSPROCS_GROUP" vllm/plugins/__init__.py vllm/v1/sample/logits_processor/__init__.py

# 2. Watch the plugin loader. INFO is enough for non-default groups; the default
#    group logs at DEBUG.
VLLM_LOGGING_LEVEL=DEBUG vllm serve facebook/opt-125m --enforce-eager 2>&1 | grep -i plugin

# 3. Install the in-tree fixture and confirm the CustomOp swap really happens.
pip install -e tests/plugins/vllm_add_dummy_platform
python -c "from vllm.model_executor.custom_op import op_registry_oot; import vllm_add_dummy_platform; vllm_add_dummy_platform.register_ops(); print(sorted(op_registry_oot))"
pip uninstall vllm_add_dummy_platform -y

# 4. Prove the scheduler seam without a GPU-heavy run.
VLLM_ENABLE_V1_MULTIPROCESSING=0 pytest -v -s tests/plugins_tests/test_scheduler_plugins.py

# 5. Bisect kernel selection by name.
VLLM_DISABLED_KERNELS=CutlassFP8ScaledMMLinearKernel vllm serve MODEL --quantization fp8

Step 5 is the one to remember. When an fp8 model picks a slower kernel than you expect, disabling the winner by class name forces the selector to the next candidate and prints every rejection reason on the way.

Advanced indexing returns a copy; its in-place clamp does not update the original tensor. Explicit indexed assignment does. This independent PyTorch example validates the update only, not plugin discovery or runner eligibility.

import torch
torch.set_num_threads(1)
torch.manual_seed(7)
logits = torch.tensor([[3., 1., -2.], [4., 2., 0.], [5., -1., 2.]])
before = logits.clone()
rows = torch.tensor([0, 2])
logits[rows].clamp_(max=1.)
assert torch.equal(logits, before)
logits[rows] = logits[rows].clamp(max=1.)
assert torch.equal(logits[1], before[1])
assert bool((logits[rows] <= 1.).all())
assert logits[0, 2].item() == -2.

For per-row ceilings, select and reshape them to [selected_rows, 1] before applying the bound. Reorder the state alongside request rows after batch compaction. See runner eligibility and batch state before enabling a custom processor.

• docs/design/plugin_system.md — the canonical plugin doc, including the deprecation list you should read before every version bump.
• docs/features/custom_logitsprocs.md — all three registration routes for logits processors, plus the AdapterLogitsProcessor pattern for wrapping a per-request processor into the batch interface.
• docs/design/endpoint_plugins.md, docs/design/io_processor_plugins.md, docs/design/lora_resolver_plugins.md — the three narrower plugin surfaces this chapter only names.
• docs/contributing/README.md — PR categories, DCO, pre-commit, and the kernel checklist. Read L258-L276 before any [Kernel] PR.
• docs/contributing/model/registration.md — the model seam, owned by §8.4.
• vllm-project/bart-plugin — the plugin system's own reference example: an out-of-tree model shipped as an installable package, linked from docs/design/plugin_system.md:L48.
• SGLang's extension surface is a different shape — grammar backends, the frontend DSL, and the router. §12.4 owns it, and §13.1 puts the two side by side.