vLLM Feature Map: What Exists, When It Benefits, and What Changed in Each Version

Status: Draft (2026-10-27), partially measured. Feature lists and options were verified directly on 2026-10-27 from the official vLLM 0.32.0 documentation (repository docs/) and release notes from 0.8.0 to 0.32.0. Measurements were conducted using the official 0.31.0 (released 2026-10-05) container. The 0.32.0 image was not used as it required a fresh download, and discrepancies between the two versions' documentation are noted in each section. Added on 2026-10-27. Upon receiving approval, measurements were retaken using the vLLM 0.32.0 image along with Qwen3-0.6B, Qwen3-0.6B-GPTQ-Int8, Qwen3-4B-AWQ, and Qwen3-VL-2B-Instruct. A comparison of the two versions was added to Section 3-1, and a "Re-measurement on 2026-10-27" subsection was added to each part. The 0.31.0 figures were not removed. Writing Rules: Measured data serves as the baseline; unmeasured items are marked as "Unmeasured", and unverified items are marked as "Unverified".

Table of Contents

• 1. What This Series Covers
• 2. Feature Map
• 3. Summary of Direct Measurements
  • 3-1. Re-measurement on 2026-10-27: v0.32.0 and Conversational Models
  • Limitations of This Measurement
• 4. Version-by-Version Changes
• 5. Where to Start
• References

1. What This Series Covers

vLLM is a serving engine for language models. It started with PagedAttention (Kwon et al., SOSP 2023), but now includes a vast array of features such as schedulers, caches, decoding acceleration, output enforcement, quantization, multi-GPU support, and operational tools. This site already features several articles covering vLLM.

• "Anatomy of LLM KV Cache": Principles of KV Cache, PagedAttention, Continuous Batching, Prefix Caching, LMCache, NVIDIA Dynamo
• "Toss ML/AI Infra Preparation": vLLM Internal Architecture (Part 4), Comparison with SGLang (Part 5), Load Testing 1.7B Model (Part 14), LoRA (Part 15), Dynamo Practice (Part 24), OpenTelemetry (Part 25)

This series focuses on filling in the features not covered by those articles. Overlapping features are summarized briefly and linked to previous articles.

2. Feature Map

"What it saves" refers to the cost reduced by the feature, and "Benefit conditions" are the conditions where effectiveness was observed in measurements or documentation.

OpenAI-compatible APIs (/v1/completions, /v1/chat/completions, /v1/embeddings, /v1/responses , etc.) and /metrics were omitted from the table because they act as gateways to all of these features rather than being features themselves. All measurements in this series were performed using this API, and the metrics are gathered in Part 8.

Some features cannot be used together. According to the compatibility table in the 0.32.0 documentation, speculative decoding cannot be used alongside LoRA, pooling, or prompt embedding inputs (❌). Even during our actual measurements, version 0.31.0 emitted warnings when n-gram speculative decoding was enabled, stating "Disabling asynchronous schedule" and "Model Runner V2 does not yet support ngram, falling back to V1 runner."

3. Summary of Measured Values

All measurements were conducted on a single 8GB laptop GPU (Ada generation) with vLLM 0.31.0, Qwen3-Reranker-0.6B (bf16, a 0.6B causal language model fine-tuned for reranking), and a container memory limit of 4GiB or 4.5GiB. The same conditions were repeated three to four times to take the median, and the first batch after spinning up the server was excluded as warmup.

Features that were not measured include tensor, pipeline, data, and expert parallelism (single GPU), disaggregated serving (for the same reason), and EAGLE, MTP, and suffix speculative decoding. EAGLE was skipped because draft weights (pickle format with its own license) and suffix require additional packages. Multimodal inputs and AWQ/GPTQ, which were unmeasured in the initial edition, were measured in the remeasurement in Section 3-1.

3-1. October 27, 2026 Remeasurement: 0.32.0 and Chat Models

Limitations of this measurement

• The model used in the initial measurements cannot hold a conversation. Since we decided to use existing weights we already had rather than downloading new ones, we initially used Qwen3-Reranker-0.6B as the generative model (re-measured with chat Qwen3-0.6B in Section 3-1). Because it is a model fine-tuned for re-ranking (a "yes/no" judgment task), it almost always replies with "No." when asked via chat, and tends to repeat the same phrase when prompted to write longer text. While this does not affect content-agnostic measurements such as throughput, latency, and memory, the "content" of tool calls and structured outputs, as well as speculative decoding acceptance rates, may differ from general chat models. We have noted this in each section.
• Small models, small GPUs. For the 0.6B model, fixed costs (kernel execution, scheduling) account for a large share of the runtime, which will differ proportionally for larger models.

4. What Changed Across Versions

vLLM releases a new version about every two weeks. I have picked out only the changes that matter when reading previous posts or migrating old configurations (checked from the release notes).

Some features that disappeared during the transition from V0 to V1 are also missing. The V1 guide in 0.32.0 best_of, per-request logit processors, GPU↔CPU KV exchange (swap on preemption), and per-request structured output backends are marked as "Removed". In V1, preemption defaults to recomputation (RECOMPUTE) rather than swapping.

5. Where to start

• If throughput is lacking: Part 2 (Batching and Chunking), Part 6 (Quantization), Part 7 (Offloading)
• If individual response latency matters: Part 3 (Speculative Decoding), Part 6 (Quantization)
• If building agents: Part 4 (Structured Outputs), Part 5 (Tool Calling)
• If operating a server: Part 8 (Sleep, Startup, Metrics, Authentication)
• If models do not fit in the GPU: Part 9 (Parallelism), Part 6 (Quantization)

References

• vLLM Documentation (0.32.0): https://docs.vllm.ai/en/latest/ , Repository docs/features/, docs/configuration/, docs/usage/
• vLLM Release Notes 0.8.0~0.32.0: https://github.com/vllm-project/vllm/releases
• Kwon et al., "Efficient Memory Management for Large Language Model Serving with PagedAttention", SOSP 2023. https://arxiv.org/abs/2309.06180
• Yu et al., "Orca: A Distributed Serving System for Transformer-Based Generative Models", OSDI 2022.
• Leviathan et al., "Fast Inference from Transformers via Speculative Decoding", ICML 2023. https://arxiv.org/abs/2211.17192
• Dong et al., "XGrammar", MLSys 2025. https://arxiv.org/abs/2411.15100
• Zhong et al., "DistServe", OSDI 2024. https://arxiv.org/abs/2401.09670
• Previous Series: Anatomy of LLM KV Cache, Preparing Toss ML/AI Infra