Comparing vLLM and SGLang, and How to Benchmark Properly

Status: Draft (Updated 2026-10-07), with some measured data reflected. This article is written based on descriptions verified in official documentation. Actual measurements taken firsthand are limited to the parts citing the experiments in Section 4 and "Measured: vLLM vs. SGLang on the Same Model, Same GPU" at the end of Section 5 (one small model, one GPU). Comparisons involving large models, multi-GPU setups, and other workloads are unmeasured. Please read "Versions and Dates Verified" and "What Was Not Verified" at the end of the article first. Writing rule: Measured data is the default, and anything not measured is marked as "Unmeasured".

Table of Contents

• 1. Design Philosophy
• 2. Feature Comparison Table
• 3. Selection Criteria: By Workload Shape
• 4. Launching the Server and Calling via OpenAI-Compatible APIs
• 5. How to Benchmark Properly
  • Measured: vLLM vs. SGLang on the Same Model, Same GPU
• 6. Try It Yourself
• 7. Potential Interview Questions
• 8. Common Misconceptions
• References
• Verified Versions and Dates, and What Was Not Verified

Choosing between the two engines is not a matter of taste, but of workload shape and measurement. This article outlines how vLLM and SGLang designed their prefix caches differently, how far they have converged by now, what to suspect first for each workload, and how to design and read benchmarks that support those judgments. The author has not operated or measured SGLang firsthand. There are no self-measured comparisons of the two engines' superiority; the content is based solely on official documentation, source code, papers, and experience measured with vLLM in the author's experimental environment.

What You Can Explain in an Interview After Reading This Article

1. You can distinguish and articulate the differences between block hash chains (vLLM) and radix trees (SGLang), and the fact that their features now overlap significantly.
2. You can discuss selection criteria by workload shape—such as prefix reuse, structured outputs, and model support—along with their limitations.
3. You can explain benchmark designs incorporating open-loop loads, Poisson arrivals, prefix cache pitfalls, and percentiles, as well as the procedure for converting results into GPU counts.

1. Design Philosophy

vLLM originated as a general high-throughput engine from the PagedAttention paper (SOSP 2023). In the V1 architecture, the API server process communicates with the engine core process (scheduler and KV manager) via ZMQ, and the engine core manages workers—one per GPU (official architecture documentation). Model Runner V2 has been the default since v0.29.0 (release notes; some ROCm models and others are exceptions).

The SGLang paper (arXiv 2312.07104) started not as an engine, but as a system for efficiently executing LLM programs. Its frontend language provides generate (gen, select) and parallel control (fork, join) primitives, and its runtime features two techniques. RadixAttention keeps the KVs of prompts and generated results in a radix tree using LRU even after requests finish, employing cache-aware scheduling that prioritizes requests with longer matching prefixes. Compressed finite state machines decode multiple tokens determined by constraints in a single step. The paper claims up to a 6.4x increase in throughput and up to a 3.7x reduction in latency. The conditions were mostly A10G 24GB (some A100), Llama-7B/70B, Mixtral-8x7B, etc., under prefix-reuse-heavy workloads like agents, JSON decoding, RAG, and multi-turn conversations, while the compared vLLM version was v0.2.5. A footnote in the paper notes that it used an older version because RadixAttention was only partially integrated into the latest vLLM as an experimental feature at that time. These are not figures compared against current v0.30.0.

vLLM: 블록 해시 사슬 (블록 16 토큰)       SGLang: 라딕스 트리 (page-size 기본 1)
요청 A  [B0]-[B1]-[B2]                    root
요청 B  [B0]-[B1]-[B3]                     └ "시스템 프롬프트 ..." (공통 노드 하나)
해시 = H(부모 해시, 블록 토큰, 추가 키)           ├ "질문 A ..." -> 생성 결과
해시 맵 조회, 빈 블록 큐에서 LRU 방출              └ "질문 B ..."
                                          리프부터 방출(기본 lru), 사용 중 노드는 잠김

SGLang documentation's --page-size default is 1, meaning prefix matching is possible at the token level under document defaults, whereas vLLM only hits in units of full blocks (16 tokens) (Part 04). Recent SGLang release notes state that in v0.5.19, the unified radix tree became the default for all configurations; in v0.5.21, the prefix cache became the Rust core default (falling back to Python for unsupported configurations); and legacy implementations (SWARadixCache, MambaRadixCache, HiRadixCache) were removed. Around the same time, vLLM enabled prefix caching by default and added support for structured outputs (xgrammar, guidance). Therefore, differences in philosophy do not directly mean differences in feature availability.

2. Feature Comparison Table

This simply indicates whether an item exists in the documentation and is not a measure of maturity or performance superiority. The baseline is vLLM v0.30.0 docs and SGLang v0.5.21 docs.

vLLM documentation notes that PD (prefill-decode) separation does not increase throughput, limiting its purpose to separate tuning of TTFT and ITL and tail latency control.

3. Selection Criteria: By Workload Shape

Model support speed was not quantitatively verified. Instances observed via release notes are as follows: Qwen3.8-Flash-Next and Hy4-preview were added to new models by vLLM v0.29.0 (2026-09-09) before SGLang v0.5.20 (09-18), whereas GLM-5.3-Flash, K2-Horizon, and Nanbeige4.2 saw SGLang v0.5.20 beat vLLM v0.30.0 (09-22). Since these only represent release inclusion timing and not day-0 status, this does not serve as evidence that either side is faster. Both engines have Transformers-based fallbacks, allowing models without native implementations to be spun up anyway.

4. Launching the Server and Calling via OpenAI-Compatible APIs

vllm serve Qwen/Qwen2.5-1.5B-Instruct --max-model-len 8192            # 기본 포트 8000
sglang serve Qwen/Qwen3-0.6B --host 0.0.0.0 --port 30000              # 문서 quickstart (기본 host 127.0.0.1)
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" \
  -d '{"model":"Qwen/Qwen2.5-1.5B-Instruct","messages":[{"role":"user","content":"안녕"}],"max_tokens":32}'

Things to know before operations. vLLM's --api-key only authenticates the /v1, /v2, /inference path and does not authenticate other paths like /invocations , so official documentation warns against relying on this alone for security. vLLM servers apply the model repository's generation_config.json by default, which can cause sampling defaults to vary by model (turn off with--generation-config vllm ). SGLang requires turning on --enable-metrics to emit Prometheus metrics, and you can receive the number of cache-hit tokens in --enable-cache-report of the response via prompt_tokens_details .

5. How to Benchmark Properly

Determine the workload shape first. Input and output length distributions, prefix sharing ratio and popularity distribution, arrival patterns, and SLOs to maintain. Obtain these from actual logs if possible; otherwise, state assumptions and observe sensitivity.

Closed-loop vs. open-loop. Closed-loop fixes the concurrent user count, waits for a response, and then sends the next request (--max-concurrency). If the server slows down, the load decreases accordingly, masking overload conditions. Open-loop sends requests at a designated arrival rate regardless of server state (--request-rate). The model below (an explanatory simulation, not actual measurements) looks at a single server taking 0.25 seconds per request (capacity of 4 req/s).

열린 루프 3 req/s     평균   0.64s  p99   2.26s
열린 루프 4.5 req/s   평균  57.43s  p99 114.55s   <- 용량 초과, 시간이 갈수록 대기열 증가
닫힌 루프 동시 4      평균   1.00s  p99   1.00s    <- 사용자 수만큼의 대기만 보임

Both tools' --request-rate defaults to inf , sending all requests at time 0 (confirmed from source code). While maximum throughput is visible, wait times get mixed into TTFT, making it unusable for latency measurement. Providing a finite value creates arrivals via a Poisson process, and vLLM's --burstiness uses 1.0 for Poisson, with smaller values representing a Gamma distribution where requests bunch up together. You can gradually ramp up the request rate using --ramp-up-strategy .

Length distribution. vLLM's random dataset has a --random-range-ratio default of 0.0, resulting in fixed lengths. Use ShareGPT or your own logs for actual distributions. If a model finishes early due to EOS, output lengths vary, so to fix lengths vLLM uses --ignore-eos (SGLang ignores EOS by default and turns it off via --disable-ignore-eos ). Warm-up. vLLM has a --num-warmups default of 0, while SGLang has a --warmup-requests default of 1. The first request mixes in compilation and CUDA graph costs.

The most common pitfall: Prefix cache. vLLM benchmark documentation warns against sending identical prompts to the same server vllm bench serve warns that repeating this can leave the prompt from the previous run in the prefix cache and inflate throughput, and the same goes for synthetic datasets with fixed seeds. The experiments in Part 04 illustrate this magnitude. In a second pass reading the same document again, the TTFT p50 dropped from 575.5ms to 48.6ms (measured once with LMCache, concurrency 1, 1.5B model). The difference made by repetition can be greater than the difference between engines. Here is how to respond.

• Clear the cache between runs. vLLM provides vllm bench sweep serve to clear the cache or restart the server, while SGLang's bench_serving has --flush-cache which calls the /flush_cache of the SGLang server and the /reset_prefix_cache of the vLLM server (SGLang documentation and vLLM VLLM_SERVER_DEV_MODE=1 comments note that vLLM must be envs.py for it to open).
• Change the seed, or create realistic hit rates using datasets designed with intentional prefix sharing (vLLM prefix_repetition, SGLang generated-shared-prefixand --gsp-group-distribution zipf).
• Record the prefix hit rate (prefix_cache_hits_total ÷ queries_total) alongside the results. Compare with the features that would be turned on in production enabled.

Percentiles, not averages. The default outputs are vLLM's --metric-percentiles default 99(including average and median), and SGLang's average, median, and p99. If you need p95, specify it like --metric-percentiles 50,95,99 . Also look at the sample size. With 100 requests, p99 is determined by the top one or two values, and the experiments in Part 04 had 10 requests per pass, making p95 the maximum value. Since vLLM documentation states that the definitions of TTFT, TPOT, and ITL may vary by tool, it is fairer to benchmark both engines using the same client. For SGLang's bench_serving , you can call the vLLM server with --backend vllm . Run the workload generator on a machine separate from the server to prevent it from becoming a bottleneck, and when using --max-concurrency , measure client latency (client_queue_time) separately. The vLLM documentation recommends GuideLLM for production benchmarks.

Tool names have changed. In the SGLang v0.5.21 source, sglang.bench_serving issues a deprecation warning and has been moved to sglang.benchmark.serving (document examples still use the old name). For vLLM, it is vllm bench serve .

Connect to capacity planning. 1) Set an SLO (e.g., p99 TTFT 2s, p99 TPOT 50ms). 2) Measure the p99 in the steady-state region while increasing the request rate. 3) The maximum request rate that maintains the SLO is the capacity of a single instance (based on goodput). 4) Number of instances = Peak request rate ÷ (Capacity × Target utilization) plus a margin for failures. Using illustrative numbers of a peak of 20 req/s, capacity of 4 req/s, and target utilization of 0.7, 20 ÷ 2.8 = 7.1, which rounds to 8 units plus 1 standby unit for a total of 9. The shape of the curve matches the description in the PagedAttention paper: as the request rate increases, latency grows gradually until it exceeds capacity, at which point the queue continuously grows and latency spikes dramatically.

p99 TTFT
   │                              /
SLO ┼ - - - - - - - - - - - - - -/- -    <- 이 교점의 요청률이 용량
   │                       _____/
   │  ____________________/   (이 지점부터 급증)
   └────────────────────────────── 요청률

Actual measurement: vLLM vs SGLang on the same model and GPU

We benchmarked both engines using the same model (Qwen3-1.7B, bf16), the same GPU (a single 8GB-class card), and the same client (different random inputs per request, 128 output tokens, measuring TTFT and ITL via streaming). vLLM 0.31.0 used --gpu-memory-utilization 0.85 --max-num-seqs 64, SGLang 0.5.21 used --mem-fraction-static 0.8 --max-running-requests 64, and the context length was 4,096 for both. Because the meanings of the two options differ, the memory settings are not identical. For the input 512 table below, the server was restarted three times and the values are the median of the three runs. The values measured on the first run and the values averaged over three runs were within 1% of each other, with the sole exception of vLLM concurrency 64 below. The input 2,048 table shows single-run values without repetition.

The vLLM column for concurrency 64 varied every time the server was launched. Across three server runs, the throughput was 792, 842, and 843 tok/s, and the median TTFT was 5.7, 2.1, and 2.2 seconds. Only the first launched server was slow, and its server log showed a KV cache of 22,048 tokens (the other two had 26,560 tokens). Even with the same options, vLLM measures memory to determine the KV allocation space, so its capacity varied on every server launch (from 22,048 to 27,504 tokens). At this point where 64 concurrent requests nearly fill the cache with 640 tokens each, a 20% difference in capacity changed the TTFT by 2.7x (Part 14). SGLang had 25,282 tokens in all three runs, and the values were within 1% of each other. The vLLM concurrency 64 in the table above is the median of three runs, and this single cell has a large repetition error.

• The throughput was nearly identical(vLLM was 3% higher at concurrency 32 and within 1% otherwise). We interpret this as happening because token generation for this model is bound by memory bandwidth (Part 21), causing both engines to hit the same hardware limit. This means the reason to choose an engine is not the raw throughput for such small models. It was different for larger models (see "Large Model" below). Multi-GPU was not measured (unmeasured).
• The size per KV cache token was identical, but capacity varied depending on settings. The SGLang log showed 25,282 tokens (K 1.35 + V 1.35GB) and vLLM showed 22,048 tokens. Since the bytes per token (approx. 114.7KB) matches the formula, the capacity difference is due to the difference in memory settings. GPU memory usage was 7,373 MiB for SGLang and 6,168 MiB for vLLM, meaning SGLang's configuration consumed more memory on the same card.
• TTFT and tail latency shapes differed. At concurrency 32, SGLang's median TTFT was 1,093ms, which is longer than vLLM's (669ms), while its ITL p99 was 33ms, shorter than vLLM's (224ms). At concurrency 64, vLLM had a lower TTFT (2.2s vs 4.0s, though vLLM varied across servers) while SGLang had a lower ITL p99 (101ms vs 231ms). In the initial single measurement, SGLang also showed a lower TTFT at concurrency 64, but that was due to the slow first server mentioned above. While this hints that how new request input processing is interleaved with decoding differs, the root cause was not broken down (estimated).
• Prefix caching had the same effect on both. With a common 3,000-token prefix, the first request TTFT was 394ms for vLLM and 399ms for SGLang, and subsequent ones were 28ms and 36ms. When disabled (--no-enable-prefix-caching, --disable-radix-cache), it returned to 393ms and 399~416ms.

Differences in a larger model (Qwen3-4B-FP8)

The same test was repeated by launching servers with Qwen3-4B-FP8 (fp8 weights to fit in 8GB), which is about 2.4 times larger than 1.7B bf16. Because the precisions differ, this should not be read as a direct comparison with 1.7B, but rather as "two engines on the same model." Both engines used a context length of 4,096 and a maximum concurrency of 64. The server logs showed KV cache sizes of 8,880 and 13,232 tokens for vLLM, and 13,036 tokens in both runs for SGLang.

• Throughput, which was identical for the small model, diverged in the large model. SGLang was 11% higher at concurrency 8 and 23~39% higher at concurrency 32 (the two vLLM values at concurrency 32 are from the server with 8,880 KV tokens and the server with 13,232 KV tokens. Comparing servers with similar KV sizes gives 365 vs 450, a 23% difference). The cause was not broken down. It is likely a difference in the kernels reading fp8 weights, but this was not verified (estimated).
• The maximum ITL during interruption followed the same trend as 1.7B. vLLM was 602ms (both runs) and SGLang was 770~772ms, showing vLLM was lower (compared to 227ms vs approx. 460ms for 1.7B). The TTFT of the interrupting 3,500-token input itself was 1.22s for vLLM and 0.78~0.80s for SGLang, making SGLang faster (for 1.7B, both were about 0.47s). One could interpret that decoding had longer idle times in exchange for finishing faster, but the cause was not broken down (estimated).
• vLLM's TTFT at concurrency 32 varied between 7.2s and 3.4s across servers. The server with 8,880 KV tokens was slower, which matches the phenomenon observed in 1.7B (Part 14). This is why engines should not be compared based on a single server launch.

Cost of structured outputs (JSON schema constraints)

We compared requests using SGLang 0.5.21's JSON schema constrained decoding (response_format of json_schema, with xgrammar as the default grammar backend) against requests asking for JSON solely via prompts on the same server (Qwen3-1.7B, thinking disabled, output approx. 74 tokens, different inputs per request).

• The cost of constrained decoding was small enough to be unmeasurable under this condition(-3% to +5% within margin of error). This means the grammar mask calculation was small compared to GPU token generation.
• All outputs were valid even without constraints. Because this model achieved 128/128 correct outputs with just prompts on an easy schema, the benefit of constraints (guaranteeing format) did not manifest in this test. It would likely differ for small models prone to formatting errors or for complex schemas, which were not tested (unmeasured). Because these are single-run values, repetition error is unknown.
• An issue encountered during measurement. When this test was first run with a memory ratio of 0.8, the server connection dropped on the first request. Lowering the ratio to 0.7 and rerunning resulted in the Triton kernel ... device-loaded after serving started (free device mem: 0.64 GiB). Pre-load it during engine init to avoid CUDA OOM warning being printed in the server logs. This implies that since constrained decoding loads GPU kernels for the first time after service start, the server may crash on the first request unless sufficient memory headroom (approx. 0.6GiB or more) is left (the cause is inferred from the warning message and context, as the termination log at 0.8 was not preserved).

Reducing weights to fp8: Both engines showed the same trend

Under the same server configuration, to SGLang, --quantization fp8 was added (cutting the 3.28GB bf16 weights roughly in half).

The increase from 66 to 100 tok/s at concurrency 1 is interpreted as the effect of reduced weight bytes read per generated token (the bandwidth upper-bound formula from Part 21). Since it matches the 105.7 tok/s for 1.7B fp8 measured with vLLM in Part 21, this gain is a property of precision rather than the engine. Quality (accuracy) was not measured in this test (unmeasured).

When Long Inputs Intervene: Default Behaviors Differ

We inserted three 3,500-token input requests into a single request currently being decoded and measured the maximum Inter-Token Latency (ITL) across the intervened interval.

• In SGLang, reducing the chunk size under default settings did not reduce stalling. Even when the chunk size was reduced to 512, the maximum ITL was 472ms, which matched the time it takes to process 3,500 tokens all at once (approx. 470ms). We interpret this as decoding pausing until input processing finished. --enable-mixed-chunk When enabling , it matched the time of a single chunk (approx. 230ms for 2048 tokens, approx. 69ms for 512 tokens), and this value matched vLLM's result. The documentation's option description (mixing chunking and decoding in a single batch) aligns with the test results.
• vLLM appears to mix chunking with decoding by default. This is because the default setting's stall of 227ms matches the time for a 2,048-token chunk (approx. 250ms). However, this is an estimate since vLLM's default chunk size does not appear in the logs.
• We checked SGLang's --chunked-prefill-size automatic determination value. On this GPU, it was logged as 2048 (listed as None in the documentation table), and max_prefill_tokens=16384 was logged together. This may vary on GPUs with different memory sizes (unmeasured).
• When running concurrency 32 across three server runs on a server using mixed chunking and 512 together, the throughput was 835–837 tok/s (default SGLang: 807–809), TTFT median was 321–323ms (default: 1,089–1,094ms), and ITL p99 was 66–67ms (default: 31–35ms), with the three runs matching within 1%. The tradeoff of reducing and mixing chunks—reducing TTFT by about 3.4x while roughly doubling the ITL tail—repeated consistently. vLLM's --max-num-batched-tokens 512 under the same load yielded a throughput of 834–835, TTFT of 335–338ms, and ITL p99 of 70ms (default: 224ms). We did not dissect why this happens (estimated).
• Server startup times were similar. For SGLang, initial startup took 60 seconds and restarts took 30–36 seconds; for vLLM, initial startup took 71 seconds and restarts took 35–55 seconds (with model files present).

The most practical takeaway from this test is that default settings matter more than engine names. Options with the same name have different default values, so to make a valid comparison, you must check chunk sizes, mixed chunking, memory settings, and prefix cache in the logs, align them, and then run measurements. The phrase "aligning conditions" mentioned in the first half of Section 5 of this article manifested as concrete numbers in this test.

6. Hands-On

All operations are read-only or local computations.

curl -sL https://raw.githubusercontent.com/vllm-project/vllm/v0.30.0/vllm/benchmarks/serve.py \
  | grep -n -E '"--(request-rate|burstiness|max-concurrency|num-warmups|goodput|metric-percentiles)"'
curl -sL https://raw.githubusercontent.com/sgl-project/sglang/v0.5.21/python/sglang/benchmark/serving.py \
  | grep -n -E '"--(request-rate|max-concurrency|warmup-requests|flush-cache)"'

The open-loop vs. closed-loop comparison in Section 5 is the output of the code below (it is an explanatory simulation rather than actual measurement; increasing the request count will also increase latency in the capacity-exceeded region).

import random, statistics as st
random.seed(1)
S, N = 0.25, 4000                       # 요청 하나에 0.25초 = 용량 4 req/s, 요청 4000개

def open_loop(rate):                    # 포아송 도착: 서버가 느려도 요청은 제 시간에 온다
    t = free_at = 0.0; lat = []
    for _ in range(N):
        t += random.expovariate(rate)
        free_at = max(t, free_at) + S
        lat.append(free_at - t)
    return lat

def closed_loop(users):                 # 사용자 고정: 응답을 받아야 다음 요청을 보낸다
    ready, free_at, lat = [0.0] * users, 0.0, []
    for _ in range(N):
        i = ready.index(min(ready)); t = ready[i]
        free_at = max(t, free_at) + S
        lat.append(free_at - t); ready[i] = free_at
    return lat

def row(name, lat):
    s = sorted(lat)
    print(f"{name:<14} 평균 {st.mean(lat):6.2f}s  p99 {s[int(.99 * (N - 1))]:6.2f}s")

for r in (3, 4.5, 5): row(f"열린 {r} req/s", open_loop(r))
for u in (4, 16):     row(f"닫힌 동시 {u}", closed_loop(u))

Evaluate the experiments from Article 04 against the standards of this article to find weaknesses. Concurrency 1, fixed documents, reusing the same document, and having only 10 documents all make p95 effectively equal to the maximum value, which are all problematic. Areas for improvement include concurrency 4 (from the experimental setup --max-num-seqs=4), Poisson arrivals, length distribution, and recording prefix hit rates. This experiment was run on a temporary pod and this article was not re-run. Risks include memory contention and GPU occupancy with other workloads on the same node, and pods are deleted via --rm and terminated 1 hour after remaining.

7. Potential Interview Questions

1. How do vLLM's and SGLang's prefix caches differ? vLLM looks up a hash chain—formed by chaining parent hashes every 16 token blocks—in a map and evicts them from the free block queue using LRU. SGLang stores token sequences in a radix tree, evicting them starting from the leaves, with a default page-size of 1. Nowadays, since vLLM also enables this by default, the difference lies less in feature availability and more in granularity, scheduling policy, and eviction policy.
2. How do you verify the claim that SGLang is better for agent workloads? Send realistic turn-structured traces in an open-loop fashion using the same hardware, model, precision, and client. Record the prefix hit rate alongside, and compare p99 TTFT and SLO-compliant throughput. Paper figures are compared against vLLM v0.2.5, so they cannot be directly applied to current versions.
3. When should you use open-loop versus closed-loop? To find capacity and SLOs, use an open-loop approach by increasing the request rate to find the surge point. A closed-loop approach is used to mimic environments where an upstream system limits concurrency, and it reveals the limit where latency under overload ceases to visibly increase.
4. Benchmark results are abnormally good. What should you suspect? Check whether the prefix cache hit due to repeating the same prompt, whether warmup was omitted, whether outputs ended early due to EOS, or whether the request rate was inf causing wait times to mingle into TTFT. Verify hit rate metrics along with input and output token distributions.
5. How do you translate benchmark results into the required number of GPUs? Treat the maximum request rate that satisfies the SLO as the instance capacity, divide the peak request rate by the capacity multiplied by the target utilization, and add redundancy for failures. Calculations must be based on p99 rather than average throughput, and measurements must be retaken if the workload shape changes.

8. Common Misconceptions

• "SGLang is always fast (or vLLM is always fast)." It is workload-dependent. The paper's 6.4x figure is an upper bound compared to vLLM v0.2.5 on workloads with heavy prefix reuse, and Section 6.2 of the paper explains that this improvement stems not only from KV cache reuse but also from intra-program parallelism and faster constrained decoding. In other words, it is not solely an effect of the prefix cache. Conversely, the same section notes that there is almost no improvement in multi-turn conversations with long outputs.
• "Only SGLang has a prefix cache." vLLM also enables it by default.
• "Maximum throughput equals capacity." Throughput that satisfies the SLO (goodput) is the capacity.
• "You can compare using average latency." The tail determines service quality, and averages hide queue surges.
• "You can decide based on numbers from a single run." Run repeatedly to observe variance, and record with fixed cache states and versions.

References

Only materials actually opened and inspected are listed.

• SGLang paper: https://arxiv.org/abs/2312.07104 (body verified via arxiv.org/html), PagedAttention paper: https://arxiv.org/abs/2309.06180
• SGLang documentation (https://docs.sglang.io): get-started/quickstart, server_arguments under advanced_features, hyperparameter_tuning, structured_outputs, radix_eviction_policy, lora, speculative_decoding, pd_disaggregation, sgl_model_gateway, developer_guide/bench_serving, supported-models/transformers_fallback, llms.txt
• SGLang source: https://github.com/sgl-project/sglang/tree/v0.5.21 at python/sglang/bench_serving.py, python/sglang/benchmark/serving.py
• SGLang releases: https://github.com/sgl-project/sglang/releases (v0.5.19, v0.5.20, v0.5.21)
• vLLM documentation (https://docs.vllm.ai/en/latest/): design/arch_overview, benchmarking/cli, serving/online_serving/openai_compatible_server, structured_outputs under features, speculative_decoding, quantization, lora, multimodal_inputs, disagg_prefill, kv_offloading_usage, serving/expert_parallel_deployment, serving/data_parallel_deployment, sitemap.xml
• vLLM source: https://github.com/vllm-project/vllm/tree/v0.30.0 at vllm/benchmarks/serve.py, vllm/benchmarks/datasets/datasets.py, vllm/envs.py. Releases: https://github.com/vllm-project/vllm/releases (v0.29.0, v0.30.0)

Verified Versions, Dates, and Unverified Items

• Verification date is 2026-10-05. The latest stable vLLM is v0.30.0 (2026-09-22), and the latest SGLang is v0.5.21 (2026-10-02).
• SGLang Hierarchical Cache (HiCache) failed to start in this environment. When enabling the hierarchical cache using host memory, the server terminated during startup with Not enough host memory available. Requesting 1.85 GB but only have -10.74 GB free . Reading the source (pool_host/base.py), it was a check (HICACHE_HOST_MEMORY_RESERVE_BYTES) that always reserves and subtracts a 10GiB buffer from available host memory for comparison. Since this test node had about 15GB of RAM and about 10GB of available memory, it could not pass regardless of the request size. Thus, scenario comparisons like vLLM's LMCache (Section 8) could not be performed. This can be interpreted as an operational guideline stating that to use hierarchical cache, available host memory must be greater than 10GiB plus the request volume (source confirmed; not tested on large hosts).
• Unverified items: Performance comparison of the two engines on current versions under conditions outside Section 5 measurements (paper figures are based on vLLM v0.2.5), decision rules for SGLang's --chunked-prefill-size automatic determination value (logged as 2048 on the 8GB-class GPU in this test, but the rule was not verified), engine comparison on large models and multi-GPU setups, current utilization level of SGLang's frontend language (verified only up to the existence of a tutorial in the docs), day-0 model support status, and whether /reset_prefix_cache is VLLM_SERVER_DEV_MODE=1 that it only opens in (confirmed only via environment variable comments and SGLang documentation), and the realism of the simulation used for benchmark explanations. The fact that latency increases over time under open-loop overload is based on simulation and general queuing theory and was not verified by opening a separate source.