Tag: #chunked-prefill
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 1 posts
Inside vLLM (3) — How Continuous Batching Keeps the GPU Busy
Explains why static batching leaves the GPU idle, and how vLLM iteration-level scheduling removes that idle time. Covers the difference in character between prefill and decode, V1 unified scheduler and its token budget,
2026-08-12 · 7 min read #vllm#continuous-batching#chunked-prefill#llm#inference