Tag: #prefix-caching
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 1 posts
Inside vLLM (5) — Prefix Caching, and Why System Prompt Design Is Performance
A rundown of when vLLM prefix caching hits and when it misses, based on the official design document. Covers how block hashes chain to the block before them, why only full blocks get cached, how a single timestamp placed
2026-08-12 · 7 min read #vllm#prefix-caching#kv-cache#prompt-engineering#llm