Tag: #cache_salt
Kubernetes, DevOps and MLOps, written from things we ran and measured ourselves · 1 posts
Re-measuring continuous batching, chunked prefill, and prefix caching with repeated measurements
I re-measured vLLM's three core features using a 0.6B model. This time, I repeated each condition three to four times, used the median, and included a baseline with the features completely disabled. Running one request a
2026-10-27 · 15 min read #vLLM#연속 배칭#chunked prefill#접두사 캐시#cache_salt