Tag: #부하 시험
Kubernetes, DevOps and MLOps, written from things we ran and measured ourselves · 1 posts
How Well Does Prefix Cache Hit behind Multiple vLLM Instances in a Service?
When you group multiple vLLM pods behind a single Kubernetes Service, requests go to random pods. Because prefix caches are isolated per pod, requests with the same prefix that land on different pods are recomputed from
2026-11-13 · 11 min read #NVIDIA Dynamo#vLLM#접두 캐시#KV 캐시#쿠버네티스