Tag: #vLLM 에서 Dynamo 로
Kubernetes, DevOps and MLOps, written from things we ran and measured ourselves · 4 posts
When separating prefill and decode, separate serving measured on a small model and a slow network
We deployed Dynamo's separate serving on Kubernetes without Grove and sent the same requests as in Part 3. Splitting into two prefill workers and two decode workers within the same server increased the median time to fir
2026-11-13 · 12 min read #NVIDIA Dynamo#vLLM#분리 서빙#prefill decode#NIXLEmpirical measurement of round-robin vs. KV-aware routing when sending the same requests to Dynamo
We sent the requests measured in Part 1 back to Dynamo in the same order. When the frontend was configured for round-robin, the hit rate was identical to vLLM behind a Service. Simply passing through Dynamo does not revi
2026-11-13 · 16 min read #NVIDIA Dynamo#vLLM#KV 인지 라우팅#접두 캐시#KV 캐시What to Check Before Installing the Dynamo Operator: Webhook Scope, Two Installation Methods, and Import List
To deploy Dynamo 1.6.2 on Kubernetes, you must install the operator first. Chart inspection revealed that the default (global) method creates 12 ClusterRoles and 8 webhooks, with one of them intercepting the creation of
2026-11-13 · 15 min read #NVIDIA Dynamo#쿠버네티스 오퍼레이터#승인 웹훅#Helm#폐쇄망How Well Does Prefix Cache Hit behind Multiple vLLM Instances in a Service?
When you group multiple vLLM pods behind a single Kubernetes Service, requests go to random pods. Because prefix caches are isolated per pod, requests with the same prefix that land on different pods are recomputed from
2026-11-13 · 11 min read #NVIDIA Dynamo#vLLM#접두 캐시#KV 캐시#쿠버네티스