Tag: #prefill decode
Kubernetes, DevOps and MLOps, written from things we ran and measured ourselves · 1 posts
When separating prefill and decode, separate serving measured on a small model and a slow network
We deployed Dynamo's separate serving on Kubernetes without Grove and sent the same requests as in Part 3. Splitting into two prefill workers and two decode workers within the same server increased the median time to fir
2026-11-13 · 12 min read #NVIDIA Dynamo#vLLM#분리 서빙#prefill decode#NIXL