Tag: #분산추론
Kubernetes, DevOps and MLOps, written from things we ran and measured ourselves · 1 posts
Distributed Inference and NVIDIA Dynamo: Parallelism, Prefill/Decode Disaggregation, and KV Cache Movement
This article outlines four parallelism and communication characteristics used when a model does not fit on a single GPU, explains why prefill/decode disaggregation is necessary and how the cost of KV cache transfer is pa
2026-10-07 · 33 min read #LLM서빙#분산추론#Dynamo#병렬화#prefill-decode분리