Tag: #배치
Kubernetes, DevOps and MLOps, written from things we ran and measured ourselves · 1 posts
Batching and Latency: Batch Size, Concurrent Sessions, and GPU Sharing
An old trick in server inference is that "grouping requests (batching) and running them all at once uses the GPU more efficiently." We look at Clipper's adaptive batching, Clockwork's "one at a time," Triton's dynamic ba
2026-10-15 · 12 min read #비전 모델#실시간#배치#동적 배치#GPU 공유