Tag: #CacheBlend
Kubernetes, DevOps and MLOps, written from things we ran and measured ourselves · 1 posts
LMCache: Taking KV Cache Out of the GPU and Across Engines
Prefix cache in GPU memory gets evicted when space is tight, and it is isolated per engine instance. LMCache offloads KV to CPU memory, disk, or remote storage, and retrieves it when the same prefix reappears instead of
2026-10-13 · 34 min read #KV 캐시#LMCache#vLLM#KV 오프로딩#CacheBlend