Tag: #GQA
Kubernetes, DevOps and MLOps, written from things we ran and measured ourselves · 1 posts
Why the KV Cache is Needed: Remembering to Avoid Recomputation
Language models generate tokens one by one, looking back at all previous tokens with every generation. Because the K and V of past tokens do not change once calculated, we store and reuse them. This is the KV cache. Meas
2026-10-13 · 19 min read #KV 캐시#LLM 서빙#트랜스포머#어텐션#GQA