Skip to main content

Learn Learning paths Courses

LLM Serving

Learn the machinery of serving without a model.

고급 · Lessons 25 · Lab 7

Curriculum

Why Serving Differs From Training

prefill and decode Are Different Jobs reading
Quiz: Serving Metrics quiz

Token Streaming and SSE

Why Streaming Changes the UX reading
Implementing a Token Streaming Server Yourself lab
Quiz: Streaming quiz

Continuous Batching and Queueing

What Static Batching Throws Away reading
Simulating a Continuous Batching Scheduler lab
Quiz: Batching quiz

Choosing a Model Server, Reproducibly

Fix the Measurement Contract Before the Tool Name reading
Quiz: Choosing a Model Server quiz

GPU Memory Sizing and Quantisation

Building the Memory Budget by Hand reading
Building a KV Cache Memory Calculator lab
Quiz: Memory Sizing quiz

When the KV Cache Doesn't Fit on the GPU

KV-Aware Routing and Disaggregated Serving

Gateway, Rate Limiting and Cost Control

Evaluation and Regression Prevention

Reference docs