Tag: #structured-generation
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 1 posts
The New Throne Beyond vLLM: 5 Reasons SGLang Is Changing the LLM Inference Landscape ♪ Listenable
From SGLang's RadixAttention-based KV cache management and 29% throughput advantage through hyper-specialized design, to its 4,000-line Python zero-overhead scheduler, Prefill-Decode Disaggregation, and Compressed FSM-ba
2026-03-01 · 35 min read #sglang#llm-inference#vllm#radix-attention#model-serving