Tag: #speculative-decoding
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 7 posts
7 posts are tagged #speculative-decoding; the most recent was published on 2026-06-26.
Often tagged together: #llm 5 #vllm 5 #2026-03 4 #inference 3 #kv-cache 3
Most read in the last 90 days:
Making Inference Fast — Speculative Decoding and Throughput Optimization
From the fundamental reason LLM decode is slow, to how speculative decoding boosts speed, variants such as Medusa and EAGLE, chunked prefill and prefill/decode disaggregation, the latency versus throughput trade-off, and
2026-06-26 · 13 min read #speculative-decoding#throughput#inference#mlops#latencyComplete LLM Inference Optimization Guide 2025: vLLM, TensorRT-LLM, KV Cache, Speculative Decoding ♪ Listenable
Everything about LLM inference optimization! vLLM (PagedAttention), TensorRT-LLM (FP8/INT4), KV Cache management, Speculative Decoding, Continuous Batching, FlashAttention, Quantization (GPTQ/AWQ/GGUF), Model Serving (Tr
2026-04-14 · 25 min read #llm-inference#vllm#tensorrt-llm#kv-cache#speculative-decodingLLM Inference Optimization Complete Guide: KV Cache, Speculative Decoding, Continuous Batching ♪ Listenable
A complete guide to pushing LLM inference to its limits. Covers KV Cache, Speculative Decoding, Continuous Batching, PagedAttention, FlashInfer, multi-GPU inference, and DeepSeek MLA in depth.
2026-03-17 · 25 min read #llm#inference#optimization#kv-cache#speculative-decodingThe Complete Guide to LLM Inference Optimization: vLLM, TensorRT-LLM, Speculative Decoding ♪ Listenable
A comparative analysis of key technologies for maximizing LLM inference performance — vLLM, TensorRT-LLM, Speculative Decoding, and KV Cache optimization — with practical code examples and benchmarks.
2026-03-14 · 19 min read #llm#inference-optimization#vllm#tensorrt-llm#speculative-decodingComplete Guide to vLLM Production Serving Optimization: From PagedAttention to Kubernetes Deployment
A comprehensive production-focused guide covering vLLM core architecture including PagedAttention, optimization techniques such as Continuous Batching, Tensor Parallelism, Speculative Decoding, and Prefix Caching, detail
2026-03-07 · 18 min read #llm#vllm#paged-attention#continuous-batching#tensor-parallelismLLM Speculative Decoding Serving Optimization Playbook
LLM Speculative Decoding Serving Optimization Playbook - a hands-on adoption guide as of 2026
2026-03-04 · 14 min read #llm#speculative-decoding#2026-03Accelerating LLM Inference 2-3x with Speculative Decoding: From Theory to Production
A deep dive into the mathematical foundations of Speculative Decoding, the Draft-Verify pipeline, acceptance probability analysis, practical implementation in vLLM/TensorRT-LLM, and Apple's Mirror Speculative Decoding.
2026-03-02 · 8 min read #llm#speculative-decoding#inference#optimization#vllm