Tag: #tensor-parallel
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 1 posts
Inside vLLM (7) — Deployment Tuning, Common Pitfalls, and OOM Triage
A practical order for tuning a vLLM deployment. Covers what gpumemoryutilization actually sets, when to use tensor parallelism versus pipeline parallelism, how to choose quantization and a KV cache data type, and a diagn
2026-08-12 · 10 min read #vllm#gpu#quantization#tensor-parallel#llm