Tag: #Prometheus
Kubernetes, DevOps and MLOps, written from things we ran and measured ourselves · 1 posts
Operational Features: Sleep Mode, Startup Time, /metrics, and the Paths That --api-key Ignores
We tested sleep mode to clear the GPU without shutting down the server, compile caches to reduce startup time, /metrics for monitoring, and authentication firsthand. Step 2 sleep cleared GPU memory from 6,168MiB to 606Mi
2026-10-27 · 17 min read #vLLM#sleep mode#잠자기 모드#기동 시간#torch.compile