Tag: #cuda-optimization
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 1 posts
How Matrices Fly on GPU: Complete Deep Dive from GEMM to FlashAttention
Matrix multiplication accounts for over 80% of deep learning compute. This post dissects every layer of optimization from naive O(n³) code to cache blocking, cuBLAS, Tensor Cores, and the IO-aware revolution of FlashAtte
2026-03-18 · 15 min read #gemm#matrix-multiplication#flash-attention#gpu#parallel-computing