Tag: #matrix-multiplication
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 3 posts
Google TPU Deep Dive: How Systolic Arrays Solve Matrix Multiplication Perfectly
A complete technical breakdown of how Google's Systolic Array achieves extreme efficiency for matrix multiplication. From INT8 inference and bfloat16, to XLA compiler optimizations and TPU Pod distributed inference - wit
2026-03-18 · 15 min read #tpu#google#systolic-array#model-serving#jaxNVIDIA GPU and CUDA Architecture Deep Dive: Why GPUs Dominate AI
From H100 hardware specs to Tensor Core WMMA API. A complete breakdown of the SIMT execution model, shared memory tiling, warp divergence, and why GPUs are purpose-built for AI workloads.
2026-03-18 · 14 min read #cuda#gpu#nvidia#matrix-multiplication#parallel-computingHow Matrices Fly on GPU: Complete Deep Dive from GEMM to FlashAttention
Matrix multiplication accounts for over 80% of deep learning compute. This post dissects every layer of optimization from naive O(n³) code to cache blocking, cuBLAS, Tensor Cores, and the IO-aware revolution of FlashAtte
2026-03-18 · 15 min read #gemm#matrix-multiplication#flash-attention#gpu#parallel-computing