Tag: #flashattention
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 2 posts
The Evolution of Attention — MQA, GQA, FlashAttention, and Long Context
We analyze the memory and compute cost of standard attention, then explain how MQA and GQA shrink the KV cache and how FlashAttention optimizes IO. We compare sliding-window and long-context techniques and trace how all
2026-06-26 · 18 min read #llm#attention#flashattention#gqa#mqaTop LLM Papers 2024-2026 - Llama, DeepSeek, Qwen, Mistral, Phi, RLHF, DPO, CoT, RAG, FlashAttention, vLLM Reading List
A curated reading list of 30+ must-read LLM papers for engineers building with LLMs in 2024-2026. Covers foundation models (Llama 3/4, DeepSeek-V3/R1, Qwen3, Mistral, Phi-4, Gemma 3), training innovations (MoE, MLA, GQA)
2026-05-16 · 19 min read #llm#papers#llama#deepseek#qwen