Tag: #scaling-laws
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 2 posts
LLM Landmark Papers Roundup 2026 - Transformer / Scaling Laws / Flash Attention / Mamba / DeepSeek-R1 / Titans Deep Dive ♪ Listenable
From the 2017 Attention Is All You Need paper to 2026 Titans and DeepSeek-R1, a thematic roundup of the 50+ landmark papers that built the LLM era. Transformer, BERT, GPT 1-3, Scaling Laws, Chinchilla, InstructGPT, PaLM,
2026-05-16 · 21 min read #llm-papers#transformer#scaling-laws#flash-attention#mambaLarge-Scale Model Training Complete Guide: Strategies for Pre-training 100B+ Parameter LLMs ♪ Listenable
A complete practical guide to training hundred-billion parameter LLMs. Covers scaling laws (Chinchilla), Megatron-LM, 3D parallelism, checkpointing strategies, training stability, and data mixing with hands-on examples.
2026-03-17 · 21 min read #large-scale-training#llm#megatron-lm#distributed-training#scaling-laws