Tag: #pretraining
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 2 posts
The Training Recipe — From Pre-training to Post-training, and What Reports Write Down
Comparing, exactly as written in the reports, the three stages and six context extensions of Llama 3, the three-stage pre-training of Qwen3, the learning-rate schedule and two-phase YaRN extension of DeepSeek-V3, and the
2026-08-12 · 6 min read #ai-papers#model-internals#pretraining#training-recipe#data-mixtureHow These Models Were Actually Built — Dissecting the 2026 Open-Weight Pipeline
Reading the cards and technical reports of the open-weight models sitting near the top of Hugging Face as of August 2026, this post lays out the production pipeline in order, from data collection to quantized deployment.
2026-08-02 · 17 min read #llm#pretraining#moe#post-training#quantization