Tag: #Newton-Schulz
Kubernetes, DevOps and MLOps, written from things we ran and measured ourselves · 1 posts
Muon Commentary: An Optimizer that Orthogonalizes the Gradient Matrix
When reading 2026 LLM pretraining papers, Muon frequently appears in place of AdamW. Muon replaces the momentum matrix with a matrix whose singular values are all 1, evenly distributing updates that were clustered in a f
2026-10-31 · 15 min read #Muon#옵티마이저#AdamW#Newton-Schulz#MUON+