Tag: #학습률 탐색
Kubernetes, DevOps and MLOps, written from things we ran and measured ourselves · 1 posts
AdamW vs Muon vs MUON+ Measured on a Small GPT
We trained a 34M-parameter GPT on 100M tokens of FineWeb-Edu, varying only the optimizer. Even after independently tuning the learning rate for each optimizer, matching the actual weight decay intensity, and aligning the
2026-10-31 · 17 min read #Muon#MUON+#AdamW#옵티마이저#LLM 사전학습