Tag: #distributedtraining
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 1 posts
PyTorch Internals & Advanced Optimization: autograd, torch.compile, FSDP, and Triton
A complete guide to PyTorch internals: autograd engine, torch.compile() with TorchInductor, FSDP distributed training, gradient checkpointing, and custom CUDA operations.
2026-03-17 · 10 min read #pytorch#torch-compile#fsdp#triton#mixedprecision