Tag: #pytorch

← All posts

• AI for Everyone, Part 6 — Drawing Digits From Words With a 1.11M Diffusion Model

  We built a conditional diffusion model with 1.11 million parameters that draws a 0 when you type "zero". The forward process that adds noise is a single formula; the reverse process that restores the image is that same l

  2026-08-24 · 8 min read #ai#diffusion#ddpm#generative#pytorch

• AI for Everyone, Part 5 — Colourising Photos With a 0.47M U-Net, and Why the Colours Came Out Washed Out

  The smallest model in this series — a 472K-parameter U-Net — restored colour to greyscale CIFAR-10 images. Shapes survived intact, but the colours came out noticeably washed out. That is not a capacity problem; it is a c

  2026-08-23 · 7 min read #ai#computer-vision#unet#colorization#pytorch

• AI for Everyone, Part 4 — Captioning Images With 1.37M Parameters, and Why Part 3 Bug Was Absent Here

  We joined a CNN encoder to a transformer decoder and captioned Fashion-MNIST images. With 1.37 million parameters and ten minutes of training, the label hit rate reached 91%. This part looks at what cross-attention does

  2026-08-22 · 7 min read #ai#captioning#multimodal#transformer#pytorch

• AI for Everyone, Part 3 — Loss of 0.0017, Accuracy of 7.5%: The Culprit Was One Padding Slot

  We built a VQA model — one that answers questions about an image — with 1.48 million parameters. Training loss fell to 0.0017 while accuracy sat at 7.5%, worse than guessing. The cause was not the model but a single line

  2026-08-21 · 8 min read #ai#vqa#multimodal#debugging#pytorch

• AI for Everyone, Part 1 — Training a 16M-Parameter Language Model From Scratch in 15 Minutes

  Training a language model from scratch on a single GPU. With the TinyStories dataset and a 16-million-parameter decoder-only transformer, we produced readable English fairy tales in 15 minutes. We look at why the causal

  2026-08-19 · 9 min read #ai#llm#transformer#pytorch#hands-on

• A Map of the LLM Training Stack in 2026 — What Each Layer Does For You, and What It Hides

  Sorts LLM training frameworks into three layers and maps out their lineage. The bottom layer is the execution engines — PyTorch distributed, DeepSpeed, Megatron-Core. The middle layer is training loops like torchtitan an

  2026-08-02 · 13 min read #mlops#llm-training#pytorch#trl#framework

• GPU Compiler and Framework Landscape — One Problem, Turning a Graph into a Kernel, a Different Answer at Every Layer ♪ Listenable

  This post puts NVCC and PTX, LLVM, MLIR, Triton, torch.compile, XLA, IREE, and TVM on one map. Different names, different owners, but they all solve the same problem: turning a computation graph into an executable kernel

  2026-08-02 · 21 min read #gpu#compiler#mlir#triton#pytorch

• torchcomms in PyTorch 2.13 — The New Distributed Communication Backend Coming for c10d

  Among the highlights of PyTorch 2.13, released on July 8, 2026, the most structural change is torchcomms — PyTorch Distributed's new communication backend has begun landing in core's CI and DeviceMesh paths. torchcomms i

  2026-07-17 · 10 min read #pytorch#distributed-training#nccl#deep-learning

• Running Small Models Hands-On with a Single RTX 5090 — microGPT, OCR, Music Generation

  I SSHed into a single RTX 5090 (Blackwell, 32GB) and ran a trio of small models by hand. I trained a char-level GPT from scratch in 28 seconds (10.75M parameters, 1.17M tokens/s), pitted a dedicated OCR model (TrOCR) aga

  2026-07-11 · 8 min read #pytorch#gpu#llm#ocr#hands-on

• The 2026 Vision Model Development & Fine-Tuning Guide — CNN, ViT, DETR, SAM 2, VLMs and a Real Decision Tree ♪ Listenable

  Vision model development in 2026 is no longer 'grab a ResNet and call it a day.' Between CNNs, ViTs, DETR variants, SAM 2, and VLMs like LLaVA, Qwen-VL, Gemini Vision, and Claude Vision, your choice for the same photo ca

  2026-05-14 · 20 min read #computer-vision#vision-model#cnn#vit#detr

• [Deep RL] 03. PyTorch Deep Learning Basics: From Tensors to Neural Networks ♪ Listenable

  Learn PyTorch tensor operations, automatic differentiation, and neural network components, then implement TensorBoard monitoring and a GAN example using Atari images.

  2026-03-19 · 20 min read #reinforcement-learning#deep-learning#ai#pytorch

• Building LLM from Scratch: Complete Guide to Understanding GPT through Code ♪ Listenable

  A complete guide to building and understanding Large Language Models from scratch. Implement a full GPT architecture in PyTorch — from tokenizer to Transformer, pretraining, and fine-tuning.

  2026-03-17 · 21 min read #llm#gpt#transformer#from-scratch#deep-learning

• Torch-Titan Complete Guide: Everything About Large-Scale Distributed Training with PyTorch

  A complete guide to mastering large-scale LLM distributed training with PyTorch Titan (torchtitan). Covers FSDP2, pipeline parallelism, tensor parallelism, 4D parallelism, Flash Attention, and mixed precision with hands-

  2026-03-17 · 19 min read #torch-titan#distributed-training#pytorch#fsdp#deep-learning

• PyTorch Internals & Advanced Optimization: autograd, torch.compile, FSDP, and Triton

  A complete guide to PyTorch internals: autograd engine, torch.compile() with TorchInductor, FSDP distributed training, gradient checkpointing, and custom CUDA operations.

  2026-03-17 · 10 min read #pytorch#torch-compile#fsdp#triton#mixedprecision

• PyTorch Advanced Techniques Complete Guide: torch.compile, Custom Ops, Memory Optimization

  A complete guide to mastering PyTorch advanced techniques. Covers torch.compile, custom operators, memory optimization, Gradient Checkpointing, torch.vmap, functorch, and PyTorch Profiler with hands-on examples.

  2026-03-17 · 17 min read #pytorch#advanced#torch-compile#memory-optimization#custom-operators

• DeepSpeed Complete Guide: ZeRO Optimization and Large-Scale Model Training

  A complete guide to mastering Microsoft DeepSpeed. Learn ZeRO-1/2/3 optimization, Offload, pipeline parallelism, mixed precision, MoE, and DeepSpeed Inference with practical configurations and code examples.

  2026-03-17 · 15 min read #deepspeed#zero-optimization#distributed-training#llm#pytorch

• PyTorch Complete Guide: Zero to Hero — From Tensors to Distributed Training ♪ Listenable

  A comprehensive guide to mastering PyTorch from the basics to advanced techniques. Learn tensor operations, automatic differentiation, CNN/RNN/Transformer implementation, and distributed training with practical examples

  2026-03-17 · 21 min read #pytorch#deep-learning#ai#python#neural-network

• Deep Learning Debugging Complete Guide: From Diagnosing Training Failures to Performance Optimization

  A complete guide to systematically diagnosing and resolving deep learning training failures. Covers Loss NaN, vanishing/exploding gradients, overfitting, slow convergence, and out-of-memory errors with real-world code ex

  2026-03-17 · 19 min read #deep-learning#debugging#pytorch#training#optimization

• The Complete torchaudio Guide — From Audio Processing to Speech Recognition, TTS, and Music Analysis

  From audio loading and spectrogram transforms to Mel filter banks, MFCC, speech recognition (Wav2Vec2/Whisper), TTS, speaker diarization, and noise reduction — everything about audio AI with PyTorch.

  2026-03-02 · 9 min read #ai-platform#pytorch#torchaudio#audio#speech-recognition

• The Complete torchvision Guide — From Image Classification to Object Detection and Segmentation

  From torchvision transforms v2 and pretrained models (ResNet to ViT) to datasets, Object Detection (Faster R-CNN, YOLO), Segmentation, and hands-on fine-tuning — master computer vision in practice with PyTorch.

  2026-03-02 · 8 min read #ai-platform#pytorch#torchvision#computer-vision#cnn