Skip to main content

Learn Learning paths Courses

MiniMind — Train a Small Language Model Yourself, End to End

Bake a small model from tokenizer to DPO on two CPUs

고급 · Lessons 30 · Lab 10

Curriculum

Tokenizer

Data preparation — padding and packing

Model architecture

Pretraining and the loss curve

Checkpoints and resuming

SFT and loss masking

LoRA

DPO — preference learning

Inference — KV cache and sampling

Evaluation and limits

Reference docs