Tag: #peft

← All posts

• Do You Need a New Learning Rate When You Change LoRA Rank? — The Two Regimes μA (2026) Splits, and the Limits of That Evidence

  Two competing beliefs circulate in practice at the same time: that changing the rank in LoRA means you have to re-find the optimal learning rate, and that using 1/r scaling makes the learning rate independent of rank. Th

  2026-07-16 · 18 min read #llm#lora#fine-tuning#peft#training

• LLM Fine-tuning Frameworks 2026 — A Deep Dive into Axolotl, Unsloth, LLaMA-Factory, TRL, PEFT, and TorchTune ♪ Listenable

  A complete map of the 2026 LLM fine-tuning ecosystem. Open-source frameworks like Axolotl, Unsloth, LLaMA-Factory, TRL, PEFT, and TorchTune. LLM Foundry (MosaicML, acquired by Databricks). Cloud fine-tuning APIs from Mod

  2026-05-16 · 28 min read #llm#finetuning#axolotl#unsloth#llama-factory

• LLM Fine-Tuning 2026 Deep Dive — LoRA · QLoRA · DoRA · GaLore · Unsloth · Axolotl · TRL · PEFT · MLX-LM Complete Guide ♪ Listenable

  In 2026, the LLM fine-tuning ecosystem has exploded from the simple adapters of LoRA into a sprawling tree — QLoRA at 4-bit, DoRA with weight decomposition, GaLore with gradient projection. Hugging Face PEFT 0.14 and TRL

  2026-05-16 · 24 min read #llm-fine-tuning#lora#qlora#dora#galore

• Complete Guide to LLM Fine-tuning with Unsloth 2025: QLoRA, 4-bit Quantization, 2x Faster Training ♪ Listenable

  Everything about LLM fine-tuning with Unsloth! QLoRA/LoRA theory, 4-bit quantization (bitsandbytes), Unsloth 2x speed secret, Llama 3/Mistral/Qwen fine-tuning, data preparation, training config, VRAM optimization, GGUF/G

  2026-03-25 · 17 min read #unsloth#llm#fine-tuning#qlora#lora

• Fine-tuning in Practice: Building Your Own Model with LoRA and QLoRA

  You don't need 7 H100s. With LoRA and QLoRA, you can fine-tune a 70B model on a single consumer GPU. A practical end-to-end guide with working code.

  2026-03-18 · 7 min read #fine-tuning#lora#qlora#llm#ai-development

• LLM Fine-tuning Complete Guide: Master LoRA, QLoRA, RLHF, and DPO ♪ Listenable

  A complete guide covering every LLM fine-tuning technique. From Full Fine-tuning to LoRA, QLoRA, RLHF, DPO, and Instruction Tuning — master them all with HuggingFace PEFT and hands-on examples.

  2026-03-17 · 21 min read #llm#fine-tuning#lora#qlora#rlhf

• HuggingFace Ecosystem Complete Guide: Master Transformers, Datasets, PEFT, and Accelerate

  A complete guide to mastering the entire HuggingFace ecosystem. Learn Transformers, Datasets, Tokenizers, PEFT, Accelerate, Diffusers, and Hub API with hands-on examples.

  2026-03-17 · 18 min read #huggingface#transformers#peft#accelerate#nlp

• LLM Fine-tuning Practical Guide: Efficient Model Adaptation with LoRA, QLoRA, and PEFT

  A comprehensive guide to LLM fine-tuning theory and practice. Covers LoRA low-rank decomposition principles, QLoRA 4-bit quantization techniques, PEFT library usage, dataset preparation strategies, and hyperparameter tun

  2026-03-13 · 10 min read #llm#fine-tuning#lora#qlora#peft

• Practical Guide to LLM Fine-Tuning: Efficient Domain Adaptation with LoRA, QLoRA, and PEFT ♪ Listenable

  An in-depth analysis of LoRA low-rank decomposition, QLoRA 4-bit quantized fine-tuning, and the Hugging Face PEFT library. Covers dataset preparation, hyperparameter tuning, evaluation metrics, comparison with full fine-

  2026-03-11 · 13 min read #llm#fine-tuning#lora#qlora#peft

• Practical LLM Fine-Tuning — Building Your Own Model with LoRA, QLoRA, and PEFT

  A practical guide to LLM fine-tuning. From the mathematical principles of LoRA to training large models on consumer GPUs with QLoRA, and leveraging the PEFT library — all with detailed code examples.

  2026-03-02 · 7 min read #llm#fine-tuning#lora#qlora#peft

• LoRA: Efficient Fine-tuning of Large Language Models — Paper Analysis

  A mathematical analysis of the core principles behind the LoRA (Low-Rank Adaptation) paper, along with practical implementation using HuggingFace PEFT.

  2026-03-01 · 17 min read #ai-papers#lora#fine-tuning#llm#peft