Tag: #lora

← All posts

• Do You Need a New Learning Rate When You Change LoRA Rank? — The Two Regimes μA (2026) Splits, and the Limits of That Evidence

  Two competing beliefs circulate in practice at the same time: that changing the rank in LoRA means you have to re-find the optimal learning rate, and that using 1/r scaling makes the learning rate independent of rank. Th

  2026-07-16 · 18 min read #llm#lora#fine-tuning#peft#training

• LLM Fine-Tuning 2026 Deep Dive — LoRA · QLoRA · DoRA · GaLore · Unsloth · Axolotl · TRL · PEFT · MLX-LM Complete Guide ♪ Listenable

  In 2026, the LLM fine-tuning ecosystem has exploded from the simple adapters of LoRA into a sprawling tree — QLoRA at 4-bit, DoRA with weight decomposition, GaLore with gradient projection. Hugging Face PEFT 0.14 and TRL

  2026-05-16 · 24 min read #llm-fine-tuning#lora#qlora#dora#galore

• AI Image Generation 2026 — Flux / Midjourney 7 / Ideogram 3 / Recraft / SD 3.5 / GPT-4o / Imagen 4 Deep Dive ♪ Listenable

  AI image generation in 2026 is no longer a single-model story. Black Forest Labs Flux 1.1 Pro and Kontext set a new bar for photorealistic open-weight quality, Midjourney 7 cemented itself as the aesthetic standard, Ideo

  2026-05-15 · 27 min read #ai-image#image-generation#flux#midjourney#ideogram

• AI Engineering in Practice — LLM API, RAG, Agents, LoRA/DPO, Vector DB, Evaluation, Observability, Prompt Injection (2025)

  The real difficulty of production LLM API calls, why RAG is not simple lookup, agent patterns (ReAct, Plan-Execute, ReWOO), when to and when NOT to fine-tune (LoRA, QLoRA, DPO), a vector DB decision matrix, why LLM evalu

  2026-04-15 · 7 min read #ai-engineering#llm#rag#ai-agent#lora

• Complete Guide to LLM Fine-tuning with Unsloth 2025: QLoRA, 4-bit Quantization, 2x Faster Training ♪ Listenable

  Everything about LLM fine-tuning with Unsloth! QLoRA/LoRA theory, 4-bit quantization (bitsandbytes), Unsloth 2x speed secret, Llama 3/Mistral/Qwen fine-tuning, data preparation, training config, VRAM optimization, GGUF/G

  2026-03-25 · 17 min read #unsloth#llm#fine-tuning#qlora#lora

• Fine-tuning in Practice: Building Your Own Model with LoRA and QLoRA

  You don't need 7 H100s. With LoRA and QLoRA, you can fine-tune a 70B model on a single consumer GPU. A practical end-to-end guide with working code.

  2026-03-18 · 7 min read #fine-tuning#lora#qlora#llm#ai-development

• LLM Fine-tuning Complete Guide: Master LoRA, QLoRA, RLHF, and DPO ♪ Listenable

  A complete guide covering every LLM fine-tuning technique. From Full Fine-tuning to LoRA, QLoRA, RLHF, DPO, and Instruction Tuning — master them all with HuggingFace PEFT and hands-on examples.

  2026-03-17 · 21 min read #llm#fine-tuning#lora#qlora#rlhf

• Generative AI & Diffusion Models: Complete Guide from Stable Diffusion to Video Generation ♪ Listenable

  A complete technical guide covering DDPM math, Stable Diffusion architecture, ControlNet, LoRA fine-tuning, and Sora video generation.

  2026-03-17 · 12 min read #generativeai#diffusion-model#stable-diffusion#controlnet#videogeneration

• LLM Fine-tuning Practical Guide: Efficient Model Adaptation with LoRA, QLoRA, and PEFT

  A comprehensive guide to LLM fine-tuning theory and practice. Covers LoRA low-rank decomposition principles, QLoRA 4-bit quantization techniques, PEFT library usage, dataset preparation strategies, and hyperparameter tun

  2026-03-13 · 10 min read #llm#fine-tuning#lora#qlora#peft

• Practical Guide to LLM Fine-Tuning: Efficient Domain Adaptation with LoRA, QLoRA, and PEFT ♪ Listenable

  An in-depth analysis of LoRA low-rank decomposition, QLoRA 4-bit quantized fine-tuning, and the Hugging Face PEFT library. Covers dataset preparation, hyperparameter tuning, evaluation metrics, comparison with full fine-

  2026-03-11 · 13 min read #llm#fine-tuning#lora#qlora#peft

• Practical LLM Fine-Tuning — Building Your Own Model with LoRA, QLoRA, and PEFT

  A practical guide to LLM fine-tuning. From the mathematical principles of LoRA to training large models on consumer GPUs with QLoRA, and leveraging the PEFT library — all with detailed code examples.

  2026-03-02 · 7 min read #llm#fine-tuning#lora#qlora#peft

• LoRA: Efficient Fine-tuning of Large Language Models — Paper Analysis

  A mathematical analysis of the core principles behind the LoRA (Low-Rank Adaptation) paper, along with practical implementation using HuggingFace PEFT.

  2026-03-01 · 17 min read #ai-papers#lora#fine-tuning#llm#peft