Tag: #gguf
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 7 posts
7 posts are tagged #gguf; the most recent was published on 2026-05-16.
Often tagged together: #quantization 6 #2026-03 5 #awq 5 #gptq 5 #llm 5
Most read in the last 90 days:
Local AI & On-Device LLMs 2026 — Ollama · LM Studio · Jan · Msty · Open WebUI · GPT4All · AnythingLLM · Faraday Deep Dive ♪ Listenable
By May 2026, local AI is no longer a hobby. An M4 Max MacBook Pro runs Llama 4 Scout 109B MoE at 24 tokens per second. Desktop runtimes like Ollama, LM Studio, Jan, and Msty unify GUI and CLI, while Open WebUI, AnythingL
2026-05-16 · 23 min read #local-ai#on-device-llm#ollama#lm-studio#janEdge AI & TinyML 2026 — LiteRT / ExecuTorch / Edge Impulse / Jetson / Coral / Hailo / Sipeed K230 / llama.cpp / Phi-4 Deep-Dive Guide ♪ Listenable
A full-stack map of the 2026 Edge AI / TinyML ecosystem — the dual standard formed after TFLite Micro was rebranded as LiteRT and ExecuTorch reached GA, the TinyML cloud workflow created by Edge Impulse, the accelerator
2026-05-16 · 32 min read #edge-ai#tinyml#tflite-micro#litert#executorchDeep Learning Model Quantization Complete Guide: Master INT8, INT4, GPTQ, AWQ, GGUF ♪ Listenable
A comprehensive guide to mastering deep learning model quantization. Understand the principles from FP32 to INT8 and INT4 quantization, and master GPTQ, AWQ, GGUF, bitsandbytes, AutoGPTQ, and llama.cpp through practical
2026-03-17 · 23 min read #quantization#model-compression#gptq#awq#ggufLLM Quantization Practical Guide: GPTQ, AWQ, GGUF Format Comparison and Precision-Performance Trade-offs ♪ Listenable
A practical guide to LLM quantization covering GPTQ, AWQ, and GGUF format comparison with precision-performance trade-off analysis.
2026-03-14 · 21 min read #llm#quantization#gptq#awq#ggufLLM Quantization Comparison Guide — GPTQ, AWQ, GGUF, bitsandbytes Practical Application ♪ Listenable
A comparative guide to LLM quantization techniques including GPTQ, AWQ, GGUF, and bitsandbytes with practical application tips.
2026-03-09 · 24 min read #llm#quantization#gptq#awq#ggufComplete LLM Quantization Comparison: GPTQ, AWQ, GGUF Practical Application Guide ♪ Listenable
From the core principles of LLM quantization to comparative analysis of GPTQ, AWQ, GGUF, and BitsAndBytes techniques, covering practical application in vLLM and llama.cpp environments and quality-performance trade-offs.
2026-03-06 · 14 min read #llm#quantization#gptq#awq#ggufComplete LLM Quantization Comparison — GPTQ vs AWQ vs GGUF ♪ Listenable
A comprehensive guide to LLM Quantization — from quantization fundamentals to comparing GPTQ, AWQ, and GGUF methods, vLLM/llama.cpp integration, and practical benchmarks.
2026-03-03 · 6 min read #llm#quantization#gptq#awq#gguf