Tag: #ollama
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 7 posts
7 posts are tagged #ollama; the most recent was published on 2026-06-12.
Often tagged together: #quantization 4 #vllm 4 #english 3 #inference 3 #llama-cpp 3
Most read in the last 90 days:
- Local LLM Inference Optimization — From Quantization to Breaking the VRAM Ceiling
- The Complete Guide to vLLM & Ollama: LLM Serving Engine Setup, Parameters, and Environment Variables
- Local AI & On-Device LLMs 2026 — Ollama · LM Studio · Jan · Msty · Open WebUI · GPT4All · AnythingLLM · Faraday Deep Dive
Local LLM Inference Optimization — From Quantization to Breaking the VRAM Ceiling ♪ Listenable
Privacy concerns, cost pressure, and big-tech fatigue are driving a local LLM revival. We map the entire landscape of local inference optimization: VRAM-first hardware thinking, GGUF and AWQ quantization, llama.cpp vs vL
2026-06-12 · 16 min read #llm#inference#quantization#llama-cpp#vllmLLM Serving & Local Inference in 2026 — vLLM / llama.cpp / MLX / Ollama / LM Studio / SGLang / TGI Deep Dive ♪ Listenable
A map of the 2026 LLM serving and inference landscape. Datacenter camp (vLLM, SGLang, TGI, Triton, TensorRT-LLM), local camp (llama.cpp, MLX, llamafile, Ollama, LM Studio, GPT4All), emerging camp (KTransformers, MLC LLM,
2026-05-16 · 25 min read #llm#model-serving#inference#vllm#llama-cppLocal AI & On-Device LLMs 2026 — Ollama · LM Studio · Jan · Msty · Open WebUI · GPT4All · AnythingLLM · Faraday Deep Dive ♪ Listenable
By May 2026, local AI is no longer a hobby. An M4 Max MacBook Pro runs Llama 4 Scout 109B MoE at 24 tokens per second. Desktop runtimes like Ollama, LM Studio, Jan, and Msty unify GUI and CLI, while Open WebUI, AnythingL
2026-05-16 · 23 min read #local-ai#on-device-llm#ollama#lm-studio#janThe 2026 Rising Open-Source Map — A Practitioner's Survey of OpenClaw, n8n, Langflow, Dify, and Ollama
A practitioner's survey of the open-source projects that exploded on GitHub in 2026. What OpenClaw, n8n, Langflow/Dify/Flowise, and Ollama are, why they're trending, when to use them, what the risks are — and how to vet
2026-05-14 · 16 min read #open-source#github#trending#openclaw#n8nAI Desktop Apps in 2026 — A Snapshot of Granola, Cleft, Lex, Highlight, Raycast AI, Ollama, and the Quiet Rise of the Ambient-AI Category ♪ Listenable
Spring 2026: the center of gravity for everyday AI use has quietly moved from chatbot tabs to desktop apps. Granola for meeting notes, Cleft / Superwhisper / MacWhisper for local dictation, Lex for AI-first writing, High
2026-05-14 · 36 min read #ai-desktop#granola#cleft#lex-app#highlight-aiOpen-Source LLM Landscape Guide: Models, Tools, and Deployment in 2026 ♪ Listenable
A comprehensive guide to the open-source LLM ecosystem in 2026. Covers the leading model families (Llama, Mistral, Gemma, Qwen, DeepSeek), local inference tools (Ollama, llama.cpp, vLLM), fine-tuning techniques (LoRA, QL
2026-03-17 · 20 min read #open-source#llm#llama#mistral#gemmaThe Complete Guide to vLLM & Ollama: LLM Serving Engine Setup, Parameters, and Environment Variables ♪ Listenable
A comprehensive deep dive comparing vLLM PagedAttention architecture and Ollama local LLM runtime environments. Covers everything from installation, server startup, API calls, key CLI arguments, sampling parameters, envi
2026-03-01 · 46 min read #vllm#ollama#model-serving#inference#gpu