Tag: #npu
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 3 posts
On-Device and Edge AI — When AI Moves Inside the Device ♪ Listenable
AI inference, once confined to the cloud, is moving into smartphones, PCs, and embedded devices. This post examines the three drivers (latency, privacy, cost), the rise of NPUs and on-device LLMs, and the investment and
2026-06-18 · 19 min read #ai#edge-ai#on-device#npu#llmEdge AI and the NPU — On-Device Inference Accelerators ♪ Listenable
We lay out why edge AI runs inference on the device instead of the cloud (latency, privacy, cost) and the concept of the NPU that makes it possible. From Apple Neural Engine, Qualcomm, Edge TPU, and ARM Ethos to model co
2026-06-16 · 22 min read #edge-ai#npu#on-device#inference#quantizationNPU Deep Dive: How Transformer Architecture Runs Directly on Silicon ♪ Listenable
A complete technical breakdown of how NPUs differ from CPUs and GPUs, how every transformer operation maps to hardware, and why LLM inference is memory-bound — not compute-bound. From Apple ANE to Qualcomm Hexagon, Groq
2026-03-18 · 22 min read #npu#transformer#ai-hardware#quantization#kv-cache