Tag: #on-device
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 3 posts
On-Device and Edge AI — When AI Moves Inside the Device ♪ Listenable
AI inference, once confined to the cloud, is moving into smartphones, PCs, and embedded devices. This post examines the three drivers (latency, privacy, cost), the rise of NPUs and on-device LLMs, and the investment and
2026-06-18 · 19 min read #ai#edge-ai#on-device#npu#llmEdge AI and the NPU — On-Device Inference Accelerators ♪ Listenable
We lay out why edge AI runs inference on the device instead of the cloud (latency, privacy, cost) and the concept of the NPU that makes it possible. From Apple Neural Engine, Qualcomm, Edge TPU, and ARM Ethos to model co
2026-06-16 · 22 min read #edge-ai#npu#on-device#inference#quantizationEdge AI Complete Guide 2025: On-Device Inference, Model Optimization, TensorRT/ONNX/CoreML ♪ Listenable
Everything about Edge AI! On-device inference (TensorRT/ONNX Runtime/CoreML/TFLite), model optimization (quantization/pruning/knowledge distillation), hardware (NVIDIA Jetson/Apple Neural Engine/Qualcomm NPU), Federated
2026-04-13 · 21 min read #edge-ai#on-device#inference#tensorrt#onnx