Tag: #llm
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 209 posts
209 posts are tagged #llm; the most recent was published on 2026-09-06.
Often tagged together: #2026-03 69 #ai 65 #rag 27 #inference 24 #ai-agent 23
Most read in the last 90 days:
- LLM Inference Serving Framework Comparison: TensorRT-LLM vs vLLM vs SGLang Production Deployment Strategy
- Complete Guide to LLM Fine-tuning with Unsloth 2025: QLoRA, 4-bit Quantization, 2x Faster Training
- AI Agent Frameworks 2026 Deep Dive — LangGraph, AutoGen, CrewAI, Semantic Kernel, OpenAI Agents SDK, Claude Agent SDK, Pydantic AI
How Reasoning Models Are Built — From Chain-of-Thought to GRPO, in Twelve Papers
o1, R1 and QwQ are not new architectures. They are long output optimised toward a gradeable objective. This traces the lineage through twelve papers — Chain-of-Thought (2022), STaR, PRM, DeepSeek-R1's GRPO, s1's budget f
2026-09-06 · 12 min read #ai#llm#reasoning#reinforcement-learning#grpoAI for Everyone, Part 1 — Training a 16M-Parameter Language Model From Scratch in 15 Minutes
Training a language model from scratch on a single GPU. With the TinyStories dataset and a 16-million-parameter decoder-only transformer, we produced readable English fairy tales in 15 minutes. We look at why the causal
2026-08-19 · 9 min read #ai#llm#transformer#pytorch#hands-onEnglish for AI Engineering Meetings: Saying You Do Not Know Without Losing Credibility ♪ Listenable
The English an AI team meeting needs most is not the language of certainty but the language that makes uncertainty credible. Reporting eval results together with their limits, flagging a regression whose cause is still u
2026-08-16 · 31 min read #english#business-english#meeting#ai#llmGemini 3.7 Flash, Its Introductory Price and Its Three-Week Cadence — Why Model Cost Is a Conditional Value, Not a Fixed One
What a practitioner should take from the Gemini 3.7 Flash announcement is not the size of the benchmark gains but two other things: an introductory price that doubles on a named date, and the fact that the previous model
2026-08-14 · 7 min read #llm#cost-optimization#benchmark#api-design#capacity-planningCerebras Ultrafast and the Bottleneck in an Agent Loop — The Time 750 Tokens a Second Cannot Remove
Cerebras and OpenAI announced an inference tier delivering up to 750 output tokens per second. The mechanism the announcement names is not compute but data movement, achieved by keeping weights in SRAM on the wafer. This
2026-08-14 · 7 min read #llm#inference#hardware#latency#performanceThe Hybrid Attention in Qwen3.8-27B — Only 16 of 64 Layers Grow a KV Cache ♪ Listenable
The reason a 27B model can carry a 262K-token context on a laptop is not its parameter count but its layer layout. Qwen3.8-27B places 48 of its 64 layers as linear attention (Gated DeltaNet) and only 16 as ordinary atten
2026-08-14 · 8 min read #llm#inference#quantization#local-llm#attentionThe Technique of Hallucinating Instead of Classifying, and How to Validate It — Why a Fake Label Beats the Raw Query
Instead of putting a taxonomy of hundreds of entries into a prompt, have a small model invent a plausible fake classification and map it onto the real taxonomy with embeddings. Why that can work is explained by asymmetry
2026-08-14 · 7 min read #llm#embedding#classification#search#retrievalGrowing into a Harness Engineer — Why the Job Exists and What to Practice
The title harness engineer is still rare in job postings, but the work already exists in every team shipping agents. The final part 8 of the harness engineering series covers why this job emerged, how existing software s
2026-08-12 · 4 min read #llm#agent#harness-engineering#하네스엔지니어링#AI에이전트Loop Design — Between Infinite Loops and Giving Up Early
Agent loops fail in two directions: the infinite loop that repeats the same call dozens of times, and the early stop that quits at the first obstacle. Part 4 of the harness engineering series covers retry caps, the three
2026-08-12 · 5 min read #llm#agent#harness-engineering#하네스엔지니어링#AI에이전트The Evaluator Bottleneck — A Weak Grader Caps the Whole System
If the score does not move no matter how much you fix the harness, the bottleneck may be the evaluator, not the harness. You cannot select for a quality you cannot measure, which is why a weak grader becomes the ceiling
2026-08-12 · 5 min read #llm#agent#harness-engineering#하네스엔지니어링#AI에이전트Reward Hacking — The Metric Rises While the Task Fails
If the cheapest way for an agent to pass the tests is to edit the tests, the agent will edit the tests. Reward hacking is not a bug; it is the exact optimization of the goal we wrote down. Part 6 of the harness engineeri
2026-08-12 · 5 min read #llm#agent#harness-engineering#하네스엔지니어링#AI에이전트Open Source Worth Watching Right Now (1) AI Agents and LLM Tooling
The LLM application stack has split into layers: inference servers, orchestration, gateways, agents, and RAG. This post introduces 12 open source projects that are actually used at each layer, grouped by role rather than
2026-08-12 · 6 min read #open-source#llm#ai-agent#ai-platform#ragTool Surface Design — One Schema Line Moves the Success Rate
Adding more tools and watching the agent success rate drop is not rare. The tool surface is the agent interface, and the names, descriptions, parameters, failure returns, and response sizes are all design material. Part
2026-08-12 · 5 min read #llm#agent#harness-engineering#하네스엔지니어링#AI에이전트The Context Budget — Design Is What You Leave Out, Not What You Put In
The context window still has room, yet agent accuracy is dropping. Context is a finite attention budget, and tool schemas spend it too. Part 2 of the harness engineering series covers turning prompt accumulation into a p
2026-08-12 · 6 min read #llm#agent#harness-engineering#하네스엔지니어링#AI에이전트Harness Fingerprints and Versioning — Making Unrecorded Changes Traceable
The success rate moved with no prompt commit and no model change — so what do you roll back? Part 7 of the harness engineering series covers the harness fingerprint: a single normalized hash summarizing every decision th
2026-08-12 · 5 min read #llm#agent#harness-engineering#하네스엔지니어링#AI에이전트What Is Harness Engineering — The Model Is a Fixed Input; What You Ship Is Everything Around It
Two teams use the same model, so why do their agents perform so differently? For most teams the model is a fixed input, and what actually ships is the harness around it: the tool surface, the failure return format, the l
2026-08-12 · 6 min read #llm#agent#harness-engineering#하네스엔지니어링#AI에이전트Korean Dev Blog Curation 3 — AI and ML in Practice, 14 Posts I Opened and Checked
Fourteen Korean-language posts on applying AI and ML in production, selected for being concrete and reproducible. They cover an end-to-end LangChain RAG pipeline, how semantic search actually works through embeddings and
2026-08-12 · 12 min read #curation#큐레이션#ai#llm#ragChoosing Open Text Generation Models by Size Class
When picking an open text generation model, size is not a quality tier — it is a deployment constraint. This post splits the field into on-device small, single-GPU mid, and server-class large, listing the parameters, con
2026-08-12 · 7 min read #ai#llm#huggingface#open-source-llm#text-generationOpen Models That Support Korean, and the Cost of Tokenization
A line saying a model supports Korean means it processes the language, not that it is good at it. This post lays out the card values read on 2026-08-12 for Korean-focused, bilingual, and multilingual open models, explain
2026-08-12 · 6 min read #ai#llm#huggingface#open-source-llm#korean-nlpLicenses and Deployment: Reading the Types and Verifying Quantized Builds
The short identifier in an open model license field is a table of contents; the actual terms live in the full text. This post organizes the license types actually encountered on 2026-08-12, what community licenses and no
2026-08-12 · 7 min read #ai#llm#huggingface#open-source-llm#license