Tag: #ai-papers
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 79 posts
79 posts are tagged #ai-papers; the most recent was published on 2026-08-12.
Often tagged together: #2026-03 23 #transformer 15 #deep-learning 14 #robotics 12 #model-internals 8
Most read in the last 90 days:
Video Generation and Understanding Technical Reports: What to Read, and Why Constraints Beat Demos
Ten video generation and understanding technical reports, each verified by opening the arXiv abstract page directly. CogVideoX, Movie Gen, HunyuanVideo, LTX-Video, Wan, Seedance 1.0 and 2.0, plus Qwen2.5-VL, VideoLLaMA 3
2026-08-12 · 7 min read #ai-papers#paper-review#technical-report#video-generation#diffusion-transformerText LLM Technical Reports: What to Read, and How to Read Design Decisions Instead of Rankings
Nine text-LLM technical reports, each verified by opening the arXiv abstract page directly. What DeepSeek-V3, DeepSeek-R1, Qwen3, Gemma 3, Olmo 3, Kimi K2, MiniMax-01, Mellum2 and s1 each did that was new, and which limi
2026-08-12 · 7 min read #ai-papers#paper-review#technical-report#llm#moeThe Training Recipe — From Pre-training to Post-training, and What Reports Write Down
Comparing, exactly as written in the reports, the three stages and six context extensions of Llama 3, the three-stage pre-training of Qwen3, the learning-rate schedule and two-phase YaRN extension of DeepSeek-V3, and the
2026-08-12 · 6 min read #ai-papers#model-internals#pretraining#training-recipe#data-mixturePositional Encoding — RoPE and the Price of Context Extension
Showing through wavelength arithmetic how a single ropetheta value acts on context length, then comparing the 500000 of Llama 3, the ABF step in Qwen3, the YaRN settings of DeepSeek-V3 and Kimi K2, and the partial rotati
2026-08-12 · 6 min read #ai-papers#model-internals#rope#positional-encoding#long-contextAnatomy of config.json — Reading a Model From One Settings File
How every field in config.json — hiddensize, numhiddenlayers, numattentionheads versus numkeyvalueheads, headdim, intermediatesize, ropetheta, vocabsize, tiewordembeddings — shows up in memory and speed, and a hand count
2026-08-12 · 6 min read #ai-papers#model-internals#config-json#transformer#llm-architectureOCR and Document Understanding Technical Reports: What to Read, and Why Parsing Is Not Finished
Ten OCR and document understanding technical reports, each verified by opening the arXiv abstract page directly. From Donut and Nougat through GOT-OCR2.0, olmOCR, DeepSeek-OCR and its successor, GLM-OCR, Qianfan-OCR, Min
2026-08-12 · 7 min read #ai-papers#paper-review#technical-report#ocr#document-aiHow to Read Leaderboards and Benchmarks: Why SOTA Has Such a Short Shelf Life
Twelve benchmark methodology papers, each verified by opening the arXiv abstract page directly, assembled into a guide for reading leaderboard numbers. Data contamination, prompt format sensitivity, eval harness differen
2026-08-12 · 7 min read #ai-papers#paper-review#technical-report#benchmark#evaluationReading a Technical Report Critically — What Gets Written Down and What Goes Missing
How to separate the verifiable from the unverifiable in a model technical report. Covers the limits of self-reported benchmarks, the places where a config and a report disagree, values that could not be read from gated r
2026-08-12 · 7 min read #ai-papers#model-internals#tech-report#benchmarks#evaluationMoE Routing — How an Expert Gets Picked
Reading the config fields of a mixture-of-experts layer against real models. Comparing Mixtral 2-of-8, Qwen3 8-of-128, the 256 routed experts plus a shared expert in DeepSeek-V3, and the sparsity of 48 in Kimi K2, then w
2026-08-12 · 6 min read #ai-papers#model-internals#mixture-of-experts#moe#routingSpeech Recognition and Synthesis Technical Reports: What to Read, and What a Single WER Hides
Ten speech recognition and synthesis technical reports, each verified by opening the arXiv abstract page directly. Whisper, Omnilingual ASR, Qwen3-ASR, the Open ASR Leaderboard, Seed-TTS and F5-TTS, CosyVoice 2, Qwen3-TT
2026-08-12 · 7 min read #ai-papers#paper-review#technical-report#speech-recognition#text-to-speechImage Generation and Understanding Technical Reports: What to Read, and Why Design Beats Sample Images
Eleven image generation and understanding technical reports, each verified by opening the arXiv abstract page directly. Rectified flow transformers, VAR, Emu3, SANA, Janus-Pro, FLUX.1 Kontext, the Qwen-Image line, Seedre
2026-08-12 · 8 min read #ai-papers#paper-review#technical-report#image-generation#diffusion-transformerInside the Tokenizer — Why Korean Costs More Tokens, and What That Costs
How byte-level BPE works, then downloading the actual tokenizer files of Qwen3, DeepSeek-V3, and Mixtral to tokenize the same English and Korean text and compare. Covers the vocabulary-size tradeoff, why the config vocab
2026-08-12 · 6 min read #ai-papers#model-internals#tokenizer#bpe#korean-nlpNormalization and Activation — Keeping Training From Falling Apart
Confirming from config values why RMSNorm, pre-norm, and SwiGLU became the defaults, then walking through the newer devices that stop attention logits from exploding — the QK-Norm of Qwen3 and the QK-Clip of Kimi K2 — wi
2026-08-12 · 6 min read #ai-papers#model-internals#rmsnorm#swiglu#training-stabilityAttention Variants — From MHA to MLA, and How the KV Cache Shrinks
Comparing MHA, MQA, GQA, and MLA using real config values. Working out with formulas and numbers how the grouped-query attention in Qwen3, Mixtral, and GLM-4.5 and the latent attention in DeepSeek-V3 and Kimi K2 reduce p
2026-08-12 · 6 min read #ai-papers#model-internals#attention#gqa#mlaAnalyzing SOTA Segmentation and Detection Models — The Lineage of SAM, DETR, and YOLO ♪ Listenable
A walk through the SOTA lineage of object detection and image segmentation. We trace detection from two-stage R-CNN to one-stage YOLO/SSD and transformer-based DETR, cover semantic, instance, and panoptic segmentation, a
2026-06-30 · 20 min read #ai-papers#computer-vision#object-detection#segmentation#samSOTA Text Embedding Models — The Heart of Search and RAG
Text embeddings are the heart of search and RAG systems. We walk through contrastive learning and InfoNCE, dual encoders, hard negatives, the E5/BGE/GTE families, Matryoshka representation learning, and the MTEB benchmar
2026-06-30 · 14 min read #ai-papers#embedding#retrieval#rag#contrastive-learningAnalyzing SOTA 3D Vision — Monocular Depth and 3D Gaussian Splatting ♪ Listenable
A survey of SOTA trends in 3D vision. We cover monocular depth estimation (relative and metric depth) and the Depth Anything family concept, an overview of stereo and MVS, and the evolution of real-time rendering from Ne
2026-06-30 · 22 min read #ai-papers#3d-vision#monocular-depth#gaussian-splatting#nerfSOTA Music and Audio Generation — Neural Codecs and Generative Models
A lineage-focused overview from audio representations (waveform, spectrogram, neural codec) to autoregressive audio language models, diffusion-based audio, and text-to-music conditioning. We analyze the principles of the
2026-06-30 · 9 min read #ai-papers#audio-generation#music-generation#neural-codec#audio-language-modelSOTA Video Generation Models Explained — Spatiotemporal Diffusion Transformers
A look at the two fundamental challenges of video generation, temporal consistency and compute cost, through the lens of spatiotemporal latent patches and diffusion transformers. We analyze the concepts Sora introduced,
2026-06-30 · 8 min read #ai-papers#video-generation#diffusion-transformer#spatiotemporal#text-to-videoAnalyzing SOTA Image Generation Models — From Diffusion to FLUX
A lineage-centered overview of the frontier of text-to-image generation, from diffusion model fundamentals through latent diffusion, DiT, rectified flow, and the FLUX family. We analyze the shared structure and differenc
2026-06-30 · 10 min read #ai-papers#diffusion-models#text-to-image#latent-diffusion#rectified-flow