Tag: #distributed-systems
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 41 posts
41 posts are tagged #distributed-systems; the most recent was published on 2026-08-09.
Often tagged together: #2026-03 9 #etcd 8 #consensus 7 #kafka 7 #raft 7
Most read in the last 90 days:
When Two Services That Passed the Type Checker Halt Waiting for Each Other — Choreography as a Different Approach
Using a language that guarantees memory safety does not stop two services from halting while each waits for a message from the other, because the field of view of a type checker ends at one process. Choreographic program
2026-08-09 · 8 min read #programming-languages#compiler#distributed-systems#type-systems#concurrencyTemporal Worker Versioning GA — The Deploy Problem Replay Created, and the Third Answer After Two Discarded Generations
Durable execution engines like Temporal survive failures by replaying an event history — and it is precisely that replay which makes shipping new code while workflows are still running the hardest problem in this model.
2026-07-17 · 13 min read #distributed-systems#durable-execution#temporal#workflow-engine#reliabilityLabel Locality Scheduling in Ray 2.56 — Placement Groups Start Seeing NVLink Racks, Not Nodes
Ray 2.56.0, released on June 29, 2026, adds a domain-level scheduling layer to placement groups as an alpha feature. Until now every placement strategy — PACK, STRICTPACK and friends — operated strictly at node granulari
2026-07-17 · 11 min read #ray#gpu#scheduling#distributed-systemsNATS Server 2.14 — Fast Batch Publish, Built-In Server Scheduling, and the Homework Due Before 2.15
NATS Server 2.14, released on April 30, 2026, adds fast batch publish — high-throughput batch publishing to JetStream with built-in flow control — cron-style recurring schedules with subject sampling, automatic conversio
2026-07-17 · 14 min read #messaging#nats#distributed-systems#release-notesCassandra 6.0-alpha1 and Accord Transactions — Where Does the 5-Year "General-Purpose Transaction" Promise Stand Now
Accord, the general-purpose transaction protocol for Cassandra proposed as CEP-15 in 2021, finally shipped in a runnable release, 6.0-alpha1, in March-April 2026. The protocol promises to handle multi-partition strict-se
2026-07-17 · 13 min read #database#cassandra#distributed-systems#transactionsKafka Diskless Topics (KIP-1150): The Cross-AZ Cost You Trade for Latency — and a Feature Not Yet Shipped
On March 2, 2026, the Apache Kafka community accepted KIP-1150, Diskless Topics — a proposal to make object storage, not broker disks, the source of truth for data, eliminating the cross-AZ replication traffic that is th
2026-07-16 · 16 min read #kafka#streaming#distributed-systems#cloud-nativeThe Bug Deterministic Simulation Testing Found — KAFKA-19880, and How to Read "Zero Bugs"
Deterministic simulation testing (DST) traps every source of nondeterminism — the clock, thread interleaving, random numbers — inside a simulator, so a heisenbug that never used to reproduce can be recalled with a single
2026-07-16 · 17 min read #testing#deterministic-simulation-testing#distributed-systems#kafka#antithesisWhy 3 of 5 Raft Nodes Died at Once — Reading Coinbase's May 7, 2026 Outage
On the evening of May 7, 2026, cooling units in a single data hall inside AWS us-east-1's availability zone use1-az4 failed simultaneously, and Coinbase suffered an outage in which most of its services — including tradin
2026-07-16 · 17 min read #distributed-systems#postmortem#consensus#aws#resilienceNo Leap Second in 2026 — and the Negative One No System Has Ever Run
On 6 July 2026 the IERS issued Bulletin C 72: no leap second at the end of December 2026. It reads like quiet news, but since Earth started spinning faster around 2016, the first-ever negative leap second is now on the t
2026-07-11 · 6 min read #leap-second#utc#ntp#distributed-systems#posix-timeDistributed Transactions: 2PC vs the Saga Pattern
Why the ACID that was easy inside one database falls apart across services, how two-phase commit works and where it breaks (coordinator, blocking, failure modes), the Saga pattern (choreography vs orchestration, compensa
2026-06-29 · 12 min read #distributed-systems#transactions#sagaThe Truth About Exactly-Once Semantics
What the three delivery guarantees (at-most-once, at-least-once, exactly-once) actually mean, why exactly-once delivery is fundamentally impossible while exactly-once processing is achievable through idempotency, dedupli
2026-06-28 · 13 min read #distributed-systems#messaging#reliabilityWhat Distributed Systems Taught Me About Relationships
Retries with backoff, timeouts, heartbeats, consistency models, backpressure, graceful degradation, idempotency, the Two Generals Problem, and trust as a cache with a TTL. The same problems that make distributed systems
2026-06-25 · 13 min read #engineering#relationships#fun#distributed-systemsThe CAP Theorem Without the Hand-Waving
The popular "pick two of three" summary is wrong. This post pins down exactly what each of the three letters means, why in practice you only choose between C and A when a partition happens, why PACELC is the more complet
2026-06-23 · 11 min read #distributed-systems#databases#theoryIdempotency and Retries: Building Reliable APIs ♪ Listenable
The network will eventually fail, and when it does you have to retry. The hard case is the request that was "already processed but whose response got lost." This post covers what idempotency is, safe versus unsafe HTTP m
2026-06-21 · 12 min read #api#reliability#distributed-systemsDistributed Messaging 2026 — Kafka 3.9 / NATS / Redpanda / Pulsar / RabbitMQ 4 / WarpStream Deep-Dive
The 2026 landscape of distributed messaging. Kafka 3.9 made KRaft the default and ZooKeeper is gone. WarpStream put the Kafka API on top of S3 and was acquired by Confluent. Redpanda pushes Kafka compatibility in C++, NA
2026-05-15 · 18 min read #messaging#distributed-systems#kafka#nats#redpandaRaft Consensus Complete Guide 2025: Leader Election, Log Replication, Safety, etcd/Consul in Practice
etcd, Consul, CockroachDB, and TiDB all use Raft because it is understandable. Leader Election, Log Replication, Safety Properties, Membership Change, and Snapshots — a deep dive with real-world system analysis.
2026-04-15 · 10 min read #raft#consensus#distributed-systems#leader-election#log-replicationDistributed Systems Complete Guide — Clocks, Consensus, Event Sourcing, Saga, CRDT, Failure Patterns (Season 2 Ep 12, 2025)
A distributed system looks like "several computers cooperating," but what it really is: "several computers whose clocks disagree, some of them broken, reaching agreement across a network that loses and delays messages."
2026-04-15 · 13 min read #distributed-systems#consensus#raft#paxos#lamport-clockDistributed Consensus Deep Dive — Paxos, Raft, ZAB, FLP, etcd, ZooKeeper, KRaft, BFT, CRDT (2025)
Why is it hard for a distributed system to merely "agree"? The shock of the FLP impossibility result, Lamport's Paxos and its infamous unintelligibility, how Raft changed the landscape, ZooKeeper's ZAB, the birth of KRaf
2026-04-15 · 15 min read #distributed-systems#consensus#paxos#raft#etcdCRDT Complete Guide 2025: Conflict-Free Replicated Data Types, Local-First Collaboration, Yjs/Automerge
Everything about CRDTs! Conflict-free Replicated Data Types principles, State-based vs Operation-based, basic CRDTs (G-Counter/PN-Counter/G-Set/2P-Set/LWW-Register), advanced (Sequence/RGA/Yjs), local-first software, Aut
2026-04-15 · 12 min read #crdt#distributed-systems#local-first#collaboration#yjsThe Essential Difficulty of Distributed Systems — CAP, PACELC, Raft/Paxos, Vector Clock, Saga, Event Sourcing (2025)
CAP is not 'pick CP or AP'. PACELC, Raft leader election, Paxos's infamy, Lamport/Vector Clock, limits of 2PC and the rise of Saga, Event Sourcing/CQRS, Outbox Pattern, truth about Exactly-Once, FLP Impossibility, Google
2026-04-15 · 13 min read #distributed-systems#cap-theorem#pacelc#raft#paxos