Tag: #kafka

← All posts

• Korean Dev Blog Curation 1 — Backend and Infrastructure, 14 Posts I Opened and Checked

  Fourteen Korean-language backend and infrastructure posts selected for being concrete and reproducible. They cover Istio Sidecar and ServiceEntry, Envoy circuit breaking and routing, the limits of the Prometheus Push Gat

  2026-08-12 · 13 min read #curation#큐레이션#backend#infra#istio

• The Generational Shift in Data Stores and Queues — Licenses, Forks and the Apache Attic

  Ten projects in the data layer that gave up their place or had their distribution terms changed, documented using only official announcements and Apache Attic records as evidence. The license changes at Redis, Elasticsea

  2026-08-12 · 9 min read #open-source#database#kafka#redis#elasticsearch

• Kafka Diskless Topics (KIP-1150): The Cross-AZ Cost You Trade for Latency — and a Feature Not Yet Shipped

  On March 2, 2026, the Apache Kafka community accepted KIP-1150, Diskless Topics — a proposal to make object storage, not broker disks, the source of truth for data, eliminating the cross-AZ replication traffic that is th

  2026-07-16 · 16 min read #kafka#streaming#distributed-systems#cloud-native

• The Bug Deterministic Simulation Testing Found — KAFKA-19880, and How to Read "Zero Bugs"

  Deterministic simulation testing (DST) traps every source of nondeterminism — the clock, thread interleaving, random numbers — inside a simulator, so a heisenbug that never used to reproduce can be recalled with a single

  2026-07-16 · 17 min read #testing#deterministic-simulation-testing#distributed-systems#kafka#antithesis

• Message Queues & Async, Explained: Queue, Kafka, RabbitMQ, Redis, asyncio

  From a plain FIFO queue to Kafka partitioned logs, RabbitMQ exchanges and routing, Redis lists, Pub/Sub, and Streams, and finally Python asyncio event loops. A clear, side-by-side intro to the core ideas behind message q

  2026-07-03 · 11 min read #message-queue#kafka#rabbitmq#redis#asyncio

• The Metamorphosis — Kafka's Portrait of the Absurd and the Alienated

  The story of a man who wakes one morning transformed into an insect. Through Kafka and The Metamorphosis we explore modern alienation and the absurd, the meaning of family and labor, and the questions this short but deep

  2026-07-02 · 16 min read #literature#kafka#the-metamorphosis#absurdism#classic-review

• Modern Hadoop & Big Data Ecosystem 2026 Deep Dive - Hadoop 3.4 · Spark 4 · Hive 4 · Kafka 4 · Flink 2 · Iceberg · Trino ♪ Listenable

  In 2026 Hadoop and the broader big-data ecosystem are not dead — they have been relocated. Hadoop 3.4 Erasure Coding and Ozone, Spark 4 ANSI mode and Spark Connect, Hive 4 Iceberg integration, Kafka 4 with KRaft GA and t

  2026-05-16 · 26 min read #english#hadoop#spark#hive#kafka

• Hadoop Ecosystem & Data Engineering 2026 Deep Dive - Hadoop, Spark, Flink, Trino, Iceberg, Delta Lake, Hudi, Airflow, dbt ♪ Listenable

  A complete look at the data engineering stack in May 2026. The truth behind the "Hadoop is dead" headline (HDFS is shrinking, but YARN and the lakehouse pattern survive), the conclusion of the Iceberg vs Delta Lake vs Hu

  2026-05-16 · 25 min read #hadoop#spark#flink#trino#presto

• Distributed Messaging 2026 — Kafka 3.9 / NATS / Redpanda / Pulsar / RabbitMQ 4 / WarpStream Deep-Dive

  The 2026 landscape of distributed messaging. Kafka 3.9 made KRaft the default and ZooKeeper is gone. WarpStream put the Kafka API on top of S3 and was acquired by Confluent. Redpanda pushes Kafka compatibility in C++, NA

  2026-05-15 · 18 min read #messaging#distributed-systems#kafka#nats#redpanda

• Streaming vs Batch Redefined: Flink, RisingWave, Materialize, CDC, Streaming SQL and the Pragmatism of Real Time (2025)

  Season 5 Ep 2. "Everything in real time" was the promise of 2020. 2025 is the era of pragmatism — the line between batch and streaming has blurred, and the real question has become "how fresh does this data have to be to

  2026-04-15 · 13 min read #streaming#batch#flink#risingwave#materialize

• Kafka and Event-Driven Architecture Deep Dive — Partition/ISR, Exactly-Once, Outbox, Schema Registry, Flink

  Many use Kafka; few understand its internals. Partitions and Leader/ISR, Log Segment physical layout, Exactly-Once Semantics truth, Transactional Outbox, Event Sourcing vs CDC, Schema Registry, and 100ms SLA techniques.

  2026-04-15 · 9 min read #kafka#event-driven#streaming#exactly-once#schema-registry

• Data Engineering Complete Guide — Lakehouse, Streaming, dbt, Orchestration, Data Mesh (Season 2 Ep 8, 2025)

  Data engineering is no longer "running ETL scripts." In 2025 a data engineer designs Lakehouse architectures (Iceberg, Delta, Hudi), combines Streaming (Flink, Kafka) with Batch (Spark), standardizes modeling with dbt, r

  2026-04-15 · 10 min read #data-engineering#lakehouse#iceberg#delta-lake#hudi

• CDC (Change Data Capture) Complete Guide 2025: Debezium, Kafka, Data Sync, Outbox Pattern

  Everything about CDC — how it works (WAL/binlog based), Debezium architecture, PostgreSQL Logical Replication, MySQL binlog, Outbox Pattern, data warehouse sync, Kafka integration, migration, common pitfalls.

  2026-04-15 · 11 min read #cdc#debezium#kafka#postgresql#mysql

• Event Streaming Deep Dive Guide 2025: Kafka Internals, Flink, Exactly-Once, Schema Evolution ♪ Listenable

  Event Streaming Deep Dive! Kafka Internals (Partitions/Segments/ISR/Controller), Consumer Groups (Rebalancing/Offset Management), Exactly-Once (Transactions/Idempotency), Schema Registry (Avro/Protobuf Evolution), Apache

  2026-04-14 · 21 min read #event-streaming#kafka#flink#exactly-once#schema-registry

• Data Engineering Fundamentals — ETL, Data Warehouses, Streaming, and Data Lakes

  How do you build data pipelines? From ETL/ELT and data warehouses to streaming, batch processing, Spark, Kafka, and Airflow.

  2026-04-13 · 17 min read #data-engineering-aws#data-engineering#etl#data-warehouse#spark

• Data Engineering Pipeline Complete Guide 2025: ETL/ELT, Spark, Airflow, Real-Time Streaming

  Everything about data engineering! ETL vs ELT, Apache Spark (PySpark), Apache Airflow (DAG/Operator/Sensor), real-time streaming (Kafka+Flink), dbt (data transformation), data warehouses (BigQuery/Snowflake/Redshift), da

  2026-03-25 · 18 min read #data-engineering#etl#elt#spark#airflow

• Event-Driven Architecture Patterns Guide 2025: CQRS, Event Sourcing, Saga Pattern

  Everything about EDA patterns! Domain events, Event Sourcing (event store), CQRS (command/query separation), Saga pattern (Choreography vs Orchestration), Outbox pattern, idempotency, eventual consistency, Kafka/RabbitMQ

  2026-03-24 · 22 min read #event-driven#cqrs#event-sourcing#saga-pattern#microservices

• EAI (Enterprise Application Integration) Complete Guide: Everything About Enterprise System Integration ♪ Listenable

  Everything about EAI in one post! Hub-and-Spoke, ESB, API-Led, event-driven architecture pattern comparison, Gregor Hohpe Enterprise Integration Patterns, MuleSoft/Apache Camel/Dell Boomi platform comparison, and a real-

  2026-03-23 · 23 min read #eai#enterprise-integration#middleware#esb#api-gateway

• WebFlux + Kafka + ZooKeeper Mastery: The Complete Guide to Reactive Streaming Architecture ♪ Listenable

  From Spring WebFlux reactive programming to Kafka streaming, ZooKeeper-to-KRaft migration, and SSE/WebSocket/gRPC Streaming APIs — complete developer guide for real-time data processing architecture with benchmarks, code

  2026-03-22 · 36 min read #webflux#kafka#zookeeper#spring-webflux#reactive-programming

• Message Queue Showdown: Kafka vs RabbitMQ vs SQS vs Pulsar vs NATS — 2025 Decision Guide ♪ Listenable

  Kafka 4.0 KRaft GA, RabbitMQ 4.2 stream filtering, Pulsar 4.1, NATS 2.11 — a complete comparison of 6 message queue systems in 2025. Architecture, benchmarks (Kafka 1M msgs/s vs RabbitMQ 50K), pricing, and Spring Boot in

  2026-03-22 · 23 min read #kafka#rabbitmq#sqs#pulsar#nats