Tag: #dagster
Writing on GPUs, LLMs, MLOps, Kubernetes — and mindset · 7 posts
7 posts are tagged #dagster; the most recent was published on 2026-05-25.
Often tagged together: #airflow 7 #dbt 4 #prefect 4 #data-engineering 3 #deep-dive 3
Most read in the last 90 days:
- Durable Execution & Workflow Engines 2026 — Temporal, Inngest, Trigger.dev, Hatchet, Restate, DBOS, Step Functions Deep Dive (Infrastructure for the Long-Running AI Agent Era)
- Workflow Engines in 2026 — Temporal, Inngest, Trigger.dev, Hatchet, Restate, DBOS, Airflow Deep Dive (Mapping the Durable Execution Era)
- Hadoop Ecosystem & Data Engineering 2026 Deep Dive - Hadoop, Spark, Flink, Trino, Iceberg, Delta Lake, Hudi, Airflow, dbt
Durable Execution & Workflow Engines 2026 — Temporal, Inngest, Trigger.dev, Hatchet, Restate, DBOS, Step Functions Deep Dive (Infrastructure for the Long-Running AI Agent Era) ♪ Listenable
In 2026, workflow engines sit on top of two massive waves — first, long-running AI agents (LLM jobs that run for minutes to days) have effectively mandated durable execution; second, Postgres is becoming the backbone of
2026-05-25 · 32 min read #durable-execution#workflow-engine#temporal#inngest#trigger-devHadoop Ecosystem & Data Engineering 2026 Deep Dive - Hadoop, Spark, Flink, Trino, Iceberg, Delta Lake, Hudi, Airflow, dbt ♪ Listenable
A complete look at the data engineering stack in May 2026. The truth behind the "Hadoop is dead" headline (HDFS is shrinking, but YARN and the lakehouse pattern survive), the conclusion of the Iceberg vs Delta Lake vs Hu
2026-05-16 · 25 min read #hadoop#spark#flink#trino#prestoCron & Scheduled Tasks in 2026 — systemd timers / Vercel Cron / AWS EventBridge Scheduler / k8s CronJobs / BullMQ / Sidekiq / Temporal Deep Dive ♪ Listenable
The 2026 landscape of scheduled tasks. Classic cron is still alive, and systemd timers became the modern Linux replacement. Cloud-side: Vercel Cron, AWS EventBridge Scheduler (replaced CloudWatch Events Rules in Nov 2022
2026-05-16 · 25 min read #cron#scheduled-tasks#systemd-timers#vercel-cron#aws-eventbridge-schedulerWorkflow Engines in 2026 — Temporal, Inngest, Trigger.dev, Hatchet, Restate, DBOS, Airflow Deep Dive (Mapping the Durable Execution Era) ♪ Listenable
The AI-agent era has dragged workflow engines back into tier-1 infrastructure for every backend team. Temporal Cloud — founded by the people who built Uber Cadence — is the de facto standard, Inngest and Trigger.dev v3 a
2026-05-15 · 24 min read #workflow#durable-execution#temporal#inngest#trigger-devData Engineering Complete Guide — Lakehouse, Streaming, dbt, Orchestration, Data Mesh (Season 2 Ep 8, 2025)
Data engineering is no longer "running ETL scripts." In 2025 a data engineer designs Lakehouse architectures (Iceberg, Delta, Hudi), combines Streaming (Flink, Kafka) with Batch (Spark), standardizes modeling with dbt, r
2026-04-15 · 10 min read #data-engineering#lakehouse#iceberg#delta-lake#hudiData Orchestration 2025 Complete Guide: dbt, SQLMesh, Dagster, Airflow, Prefect, Data Contracts, CI/CD (2025)
Season 5 Ep 4. If engines query, orchestrators govern the pipelines. The standardization of dbt and its limits, the revolt of SQLMesh, the asset-centric model of Dagster, the evolution of Airflow 2.x and 3.0, the rebirth
2026-04-15 · 13 min read #data-orchestration#dbt#sqlmesh#dagster#airflowData Engineering Complete Guide 2025: Flink vs Spark, dbt, Iceberg, Airflow — Modern Data Stack Deep Dive ♪ Listenable
Flink vs Spark stream processing comparison, dbt crossing $100M ARR, Iceberg vs Delta Lake vs Hudi lakehouse showdown, ClickHouse vs StarRocks real-time analytics, Airflow 3.0 vs Dagster vs Prefect orchestration — the co
2026-03-22 · 30 min read #data-engineering#apache-flink#apache-spark#dbt#apache-iceberg