---
title: Airflow Interview Questions and Answers (2026)
description: Real Apache Airflow interview questions — scheduling and logical_date, idempotency, backfills, XCom, executors and sensors — with code and a four-week prep plan.
url: https://usegreenroom.app/blog/airflow-interview-questions
last_updated: 2026-07-31
---
← Back to blog
Data Engineering
# Airflow interview questions and answers
July 31, 2026 · 12 min read
"The DAG is scheduled daily at midnight," the candidate explained. "So the run on the 15th processes the 15th." The interviewer asked, gently, "Are you sure?" And you could watch the candidate's face do the thing every Airflow engineer's face has done exactly once, usually while a stakeholder waits for a report that is a day behind.
That single confusion — which date a run actually covers — is the most reliable filter in Airflow interview questions, because you only get it wrong once in production. This guide covers the scheduling model, idempotency and backfills, XCom, executors and the failure modes interviewers ask about, with the follow-up sitting behind each.
## What Airflow interviews actually test
Apache Airflow interview questions are rarely about syntax — the API is easy to look up. They test whether you have operated a scheduler:
- The scheduling model — data intervals, logical_date, catchup, and why a run fires at the end of its interval.
- Idempotency and backfills — the single most important design property, and the one most candidates cannot articulate.
- Task communication — XCom, what it is for, and what it is emphatically not for.
- Executors and scaling — Sequential, Local, Celery, Kubernetes, and when you would move between them.
- Failure modes — retries, sensors and deadlock, pools, SLAs, and what you do when a DAG silently stops running.
The five bands of Airflow interview questions, with the scheduling model filtering candidates fastest.
## The scheduling model: logical_date, data intervals and catchup
This is the question from the intro, and it deserves the most preparation. Airflow schedules over data intervals. A DAG with schedule="@daily" and a run whose logical date is the 15th covers the interval from the 15th to the 16th — and it therefore fires after that interval closes, at midnight starting the 16th.
So the run labelled the 15th processes the 15th's data, but executes on the 16th. Both halves of that sentence matter, and candidates typically have one of them. In modern Airflow the naming makes it clearer: logical_date (formerly the badly-named execution_date), plus data_interval_start and data_interval_end, which are what you should actually use in your queries.
```
from airflow.decorators import dag, task
import pendulum
@dag(
schedule="@daily",
start_date=pendulum.datetime(2026, 7, 1, tz="UTC"),
catchup=False,
max_active_runs=1,
)
def daily_revenue():
@task
def load_window(data_interval_start=None, data_interval_end=None):
# Bounded by the interval — NOT by now(), which is not reproducible
sql = f"""
DELETE FROM gold.revenue WHERE day = '{data_interval_start.date()}';
INSERT INTO gold.revenue
SELECT ... FROM silver.orders
WHERE order_ts >= '{data_interval_start}'
AND order_ts < '{data_interval_end}';
"""
return run_sql(sql)
load_window()
daily_revenue()
```
Then catchup. With catchup=True (the historical default), deploying a DAG with a start_date six months ago immediately schedules every missed interval — which is either exactly what you wanted or an accidental denial-of-service on your warehouse. Saying "I set catchup=False by default and backfill deliberately when I need to" is the answer of someone who has been burned. Pair it with max_active_runs=1 for pipelines that must not overlap.
## Idempotency and backfills
The most important concept in the entire interview, and it is worth stating plainly: a task must produce the same result whether it runs once or five times. Airflow will retry your task. It will rerun it during a backfill. Somebody will clear it manually at 3am.
What that means in practice, and what interviewers want to hear:
- Delete-then-insert, or MERGE, scoped to the interval — never a bare INSERT, which duplicates rows on a rerun. The SQL above does exactly this.
- Never use now() or today() inside a task — it makes the output depend on wall-clock time, so a rerun of last Tuesday produces today's data. This is the single most common idempotency bug.
- Write to a deterministic location — a path keyed on the interval, not on a timestamp generated at runtime.
- Make the task fail loudly rather than half-succeed — a partially written output that looks complete is worse than a failure.
The natural follow-up is backfills: airflow dags backfill with a date range, or clearing task instances in the UI to rerun them. The senior answer notes the operational caution — backfilling six months of a DAG that hits a rate-limited API will get you throttled, so bound concurrency with pools or max_active_runs first. The data engineer interview questions guide covers the pipeline design context.
## XCom, TaskFlow and passing data between tasks
XCom (cross-communication) lets tasks push and pull small values through the metadata database. The TaskFlow API makes this implicit — returning a value from a @task function pushes an XCom, and passing it to another task pulls it.
The question is always really about the limit. XComs are stored in the Airflow metadata database, so they are for small values: a filename, a row count, a partition key, a model version. Pushing a DataFrame through XCom bloats the metadata DB and eventually destabilises the scheduler. The correct pattern for real data is to write it to S3, GCS or a table, and pass the path through XCom. Mentioning custom XCom backends — which transparently store large values in object storage — is a strong bonus.
Related and frequently asked: how do you share state when tasks run on different workers? You do not, beyond XCom — Airflow tasks are independent processes with no shared memory, and any assumption that a variable set in one task is visible in another is a red flag.
## Executors, sensors and scaling
Expect a straight comparison question. The Airflow executors, in the order you would grow through them:
- SequentialExecutor — one task at a time, SQLite backing. Local debugging only, never production.
- LocalExecutor — parallel subprocesses on a single machine. Genuinely fine for small teams; the ceiling is one host.
- CeleryExecutor — a pool of persistent workers behind a broker (Redis or RabbitMQ). Scales horizontally, workers stay warm, but you operate the broker and the workers.
- KubernetesExecutor — one pod per task, isolated dependencies and per-task resources, scales to zero. The cost is pod startup latency, which hurts on many short tasks.
On sensors, the trap question is deadlock: classic sensors in poke mode occupy a worker slot for their entire wait, so a few dozen sensors waiting on upstream files can consume every slot and block the tasks that would satisfy them. The fixes are mode="reschedule", which frees the slot between checks, and deferrable operators/triggers, which hand the wait to the async triggerer process. Naming deferrable operators reads as current.
Also know pools (cap concurrency against a fragile external system), depends_on_past (a run waits for the previous run's same task to succeed — powerful and a common cause of a silently stuck DAG), and trigger rules such as all_success, all_done and one_failed for cleanup and alerting branches. Our Kubernetes interview questions guide covers the layer under the KubernetesExecutor.
## The debugging question: a DAG stopped running
Mid and senior loops usually end open-ended: "a DAG hasn't produced output for two days and nobody got an alert. What do you check?" The funnel answer:
- Is the DAG paused? Unglamorous and startlingly often the answer.
- Is the scheduler healthy? A dead or thrashing scheduler stops everything; check its logs and heartbeat.
- Is a run stuck in a queued state? That points at slots — pool exhausted, max_active_runs reached, or every worker occupied by poke-mode sensors.
- Is depends_on_past blocking it? One failed historical run halts the whole chain silently.
- Did a DAG-file import error occur? A syntax error or missing import makes the DAG vanish from the UI rather than fail loudly — check the import errors panel.
- Why was there no alert? The real question underneath. Answer with SLAs, on_failure_callback, and monitoring freshness of the output table rather than only the task state — because a task that succeeds while writing zero rows is the failure nobody catches.
That last point is the one that separates candidates. Task-level success is not data correctness.
## Prefect, Dagster, cron, ChatGPT — where each fits
- The official Airflow documentation — the concepts pages on DAG runs, data intervals and deferrable operators answer most interview questions directly and are more current than courses.
- Astronomer's guides — well-written, practical, and free; particularly good on scheduling semantics.
- Prefect and Dagster — worth being able to compare honestly, because interviewers ask. Dagster is asset-oriented and has stronger data-lineage and testing ergonomics; Prefect is lighter and more Pythonic on dynamic workflows; Airflow has the largest ecosystem and operator library and is what most enterprises already run. "It depends on what the team already operates" is a legitimate answer if you can defend it.
- Plain cron — the honest baseline. If you have five independent scripts, cron is fine and Airflow is overhead; Airflow earns its cost with dependencies, retries, backfills and visibility.
- Running a local instance — highest yield on this list. Run Airflow in Docker, deploy a daily DAG, watch the first run fire a day later than you expected, then backfill it. That experience answers half the questions.
- ChatGPT — good for generating DAG code and explaining operators. It will not ask "are you sure?" when you say the run on the 15th processes the 15th.
- Greenroom — the spoken layer. Ari, the AI interviewer, runs the round out loud and pushes the follow-up when your idempotency answer is a definition rather than a design. Fair tradeoff: Ari will not debug your scheduler logs.
> The core truth: Airflow interviews test whether you have been on call for a pipeline. Scheduling semantics and idempotency are the two answers that reveal it instantly — and both are far easier to say convincingly after you have run a DAG locally and watched it behave unexpectedly.
## How to prepare for an Airflow interview
- Week 1: run Airflow locally in Docker. Deploy a daily DAG, observe when the first run actually fires, then clear a task and rerun it. Say the data-interval explanation out loud until it is boring.
- Week 2: idempotency drills — take three naive tasks that use now() or bare inserts and rewrite them to be safely rerunnable. Explain each rewrite in ninety seconds.
- Week 3: executors, sensors, pools and trigger rules. Deliberately cause a sensor deadlock locally, then fix it with reschedule mode.
- Final week: rehearse the "DAG stopped running" funnel aloud, prepare two on-call war stories in first person, and run two full spoken mocks with follow-ups.
Building the rest of the stack? The PySpark interview questions guide covers the processing engine, the Databricks interview questions guide covers Delta Lake and the lakehouse, the dbt interview questions guide covers transformation in the warehouse, and the data modeling interview questions guide covers the schema design underneath.
## Frequently asked questions
### What are the most common Airflow interview questions?
The most common questions cover how Airflow schedules over data intervals and why a daily run fires at the end of its interval, what logical_date means, idempotency and how to write safely rerunnable tasks, catchup and backfills, what XCom should and should not carry, the differences between the Local, Celery and Kubernetes executors, sensor deadlock and reschedule mode, and how you would debug a DAG that silently stopped producing output.
### What is the difference between logical_date and the actual run time in Airflow?
Airflow schedules over data intervals, so a run covers a window of time and executes after that window closes. A daily DAG run with a logical_date of the 15th covers the data interval from the 15th to the 16th and therefore actually executes at the start of the 16th. In queries you should use data_interval_start and data_interval_end rather than logical_date directly, because they express the window explicitly.
### How do you make an Airflow task idempotent?
Scope every write to the run's data interval and make it replace rather than append — a delete-then-insert or a MERGE bounded by data_interval_start and data_interval_end. Never call now() or today() inside a task, because that makes output depend on wall-clock time and breaks reruns. Write to a deterministic location keyed on the interval, and fail loudly rather than leaving a partially written output that looks complete.
### What should you not store in XCom?
XComs live in the Airflow metadata database, so they are only for small values such as a filename, a row count, a partition key or a model version. Never push a DataFrame or a large result set through XCom, because it bloats the metadata database and can destabilise the scheduler. Write the data to object storage or a table and pass the path through XCom instead, or configure a custom XCom backend.
### What is the difference between the Celery and Kubernetes executors?
CeleryExecutor runs a pool of persistent workers behind a broker such as Redis or RabbitMQ, so tasks start quickly on warm workers, but you operate the broker and the worker fleet. KubernetesExecutor launches one pod per task, giving isolated dependencies, per-task resource limits and the ability to scale to zero, at the cost of pod startup latency that hurts when you have many short tasks.
### Is Airflow still worth learning in 2026 compared to Dagster or Prefect?
Yes. Airflow has the largest operator ecosystem and remains what most enterprises already run, so it dominates job listings. Dagster is asset-oriented with stronger lineage and testing ergonomics, and Prefect is lighter for dynamic Python workflows, so both are worth being able to compare honestly in an interview. The scheduling, idempotency and backfill concepts transfer between all three.
Airflow rounds reward the engineer who has been paged, and can explain why. Greenroom runs mock data engineering interviews out loud with Ari — idempotency and on-call follow-ups included — and scores structure, clarity and depth. Free to start. Curious how it works? See how AI mock interviews work.