How to run one airflow task and all its dependencies?
11,493
You can run a task independently by using -i/-I/-A flags along with the run command.
But yes the design of airflow does not permit running a specific task and all its dependencies.
You can backfill the dag by removing non-related tasks from the DAG for testing purpose
Author by
itzjustricky
Updated on June 12, 2022Comments
-
itzjustricky almost 2 years
I suspected that
airflow run dag_id task_id execution_date
would run all upstream tasks, but it does not. It will simply fail when it sees that not all dependent tasks are run. How can I run a specific task and all its dependencies? I am guessing this is not possible because of an airflow design decision, but is there a way to get around this?
-
Hailiang Zhang about 7 yearsThen what is the best alternative to airflow to achieve the goal (run a task and its dependencies from a DAG)?
-
ismail almost 5 yearsAzkaban has this feature. in the execute flow popup you can disable any job or disable its upstream dependencies or downstream dependencies and then execute the flow. azkaban.readthedocs.io/en/latest/…