Airflow: New DAG is not found by webserver

Question

In Airflow, how should I handle the error "This DAG isn't available in the webserver DagBag object. It shows up in this list because the scheduler marked it as active in the metadata database"?

I've copied a new DAG to an Airflow server, and have tried:

• unpausing it and refreshing it (basic operating procedure, given in this previous answer https://stackoverflow.com/a/42291683/160406)
• restarting the webserver
• restarting the scheduler
• stopping the webserver and scheduler, resetting the database (airflow resetdb), then starting the webserver and scheduler again
• running airflow backfill (suggested here Airflow "This DAG isnt available in the webserver DagBag object ")
• running airflow trigger_dag

The scheduler log shows it being processed and no errors occurring, I can interact with it and view it's state through the CLI, but it still does not appear in the web UI.

Edit: the webserver and scheduler are running on the same machine with the same airflow.cfg. They're not running in Docker.

They're run by Supervisor, which runs them both as the same user (airflow). The airflow user has read, write and execute permission on all of the dag files.

Solution 1

This helped me...

pkill -9 -f "airflow scheduler"

pkill -9 -f "airflow webserver"

pkill -9 -f "gunicorn"

then restart the airflow scheduler and webserver.

Solution 2

Just had this issue myself. After changing permissions, resetting the meta database, restarting the webserver & even making some potential code changes to rectify the situation, it didn't happen.

However, I noticed that even though we were stopping the webserver, our gunicorn process was still running. Killing these processes & then starting everything back up resulted in success

Solution 3

I had the same problem on an airflow installed from a Docker image

What I did was:

1- delete all files .pyc

2- delete Metadata databse using :

for t in ["xcom", "task_instance", "sla_miss", "log", "job", "dag_run", "dag" ]:
sql="delete from {} where dag_id='{}'".format(t, dag_input)
hook.run(sql, True)

3- restart webserver & scheduler

4- Execute airflow updatedb

It resolved the problem for me.