DAG dependencies graph not updating when deleting a DAG

[alejandrofm]

Apache Airflow version

Other Airflow 2 version (please specify below)

What happened

ON Airflow 2.4.2 Dag dependencies graph show deleted DAGs that use to have dependencies to currently existing DAGs

What you think should happen instead

Deleted DAGs should not appear on DAG Dependencies

How to reproduce

Create a DAG with dependencies on other DAG, like a wait sensor.
Remove new DAG

Operating System

apache/airflow:2.4.2-python3.10

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[bbovenzi]

We generate that graph based on SerializedDagModel.get_dag_dependencies(). I'm not too sure as to when that is updated after deleting a dag. Maybe someone with more knowledge on that side of things could chime in?

[michaelmicheal]

I think this related to not excluding disabled DAGs. Can you assign this to me? I'd like to tackle it this week

[michaelmicheal]

I actually wasn't able to reproduce this behaviour. I didn't realize, but we started deleting serialized dags when removing stale DAGs here: f878854#diff-bde85feb359b12bdd358aed4106ef4fccbd8fa9915e16b9abb7502912a1c1ab3.

@alejandrofm How are you deleting the DAG? Removing it from the DAG bucket?

[michaelmicheal]

Actually I was able to reproduce. I think that there is a bug in the deletion of serialized DAGs. Should be able to figure it out and come up with a fix.

[alejandrofm]

Great, just to add the info:
We make the DAG changes on Gitlab and then retrieve the DAG folder via git-sync.