Skip to content

A dag's schedule interval can no longer be an instance of dateutils.relativedelta #19416

Description

@rcheatham-q

Apache Airflow version

2.2.1 (latest released)

Operating System

debian

Versions of Apache Airflow Providers

apache-airflow==2.2.1
apache-airflow-providers-amazon==2.3.0
apache-airflow-providers-ftp==2.0.1
apache-airflow-providers-google==6.0.0
apache-airflow-providers-http==2.0.1
apache-airflow-providers-imap==2.0.1
apache-airflow-providers-jira==2.0.1
apache-airflow-providers-mysql==2.1.1
apache-airflow-providers-postgres==2.3.0
apache-airflow-providers-redis==2.0.1
apache-airflow-providers-sqlite==2.0.1
apache-airflow-providers-ssh==2.2.0

Deployment

Other Docker-based deployment

Deployment details

Dask executor, custom-built Docker images, postgres 12.7 backend

What happened

I upgraded Airflow from 2.0.2 to 2.2.1, and some DAGs I have that used dateutils.relativedelta objects as schedule intervals stopped running

What you expected to happen

The code for the schedule_interval parameter of the DAG constructor indicates that a relativedelta object is allowed, so I expected the DAG to be correctly parsed and scheduled.

How to reproduce

Create a DAG that has a relativedelta object as its schedule interval, and it will not appear in the UI or be scheduled.

Anything else

Here is the code that causes the failure within the PR where it was introduced: link

Here are the logs for the exception, found in the scheduler logs for the file that contains the offending DAG

Details
ERROR   | {dagbag.py:528} - 'relativedelta' object has no attribute 'total_seconds'
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/airflow/models/dagbag.py", line 515, in collect_dags
    found_dags = self.process_file(filepath, only_if_updated=only_if_updated, safe_mode=safe_mode)
  File "/usr/local/lib/python3.9/site-packages/airflow/models/dagbag.py", line 298, in process_file
    found_dags = self._process_modules(filepath, mods, file_last_changed_on_disk)
  File "/usr/local/lib/python3.9/site-packages/airflow/models/dagbag.py", line 401, in _process_modules
    dag.timetable.validate()
  File "/usr/local/lib/python3.9/site-packages/airflow/timetables/interval.py", line 274, in validate
    if self._delta.total_seconds() <= 0:
AttributeError: 'relativedelta' object has no attribute 'total_seconds'

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions