Skip to content

airflow scheduler ERROR - Exception when executing SchedulerJob._run_scheduler_loop #23682

Description

@rafficghani

Apache Airflow version

2.2.2

What happened

The scheduler goes down

[2022-05-12 11:14:06,586] {scheduler_job.py:1114} INFO - Resetting orphaned tasks for active dag runs
[2022-05-12 11:14:06,806] {scheduler_job.py:644} ERROR - Exception when executing SchedulerJob._run_scheduler_loop
Traceback (most recent call last):
File "/airflow/airflowvirt/lib64/python3.6/site-packages/airflow/jobs/scheduler_job.py", line 628, in _execute
self._run_scheduler_loop()
File "/airflow/airflowvirt/lib64/python3.6/site-packages/airflow/jobs/scheduler_job.py", line 721, in _run_scheduler_loop
next_event = timers.run(blocking=False)
File "/usr/lib64/python3.6/sched.py", line 154, in run
action(*argument, **kwargs)
File "/airflow/airflowvirt/lib64/python3.6/site-packages/airflow/utils/event_scheduler.py", line 36, in repeat
action(*args, **kwargs)
File "/airflow/airflowvirt/lib64/python3.6/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/airflow/airflowvirt/lib64/python3.6/site-packages/airflow/jobs/scheduler_job.py", line 1117, in adopt_or_reset_orphaned_tasks
for attempt in run_with_db_retries(logger=self.log):
File "/airflow/airflowvirt/lib64/python3.6/site-packages/tenacity/init.py", line 382, in iter
do = self.iter(retry_state=retry_state)
File "/airflow/airflowvirt/lib64/python3.6/site-packages/tenacity/init.py", line 349, in iter
return fut.result()
File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 425, in result
return self.__get_result()
File "/usr/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
raise self._exception
File "/airflow/airflowvirt/lib64/python3.6/site-packages/airflow/jobs/scheduler_job.py", line 1162, in adopt_or_reset_orphaned_tasks
to_reset = self.executor.try_adopt_task_instances(tis_to_reset_or_adopt)
File "/airflow/airflowvirt/lib64/python3.6/site-packages/airflow/executors/celery_executor.py", line 485, in try_adopt_task_instances
self.adopted_task_timeouts[ti.key] = ti.queued_dttm + self.task_adoption_timeout
TypeError: unsupported operand type(s) for +: 'NoneType' and 'datetime.timedelta'
[2022-05-12 11:14:08,027] {process_utils.py:100} INFO - Sending Signals.SIGTERM to GPID 3929915

What you think should happen instead

No response

How to reproduce

No response

Operating System

Redhat x86_64 GNU/Linux

Versions of Apache Airflow Providers

No response

Deployment

Virtualenv installation

Deployment details

No response

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions