Celery provider 3.16.0 breaks Airflow 2.11.1

[potiuk]

Body

The 3.16.0 provider installed with Airflow 2.11.1 fails celery executor with:

Traceback (most recent call last):
    File "/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/scheduler_job_runner.py", line 1008, in _execute
      self._run_scheduler_loop()
    File "/home/airflow/.local/lib/python3.10/site-packages/airflow/jobs/scheduler_job_runner.py", line 1153, in _run_scheduler_loop
      executor.heartbeat()
    File "/home/airflow/.local/lib/python3.10/site-packages/airflow/traces/tracer.py", line 58, in wrapper
      return func(*args, **kwargs)
    File "/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/base_executor.py", line 244, in heartbeat
      self.trigger_tasks(open_slots)
    File "/home/airflow/.local/lib/python3.10/site-packages/airflow/traces/tracer.py", line 58, in wrapper
      return func(*args, **kwargs)
    File "/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/base_executor.py", line 383, in trigger_tasks
      self._process_tasks(task_tuples)
    File "/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/celery/executors/celery_executor.py", line 150, in _process_tasks
      task_tuples_to_send = [task_tuple[:3] + (self.team_name,) for task_tuple in task_tuples]
    File "/home/airflow/.local/lib/python3.10/site-packages/airflow/providers/celery/executors/celery_executor.py", line 150, in <listcomp>
      task_tuples_to_send = [task_tuple[:3] + (self.team_name,) for task_tuple in task_tuples]
  AttributeError: 'CeleryExecutor' object has no attribute 'team_name'

See https://github.com/apache/airflow/actions/runs/22018420629/job/63624588757?pr=61909#step:5:1775

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[potiuk]

cc: @vincbeck @o-nikolas

[o-nikolas]

I don't see that exception if I follow the provided link to the test run. I do see some Local and Celery executor tests timing out on K8s though?

Never mind I see the error now. I'm very confused how this is only failing now. Many tests were run on 2.11 with the full tests needed label on. Looking into it though.

[o-nikolas]

I'm not really sure how to reproduce the failure on my laptop with breeze testing docker-compose-tests @potiuk do I have to build a specific prod image first?

[potiuk]

See the ocmments in PR - it was really 2.11.1 tests in v2-11-test.

[JegadeeshManickam]

See the ocmments in PR - it was really 2.11.1 tests in v2-11-test.

We are facing the same issue, whats the version to use to fix the above

[o-nikolas]

@potiuk the latest provider release starting today should resolve this right?

[potiuk]

@JegadeeshManickam -> like with all deps, constraints matter -> look at constraints that airflow comes with (3.15.2 is currently there).

Note, that there was a need to re-publish 2.11.1 image as explained in https://lists.apache.org/thread/dy4v94h1tnowh9nn7z5l0gld8c82oocy (and described in changelog of the docker-stack) - so you might have used the previous version of the image that could have unbounded dependencies.