Skip to content

Fix task scheduling when multi_team is enabled#68634

Merged
o-nikolas merged 1 commit into
apache:mainfrom
aws-mwaa:vincbeck/fix_mt_scheduling
Jun 16, 2026
Merged

Fix task scheduling when multi_team is enabled#68634
o-nikolas merged 1 commit into
apache:mainfrom
aws-mwaa:vincbeck/fix_mt_scheduling

Conversation

@vincbeck

Copy link
Copy Markdown
Contributor

Summary

When multi_team is enabled, tasks remain in "No status" indefinitely and Dag runs never complete.

The root cause is that get_running_dag_runs_to_examine() returns a ScalarResult (a one-pass SQLAlchemy iterator). The multi-team code path added in #68108 iterates it to build a team-name mapping, exhausting the iterator before _schedule_all_dag_runs can consume it. As a result, zero Dag runs get processed and no task instances ever transition out of None state.

The fix materializes the ScalarResult into a list() so it can be safely iterated multiple times.


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@vincbeck
vincbeck requested review from ferruzzi and o-nikolas June 16, 2026 15:29
@vincbeck
vincbeck requested review from XD-DENG and ashb as code owners June 16, 2026 15:29
@boring-cyborg boring-cyborg Bot added the area:Scheduler including HA (high availability) scheduler label Jun 16, 2026
@vincbeck vincbeck added this to the Airflow 3.3.0 milestone Jun 16, 2026
@o-nikolas
o-nikolas merged commit 1b6e47a into apache:main Jun 16, 2026
77 checks passed
@o-nikolas
o-nikolas deleted the vincbeck/fix_mt_scheduling branch June 16, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:Scheduler including HA (high availability) scheduler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants