Skip to content

Add team_name tag to dag processor metrics for multi-team deployments#68599

Merged
ferruzzi merged 5 commits into
apache:mainfrom
aws-mwaa:ferruzzi/multi-team/dag-processing-metrics
Jun 18, 2026
Merged

Add team_name tag to dag processor metrics for multi-team deployments#68599
ferruzzi merged 5 commits into
apache:mainfrom
aws-mwaa:ferruzzi/multi-team/dag-processing-metrics

Conversation

@ferruzzi

Copy link
Copy Markdown
Contributor

Part 6 of 6, adding team_name to all metrics when available.

Follow-up to #68108
Can be reviewed/merged in parallel with #68367, #68589, #68593, and #68594


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@ferruzzi
ferruzzi force-pushed the ferruzzi/multi-team/dag-processing-metrics branch from 3050329 to 0cf8036 Compare June 16, 2026 06:32
@ferruzzi

Copy link
Copy Markdown
Contributor Author

Failing CI is legit, I'll fix it when I get a sec.

@ferruzzi
ferruzzi requested review from o-nikolas and vincbeck June 17, 2026 04:44

@o-nikolas o-nikolas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this one is okay, since it only affects that multi_team=True path, so by default there will be no impact. But in the future we're probably going to want to optimize this. Some ideas that come to my head:

  1. Memoizing the the DagBundleModel.get_team_name(...) call, or creating some
  2. Create a bundle_name: team mapping/cache like we do in the scheduler
  3. Could also read the bundle to team mapping from config, since it's specified in Airflow config, that would safe the DB round trip (but you'd probably want to still do one of the top two in addition to that).

@ferruzzi

Copy link
Copy Markdown
Contributor Author

That's fair. I'll get this merged so we know it'll be in 3.3.0 and follow up immediately with a cached helper like I did with the scheduler. It should be up today; I'll tag you when it's ready.

@ferruzzi
ferruzzi merged commit aa0609f into apache:main Jun 18, 2026
77 checks passed
@ferruzzi
ferruzzi deleted the ferruzzi/multi-team/dag-processing-metrics branch June 18, 2026 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants