Add team_name tags to Kubernetes executor metrics#69046
Merged
shahar1 merged 2 commits intoJul 1, 2026
Conversation
SameerMesiah97
force-pushed
the
68996-CNCF-Kubernetes-Team-Name-Metrics
branch
4 times, most recently
from
June 26, 2026 20:11
58fc3ca to
a3aa3fd
Compare
1 task
SameerMesiah97
marked this pull request as ready for review
June 26, 2026 21:45
SameerMesiah97
requested review from
hussein-awala,
jedcunningham and
jscheffl
as code owners
June 26, 2026 21:45
Contributor
|
I'd like to see the unit tests added for the missing paths. Improving the coverage while you are in there seems reasonable and they shouldn't be too difficult to follow the existing patterns. |
Contributor
Author
Okay. I was thinking of submitting a follow-up PR for them but I can them to this PR on your request. |
SameerMesiah97
force-pushed
the
68996-CNCF-Kubernetes-Team-Name-Metrics
branch
from
June 27, 2026 14:40
a3aa3fd to
fcd607a
Compare
SameerMesiah97
marked this pull request as draft
June 27, 2026 14:41
SameerMesiah97
marked this pull request as ready for review
June 27, 2026 18:10
ferruzzi
reviewed
Jun 29, 2026
ferruzzi
left a comment
Contributor
There was a problem hiding this comment.
Needs a rebase to clear the merge conflict but otherwise LGTM.
SameerMesiah97
force-pushed
the
68996-CNCF-Kubernetes-Team-Name-Metrics
branch
from
June 29, 2026 23:05
fcd607a to
1f866a7
Compare
SameerMesiah97
marked this pull request as draft
June 30, 2026 00:08
SameerMesiah97
marked this pull request as ready for review
June 30, 2026 00:08
added 2 commits
June 30, 2026 09:25
Add the team_name tag to Kubernetes executor metrics to improve per-team observability in multi-team deployments. Pass the executor's team_name to AirflowKubernetesScheduler so scheduler-emitted metrics include the tag consistently. Maintain compatibility with prior Airflow releases by initializing team_name only when it is not already defined. Update the existing metric tests to verify metrics are emitted both with and without the team_name tag.
and pod patching metrics, covering both success and failure paths. Extend the existing task adoption test to verify that the kubernetes_executor.adopt_task_instances.duration metric is emitted with the expected team_name tag when configured.
SameerMesiah97
force-pushed
the
68996-CNCF-Kubernetes-Team-Name-Metrics
branch
from
June 30, 2026 08:25
1f866a7 to
dcdd488
Compare
Contributor
Author
Done. |
ferruzzi
approved these changes
Jun 30, 2026
shahar1
approved these changes
Jul 1, 2026
14 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This change adds a
team_nametag to Kubernetes executor metrics to improve per-team observability in multi-team deployments.The following metrics now include the
team_nametag when the executor is configured with a team:kubernetes_executor.pod_creationkubernetes_executor.pod_deletionkubernetes_executor.pod_patchingkubernetes_executor.adopt_task_instances.durationkubernetes_executor.pod_creation_statuskubernetes_executor.pod_deletion_statuskubernetes_executor.pod_patching_statusThe metric
kubernetes_executor.clear_not_launched_queued_tasks.durationwas not updated because it is no longer present in the current Kubernetes executor implementation.With the exception of
kubernetes_executor.adopt_task_instances.duration, all of the above metrics are emitted fromAirflowKubernetesScheduler.Rationale
The Kubernetes executor supports multi-team scheduling, but its operational metrics could not previously be attributed to individual teams. Adding the
team_nametag enables per-team dashboards, alerting, and troubleshooting while remaining backwards compatible for deployments that do not use teams.Notes
team_nameis owned byKubernetesExecutor, while most of the affected metrics are emitted fromAirflowKubernetesScheduler. To allow scheduler-emitted metrics to include theteam_nametag, the value is passed toAirflowKubernetesSchedulerwhen it is constructed by the executor.For compatibility with previous Airflow releases,
team_nameis initialized only if it is not already defined, since the attribute was introduced as part of multi-team support.Tests
Updated the existing unit tests for
kubernetes_executor.pod_deletionandkubernetes_executor.pod_deletion_statusto verify metrics are emitted both with and without theteam_nametag. The test cases forteam_namebeing present in the metrics are skipped when airflow version is less than 3.1.0 as theteam_nameattribute was added in version 3.1.0.The following metrics do not currently have dedicated unit test coverage, so test new tests were added or existing tests were adjusted for them:
kubernetes_executor.pod_creationkubernetes_executor.pod_creation_statuskubernetes_executor.pod_patchingkubernetes_executor.pod_patching_statuskubernetes_executor.adopt_task_instances.durationBackwards Compatibility
This change is additive only. Existing metric names remain unchanged, and the
team_nametag is emitted only when the Kubernetes executor is configured with a team. Existing dashboards and integrations that do not use the new tag continue to function unchanged.Related: #68996
Was generative AI tooling used to co-author this PR?
Generated-by: [GPT 5.5] following the guidelines