Skip to content

fix: pod_override existing init_containers#62284

Merged
jscheffl merged 2 commits into
apache:mainfrom
bintocher:bintocher/issue-61074
Feb 21, 2026
Merged

fix: pod_override existing init_containers#62284
jscheffl merged 2 commits into
apache:mainfrom
bintocher:bintocher/issue-61074

Conversation

@bintocher

Copy link
Copy Markdown
Contributor

Summary

Fixes init container reconciliation in PodGenerator.reconcile_specs() to properly merge init containers by name instead of simply appending them. This aligns init container handling with the existing behavior for regular containers.

Issue

Fixes #61074

Changes

  • Added reconcile_init_containers() method to merge init containers by name, matching the pattern used for regular containers
  • Updated reconcile_specs() to use the new method instead of extend_object_field()
  • For containers with matching names, lists like volume_mounts, env, env_from, ports, and volume_devices are extended; other properties are merged with client values taking precedence
  • Added unit test for init container reconciliation with same-name containers

Testing

  • Added new test test_reconcile_specs_init_containers_same_name verifying that init containers with the same name are properly merged
  • Test confirms that client image overrides base image while base args are preserved

Previously, init containers were simply appended when using pod_override,
leading to duplicates when the same container name existed in both base
and client specs. Now init containers are properly reconciled by name,
matching the behavior of regular containers - same-name containers are
merged while others are appended.
Fixes apache#61074
@jscheffl

Copy link
Copy Markdown
Contributor

Thanks for the contribution! To better understand the fix, can you elaborate a bit on the reason you added this? I would assume if there are multiple init containers with the same name this is rather a conflict and might rather expect to raise an alarm. Merging might be OK but also feels like a source of problems and surprises if you do not know about. And to think about consistency, the same could also apply to volumes in just two lines below.

@jscheffl

Copy link
Copy Markdown
Contributor

Note: Reading the bug ticket, is maybe a hook needed to influence how init container and parameters are produced compared to assume a correct merge of properties is made?

Keep the original extend_object_field call for init_containers to avoid
mutating client_spec and producing empty lists instead of None. Only
apply name-based reconciliation when both specs have init_containers.
@bintocher

Copy link
Copy Markdown
Contributor Author

Thanks for the review @jscheffl!

The motivation comes from issue #61074: when using the Kubernetes executor in multi-namespace mode with git-sync, the Helm chart configures the same git-sync init container for all components. But a worker only needs a specific submodule, not the full repo. With executor_config's pod_override, users should be able to override the existing init container by name (e.g. change just the repo URL or args) rather than getting a duplicate.

This aligns with how regular containers already work in reconcile_containers — containers are matched by position and merged (client values override base, lists like env and volume_mounts are extended). The new reconcile_init_containers does the same but matches by name instead of position, since init containers don't have a guaranteed positional correspondence.

When names don't match, containers are still appended (same behavior as before). Name-based merge only kicks in when both base and client have an init container with the same name.

Regarding volumes — good point about consistency. However, volumes don't have the same "override by identity" semantic since they are referenced by name from volumeMounts. Extending (appending) volumes is the correct behavior as it is today — you want all volumes available. The init container case is different because you want to replace the configuration of a specific named container, not add a duplicate.


Regarding the hook approach: a hook would be a more flexible solution long-term, but it would be a much larger change involving new hook points, configuration, and documentation. This PR follows the existing pattern already established by reconcile_containers for regular containers, keeping the change minimal and consistent with the current architecture. A hook could be considered as a follow-up if more advanced customization is needed.

I also pushed a fix for the CI failures — the original code was mutating client_spec directly instead of using extend_object_field's deepcopy pattern, which caused init_containers to become [] instead of None when not present.

@jscheffl jscheffl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, arguments are convicing. Reading other code a second time.

@jscheffl
jscheffl merged commit e90d953 into apache:main Feb 21, 2026
152 of 157 checks passed
@boring-cyborg

boring-cyborg Bot commented Feb 21, 2026

Copy link
Copy Markdown

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

@bintocher
bintocher deleted the bintocher/issue-61074 branch February 21, 2026 23:42
choo121600 pushed a commit to choo121600/airflow that referenced this pull request Feb 22, 2026
* fix: merge init containers by name in pod_override
Previously, init containers were simply appended when using pod_override,
leading to duplicates when the same container name existed in both base
and client specs. Now init containers are properly reconciled by name,
matching the behavior of regular containers - same-name containers are
merged while others are appended.
Fixes apache#61074

* fix: preserve extend_object_field deepcopy for init_containers

Keep the original extend_object_field call for init_containers to avoid
mutating client_spec and producing empty lists instead of None. Only
apply name-based reconciliation when both specs have init_containers.
dominikhei pushed a commit to dominikhei/airflow that referenced this pull request Mar 11, 2026
* fix: merge init containers by name in pod_override
Previously, init containers were simply appended when using pod_override,
leading to duplicates when the same container name existed in both base
and client specs. Now init containers are properly reconciled by name,
matching the behavior of regular containers - same-name containers are
merged while others are appended.
Fixes apache#61074

* fix: preserve extend_object_field deepcopy for init_containers

Keep the original extend_object_field call for init_containers to avoid
mutating client_spec and producing empty lists instead of None. Only
apply name-based reconciliation when both specs have init_containers.
Ankurdeewan pushed a commit to Ankurdeewan/airflow that referenced this pull request Mar 15, 2026
* fix: merge init containers by name in pod_override
Previously, init containers were simply appended when using pod_override,
leading to duplicates when the same container name existed in both base
and client specs. Now init containers are properly reconciled by name,
matching the behavior of regular containers - same-name containers are
merged while others are appended.
Fixes apache#61074

* fix: preserve extend_object_field deepcopy for init_containers

Keep the original extend_object_field call for init_containers to avoid
mutating client_spec and producing empty lists instead of None. Only
apply name-based reconciliation when both specs have init_containers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers provider:cncf-kubernetes Kubernetes (k8s) provider related issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pod_override existing init_containers

2 participants