Change provider-specific dependencies to refer to providers#48843
Merged
Conversation
With the recent `uv` move, we can move some of the optional dependencies to providers - i.e. rather than refer to specific dependenciesm we can refer to the optional dependencies of the providers. This was not easy before - because with `pip` for development dependencies it would have install the provider itself instead of the dependencies, but with uv workspaces this is perfectly fine as uv will find the provider in the workspace and will install the provider sources and the optional dependencies when given extra is used. This avoids duplication in the core's dependencies and allows to depend airflow core noot on specific dependencies but on the optional dependencies of provider installed, which is as it should be. This change also bumps the dependencies of amazon provider that are affected by this change in an attempt to make "resolution too deep" problems.
potiuk
force-pushed
the
optional-extras-moved-to-providers
branch
from
April 6, 2025 11:46
ce1b53b to
9d00ad6
Compare
ashb
approved these changes
Apr 6, 2025
eladkal
approved these changes
Apr 6, 2025
jscheffl
approved these changes
Apr 6, 2025
jscheffl
left a comment
Contributor
There was a problem hiding this comment.
Cool! Crossing fingers that the resolution too Deep will go away by this!
Member
Author
|
Let's see if it will help for the "resolution too deep". I am in a plain above the Atlantic now ;) so with low network speed no chance to test it locally for image building, but 1) far less number of combos for Amazon, 2) it's likely that having the dependencies on both - extra and different versions of providers made it difficult for |
potiuk
added a commit
to potiuk/airflow
that referenced
this pull request
Apr 6, 2025
…8843) With the recent `uv` move, we can move some of the optional dependencies to providers - i.e. rather than refer to specific dependenciesm we can refer to the optional dependencies of the providers. This was not easy before - because with `pip` for development dependencies it would have install the provider itself instead of the dependencies, but with uv workspaces this is perfectly fine as uv will find the provider in the workspace and will install the provider sources and the optional dependencies when given extra is used. This avoids duplication in the core's dependencies and allows to depend airflow core noot on specific dependencies but on the optional dependencies of provider installed, which is as it should be. This change also bumps the dependencies of amazon provider that are affected by this change in an attempt to make "resolution too deep" problems.
diogotrodrigues
pushed a commit
to diogotrodrigues/airflow
that referenced
this pull request
Apr 6, 2025
…8843) With the recent `uv` move, we can move some of the optional dependencies to providers - i.e. rather than refer to specific dependenciesm we can refer to the optional dependencies of the providers. This was not easy before - because with `pip` for development dependencies it would have install the provider itself instead of the dependencies, but with uv workspaces this is perfectly fine as uv will find the provider in the workspace and will install the provider sources and the optional dependencies when given extra is used. This avoids duplication in the core's dependencies and allows to depend airflow core noot on specific dependencies but on the optional dependencies of provider installed, which is as it should be. This change also bumps the dependencies of amazon provider that are affected by this change in an attempt to make "resolution too deep" problems.
simonprydden
pushed a commit
to simonprydden/airflow
that referenced
this pull request
Apr 8, 2025
…8843) With the recent `uv` move, we can move some of the optional dependencies to providers - i.e. rather than refer to specific dependenciesm we can refer to the optional dependencies of the providers. This was not easy before - because with `pip` for development dependencies it would have install the provider itself instead of the dependencies, but with uv workspaces this is perfectly fine as uv will find the provider in the workspace and will install the provider sources and the optional dependencies when given extra is used. This avoids duplication in the core's dependencies and allows to depend airflow core noot on specific dependencies but on the optional dependencies of provider installed, which is as it should be. This change also bumps the dependencies of amazon provider that are affected by this change in an attempt to make "resolution too deep" problems.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
With the recent
uvmove, we can move some of the optional dependencies to providers - i.e. rather than refer to specific dependenciesm we can refer to the optional dependencies of the providers. This was not easy before - because withpipfor development dependencies it would have install the provider itself instead of the dependencies, but with uv workspaces this is perfectly fine as uv will find the provider in the workspace and will install the provider sources and the optional dependencies when given extra is used.This avoids duplication in the core's dependencies and allows to depend airflow core noot on specific dependencies but on the optional dependencies of provider installed, which is as it should be.
This change also bumps the dependencies of amazon provider that are affected by this change in an attempt to make "resolution too deep" problems.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in airflow-core/newsfragments.