What workaround was applied
pyproject.toml's cloudpickle extra was capped to cloudpickle>=2.2.1,<3.1.2 (previously unbounded >=2.2.1).
Why
This is a workaround for a dependency-resolution deadlock, not a real Airflow bug. Root cause, investigated for #70020:
- Airflow's
all extra includes the cloudpickle extra (unbounded floor, no ceiling).
apache-airflow-providers-snowflake>=6.5.0 requires snowflake-snowpark-python on Python >=3.12.- Every released version of
snowflake-snowpark-python (up to at least 1.53.1) pins cloudpickle<=3.1.1 in its own metadata.
cloudpickle==3.1.2 (the latest PyPI release) is incompatible with that cap.- When constraints are generated with
uv ... --resolution highest, uv tries to satisfy "highest" for both the bare cloudpickle extra and the snowflake provider simultaneously. Since they're mutually incompatible at the top, instead of settling on cloudpickle==3.1.1 (the highest version that still satisfies both), uv's backtracking downgrades apache-airflow-providers-snowflake all the way down to 6.4.0 — the last release with no snowpark requirement at all on Python 3.12.
This was confirmed by reproducing the resolution directly against the released constraints-3.3.0 files with uv pip install, and by reproducing the fix (capping cloudpickle resolves the conflict and lets uv pick the latest snowflake provider release again).
Follow-up work
Remove the <3.1.2 cap on cloudpickle in pyproject.toml once snowflake-snowpark-python relaxes its own cloudpickle pin to allow >=3.1.2 (or whatever the current cloudpickle release is at that time).
Acceptance criteria
- A released version of
snowflake-snowpark-python no longer caps cloudpickle<=3.1.1.
- Regenerating constraints (
breeze release-management generate-constraints) with the cap removed no longer regresses apache-airflow-providers-snowflake to an old version on any supported Python version.
Related
What workaround was applied
pyproject.toml'scloudpickleextra was capped tocloudpickle>=2.2.1,<3.1.2(previously unbounded>=2.2.1).Why
This is a workaround for a dependency-resolution deadlock, not a real Airflow bug. Root cause, investigated for #70020:
allextra includes thecloudpickleextra (unbounded floor, no ceiling).apache-airflow-providers-snowflake>=6.5.0requiressnowflake-snowpark-pythonon Python >=3.12.snowflake-snowpark-python(up to at least 1.53.1) pinscloudpickle<=3.1.1in its own metadata.cloudpickle==3.1.2(the latest PyPI release) is incompatible with that cap.uv ... --resolution highest, uv tries to satisfy "highest" for both the barecloudpickleextra and the snowflake provider simultaneously. Since they're mutually incompatible at the top, instead of settling oncloudpickle==3.1.1(the highest version that still satisfies both), uv's backtracking downgradesapache-airflow-providers-snowflakeall the way down to6.4.0— the last release with no snowpark requirement at all on Python 3.12.This was confirmed by reproducing the resolution directly against the released
constraints-3.3.0files withuv pip install, and by reproducing the fix (capping cloudpickle resolves the conflict and letsuvpick the latest snowflake provider release again).Follow-up work
Remove the
<3.1.2cap oncloudpickleinpyproject.tomloncesnowflake-snowpark-pythonrelaxes its owncloudpicklepin to allow>=3.1.2(or whatever the current cloudpickle release is at that time).Acceptance criteria
snowflake-snowpark-pythonno longer capscloudpickle<=3.1.1.breeze release-management generate-constraints) with the cap removed no longer regressesapache-airflow-providers-snowflaketo an old version on any supported Python version.Related
apache-airflow-providers-snowflakepinned to a >1-year-old version in theconstraints-3.3.0files)