Optimize /ui/dags endpoint serialization by john-rodriguez-mgni · Pull Request #61483 · apache/airflow

john-rodriguez-mgni · 2026-02-05T05:40:36Z

Summary

This PR addresses a significant performance issue in the /ui/dags endpoint where page load times scaled poorly with the number of DAGs (12-16 seconds for just 25 DAGs in our testing).

Two optimizations are implemented:

1. Cache URLSafeSerializer for file_token generation

Previously, a new URLSafeSerializer was instantiated and conf.get_mandatory_value() was called for every DAG in the response. Now uses @lru_cache to create the serializer once and reuse it.

2. Eliminate redundant Pydantic validation in response construction

The original pattern used:

DAGWithLatestDagRunsResponse.model_validate({
    **DAGResponse.model_validate(dag).model_dump(),
    ...
})

This caused triple serialization overhead per DAG (validate → dump → validate). The fix validates once with DAGResponse.model_validate(), then uses model_construct() to build DAGWithLatestDagRunsResponse without redundant validation.

Results

Together, these changes reduced page load time from 12-16 seconds to ~130ms in our dev environment.

Test Plan

Existing unit tests pass
Manual testing of /ui/dags endpoint with varying numbers of DAGs
Verify file_token generation still works correctly

Made with Cursor

boring-cyborg · 2026-02-05T05:40:39Z

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:

Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example DAG that shows how users should use it.
Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
Be sure to read the Airflow Coding style.
Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack

airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py

pierrejeambrun

Nice, thanks for the PR.

A few questions/suggestions, but overall looking good.

airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py

pierrejeambrun

Nice thank for the PR.

A couple of suggestion / improvements before we can merge I think.

airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py

pierrejeambrun

LGTM, we can merge after we addressed above comments.

jason810496

Nice! Thanks for the improvement.

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py

john-rodriguez-mgni · 2026-02-12T19:48:27Z

Just an update on this... since opening this PR, I have included this patch in our dev and QA environments and validated there are no issues and that it resolves the load time for the ui/dags endpoint seen with the base airflow 3.1.7 release sans patches. Earlier today I released it to our production environment and are not seeing any issues either and can confirm that load time for the ui/dags endpoint is fixed.

Let me know what else is required to move this along. Again, really appreciate your feedback and thank you for your help throughout this process!

pierrejeambrun

Let me know what else is required to move this along. Again, really appreciate your feedback and thank you for your help throughout this process!

Can you just fix the CI please (mypy static check error). There are two discussion above that explains what the problem is.

Basically we need to keep the # type: ignore[attr-defined] and revert the change to the get_latest_run_info which is creating mypy errors and isn't really needed for that PR. (that endpoint isn't the performance bottleneck)

john-rodriguez-mgni · 2026-02-16T18:22:30Z

Let me know what else is required to move this along. Again, really appreciate your feedback and thank you for your help throughout this process!

Can you just fix the CI please (mypy static check error). There are two discussion above that explains what the problem is.

Basically we need to keep the # type: ignore[attr-defined] and revert the change to the get_latest_run_info which is creating mypy errors and isn't really needed for that PR. (that endpoint isn't the performance bottleneck)

Made updates as requested!

john-rodriguez-mgni · 2026-02-18T22:20:42Z

@pierrejeambrun just checking on this to see if there was anything else required on my end, I have made the changes necessary to address the CI failures... I think we just need someone to approve the workflows.

pierrejeambrun

Thanks, CI is running it should be better 🤞

pierrejeambrun

CI need fixing some tests are failing, can you take a look please.

john-rodriguez-mgni · 2026-02-24T18:53:48Z

@pierrejeambrun it seems that all 10 CI failures failures trace back to the same root cause; the allowed_run_types field was added to DAGResponse on themain branch after this branch was created. Since the optimization switches from DAGResponse.model_validate(dag) (which dynamically picks up all model fields from the ORM object) to an explicit dict construction, the new field is missing from the dict, causing a ValidationError: allowed_run_types - Field required on DAGWithLatestDagRunsResponse. The 7 API/Serialization suite failures hit this directly, and the 3 UI e2e failures (Chromium/Firefox/WebKit) are a downstream consequence — the /ui/dags endpoint returns 500, so the dags-list e2e tests can't render anything.

The immediate fix is simple — add "allowed_run_types": dag.allowed_run_types to the dict. However, this highlights a fragility in the explicit-field approach: any new field added to DAGResponse requires a corresponding update here, which is easy to miss. To make this more robust, we could build the dict dynamically from DAGResponse.model_fields which iterates over the model's field definitions to pull attributes from the ORM object, so new fields are picked up automatically. The dict construction is still just cheap getattr calls, and there's still only one model_validate call. Overhead of iterating model_fields is negligible compared to Pydantic validation.

Thoughts?

john-rodriguez-mgni · 2026-02-26T18:52:51Z

@pierrejeambrun any thoughts on the proposed implementation: #61483 (comment)

john-rodriguez-mgni · 2026-02-27T06:53:56Z

@pierrejeambrun I have decided to build the dict dynamically from DAGResponse.model_fields to make this robust. I ran the CI test locally and confirmed that fixes the issue. I also deployed this locally to our Airflow dev instances and verified that the dags endpoint performance gain is still present. We are ready to approve the CI workflows again. Please approve them at your earliest convenience.

This PR addresses a significant performance issue in the /ui/dags endpoint where page load times scaled poorly with the number of DAGs (12-16 seconds for just 25 DAGs in our testing). Two optimizations are implemented: 1. Cache URLSafeSerializer for file_token generation - Previously, a new URLSafeSerializer was instantiated and conf.get_mandatory_value() was called for every DAG - Now uses @lru_cache to create the serializer once and reuse it 2. Eliminate redundant Pydantic validation in response construction - The original pattern used model_validate -> model_dump -> model_validate which caused triple serialization overhead per DAG - Now validates once with DAGResponse.model_validate(), then uses model_construct() to build DAGWithLatestDagRunsResponse Together, these changes reduced page load time from 12-16 seconds to ~130ms in our dev environment. Co-authored-by: Cursor <cursoragent@cursor.com>

john-rodriguez-mgni · 2026-03-02T17:28:53Z

@pierrejeambrun fix the issue with the static check (i.e. ruff wanting the imports sorted alphabetically). The browser failures do not appear to be related to my changes. The K8S System:KubernetesExecutor test is a full Kubernetes integration test and the change I am introducing has no interaction with the Kubernetes executor. Please approve the workflows at your earliest convenience.

john-rodriguez-mgni · 2026-03-03T17:08:12Z

@pierrejeambrun , @vatsrahul1001 looks like we are good to merge?

pierrejeambrun

Thanks

boring-cyborg · 2026-03-06T15:44:06Z

Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions.

github-actions · 2026-03-06T15:45:10Z

Backport failed to create: v3-1-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
❌	v3-1-test

You can attempt to backport this manually by running:

cherry_picker a915216 v3-1-test

This should apply the commit to the v3-1-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

This PR addresses a significant performance issue in the /ui/dags endpoint where page load times scaled poorly with the number of DAGs (12-16 seconds for just 25 DAGs in our testing). Two optimizations are implemented: 1. Cache URLSafeSerializer for file_token generation - Previously, a new URLSafeSerializer was instantiated and conf.get_mandatory_value() was called for every DAG - Now uses @lru_cache to create the serializer once and reuse it 2. Eliminate redundant Pydantic validation in response construction - The original pattern used model_validate -> model_dump -> model_validate which caused triple serialization overhead per DAG - Now validates once with DAGResponse.model_validate(), then uses model_construct() to build DAGWithLatestDagRunsResponse Together, these changes reduced page load time from 12-16 seconds to ~130ms in our dev environment. Co-authored-by: Cursor <cursoragent@cursor.com> (cherry picked from commit a915216)

pierrejeambrun · 2026-03-06T16:28:48Z

Manual backport #63001

This PR addresses a significant performance issue in the /ui/dags endpoint where page load times scaled poorly with the number of DAGs (12-16 seconds for just 25 DAGs in our testing). Two optimizations are implemented: 1. Cache URLSafeSerializer for file_token generation - Previously, a new URLSafeSerializer was instantiated and conf.get_mandatory_value() was called for every DAG - Now uses @lru_cache to create the serializer once and reuse it 2. Eliminate redundant Pydantic validation in response construction - The original pattern used model_validate -> model_dump -> model_validate which caused triple serialization overhead per DAG - Now validates once with DAGResponse.model_validate(), then uses model_construct() to build DAGWithLatestDagRunsResponse Together, these changes reduced page load time from 12-16 seconds to ~130ms in our dev environment. (cherry picked from commit a915216) Co-authored-by: john-rodriguez-mgni <107643943+john-rodriguez-mgni@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: Unhandled Exception in remote logging if connection doesn't exist(#59801) (#62979) Cherry-picked from 3428dc9 with conflict resolution: - context.py: Added `import inspect` (skip `import functools` as `from functools import cache` already exists) - supervisor.py: Adopted early-return pattern and explicit `del` for GC, kept simpler env var handling (no `_AIRFLOW_PROCESS_CONTEXT` which doesn't exist in v3-1-test) - test_supervisor.py: Replaced `@pytest.mark.xfail` workaround with proper `use_real_secrets_backends` fixture Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * CI: Upgrade important CI environment (#62981) * chore(deps-dev): bump the core-ui-package-updates group across 1 directory with 3 updates (#62968) Bumps the core-ui-package-updates group with 1 update in the /airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui directory: [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin). Updates `@typescript-eslint/eslint-plugin` from 8.50.0 to 8.56.1 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.56.1/packages/eslint-plugin) Updates `@typescript-eslint/parser` from 8.50.0 to 8.56.1 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.56.1/packages/parser) Updates `ts-morph` from 23.0.0 to 27.0.2 - [Release notes](https://github.com/dsherret/ts-morph/releases) - [Commits](dsherret/ts-morph@23.0.0...27.0.2) --- updated-dependencies: - dependency-name: "@typescript-eslint/eslint-plugin" dependency-version: 8.56.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: core-ui-package-updates - dependency-name: "@typescript-eslint/parser" dependency-version: 8.56.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: core-ui-package-updates - dependency-name: ts-morph dependency-version: 27.0.2 dependency-type: direct:development update-type: version-update:semver-major dependency-group: core-ui-package-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] fix: gracefully handle 404 from worker log server for historical retry attempts (#62475) (#63000) * fix: gracefully handle 404 from worker log server for historical retry attempts (cherry picked from commit 25e9284) Co-authored-by: Pradeep Kalluri <128097794+kalluripradeep@users.noreply.github.com> * [v3-1-test] Remove issue_number option from newsfragment guidance in PR template (#63006) (#63012) The CI workflow added in #62975 validates that newsfragment filenames use the PR number, so allowing issue numbers would cause false CI failures. Align the PR template with the contributing docs and the new validation. (cherry picked from commit 41969f4) Co-authored-by: Xiaodong DENG <xddeng@apache.org> * [v3-1-test] CI: Upgrade important CI environment (#62989) (#63005) * [v3-1-test] CI: Upgrade important CI environment (#62989) (cherry picked from commit 60b52b7) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * Fix rebase --------- Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * [v3-1-test] Align integration LocalStack docker-compose with e2e LocalStack config (#62980) (#62993) (cherry picked from commit 6722c4b) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com> * [v3-1-test] Fallback to no constraint builds for docker-context-files installation (#63051) (#63057) When building PROD from docker-context-files - i.e. when we run main build with providers built from sources, we should fall back to no constraints build when there is a conflict with constraints. This is a follow up after #62378 (cherry picked from commit fef2e62) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * Fill Turkish Translation Gap in v3-1-test (#63010) * Remove global from FastAPI app.py (#59772) (#62997) * Remove global from FastAPI app.py * Remove global from FastAPI app.py Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com> * chore(deps): bump actions/setup-java from 4.7.1 to 5.2.0 (#63102) Bumps [actions/setup-java](https://github.com/actions/setup-java) from 4.7.1 to 5.2.0. - [Release notes](https://github.com/actions/setup-java/releases) - [Commits](actions/setup-java@c5195ef...be666c2) --- updated-dependencies: - dependency-name: actions/setup-java dependency-version: 5.2.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/stale from 9.1.0 to 10.2.0 (#63099) Bumps [actions/stale](https://github.com/actions/stale) from 9.1.0 to 10.2.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@5bef64f...b5d41d4) --- updated-dependencies: - dependency-name: actions/stale dependency-version: 10.2.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/checkout from 4.2.2 to 6.0.2 (#63096) Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.2 to 6.0.2. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@11bd719...de0fac2) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 6.0.2 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/setup-python from 5.6.0 to 6.2.0 (#63098) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.6.0 to 6.2.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@a26af69...a309ff8) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: 6.2.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/github-script from 7.0.1 to 8.0.0 (#63090) Bumps [actions/github-script](https://github.com/actions/github-script) from 7.0.1 to 8.0.0. - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@60a0d83...ed59741) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 8.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] Split npm dependabot minor+patch and major version PRs (#62889) (#63007) * Ignore major npm dependabot upgrades * Move major versions to different group (cherry picked from commit a2e3613) Co-authored-by: Brent Bovenzi <brent@astronomer.io> * chore(deps): bump the core-ui-package-updates group across 1 directory with 2 updates (#63069) Bumps the core-ui-package-updates group with 2 updates in the /airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui directory: [@chakra-ui/react](https://github.com/chakra-ui/chakra-ui/tree/HEAD/packages/react) and [happy-dom](https://github.com/capricorn86/happy-dom). Updates `@chakra-ui/react` from 3.33.0 to 3.34.0 - [Release notes](https://github.com/chakra-ui/chakra-ui/releases) - [Changelog](https://github.com/chakra-ui/chakra-ui/blob/main/packages/react/CHANGELOG.md) - [Commits](https://github.com/chakra-ui/chakra-ui/commits/@chakra-ui/react@3.34.0/packages/react) Updates `happy-dom` from 20.7.0 to 20.8.3 - [Release notes](https://github.com/capricorn86/happy-dom/releases) - [Commits](capricorn86/happy-dom@v20.7.0...v20.8.3) --- updated-dependencies: - dependency-name: "@chakra-ui/react" dependency-version: 3.34.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: core-ui-package-updates - dependency-name: happy-dom dependency-version: 20.8.3 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: core-ui-package-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/upload-artifact from 4.6.2 to 7.0.0 (#63128) Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.6.2 to 7.0.0. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@ea165f8...bbbca2d) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-version: 7.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/setup-go from 5.5.0 to 6.3.0 (#63124) Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.5.0 to 6.3.0. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@d35c59a...4b73464) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: 6.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump aws-actions/configure-aws-credentials (#63123) Bumps [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials) from 4.0.1 to 6.0.0. - [Release notes](https://github.com/aws-actions/configure-aws-credentials/releases) - [Changelog](https://github.com/aws-actions/configure-aws-credentials/blob/main/CHANGELOG.md) - [Commits](aws-actions/configure-aws-credentials@010d0da...8df5847) --- updated-dependencies: - dependency-name: aws-actions/configure-aws-credentials dependency-version: 6.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/setup-node from 4.4.0 to 6.3.0 (#63119) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4.4.0 to 6.3.0. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@49933ea...53b8394) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: 6.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] chore(deps): bump actions/setup-go from 5.5.0 to 6.3.0 (#63133) (#63138) Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.5.0 to 6.3.0. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@d35c59a...4b73464) (cherry picked from commit 274c2b0) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: 6.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] chore(deps): bump actions/download-artifact from 4.3.0 to 8.0.0 (#63065) (#63145) Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4.3.0 to 8.0.0. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@d3f86a1...70fc10c) (cherry picked from commit 2f4646c) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-version: 8.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump the github-actions-updates group with 2 updates (#63157) Bumps the github-actions-updates group with 2 updates: [pnpm/action-setup](https://github.com/pnpm/action-setup) and [slackapi/slack-github-action](https://github.com/slackapi/slack-github-action). Updates `pnpm/action-setup` from 4.0.0 to 4.2.0 - [Release notes](https://github.com/pnpm/action-setup/releases) - [Commits](pnpm/action-setup@fe02b34...41ff726) Updates `slackapi/slack-github-action` from 2.0.0 to 2.1.1 - [Release notes](https://github.com/slackapi/slack-github-action/releases) - [Commits](slackapi/slack-github-action@485a9d4...91efab1) --- updated-dependencies: - dependency-name: pnpm/action-setup dependency-version: 4.2.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions-updates - dependency-name: slackapi/slack-github-action dependency-version: 2.1.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] perf: use load_only() in get_dag_runs eager loading to reduce data fetched per task instance (#62482) (#62996) * perf: use load_only() in eager_load_dag_run_for_validation to reduce data fetched The get_dag_runs API endpoint was slow on large deployments because eager_load_dag_run_for_validation() used selectinload on task_instances and task_instances_histories without restricting which columns were fetched. This caused SQLAlchemy to load all heavyweight columns (executor_config with pickled data, hostname, rendered fields, etc.) for every task instance across every DAG run in the result page — even though only dag_version_id is needed to traverse the association proxy to DagVersion. Add load_only(TaskInstance.dag_version_id) and load_only(TaskInstanceHistory.dag_version_id) to the selectinload chains so the SELECT for task instances fetches only the identity columns and the FK needed to resolve the dag_version relationship, significantly reducing the volume of data transferred from the database on busy deployments. Fixes #62025 * Fix static checks --------- (cherry picked from commit 13af96b) Co-authored-by: Lakshmi Sravya <38032391+LakshmiSravyaVedantham@users.noreply.github.com> Co-authored-by: pierrejeambrun <pierrejbrun@gmail.com> * perf(api): optimize /ui/dags endpoint serialization (#61483) (#63001) This PR addresses a significant performance issue in the /ui/dags endpoint where page load times scaled poorly with the number of DAGs (12-16 seconds for just 25 DAGs in our testing). Two optimizations are implemented: 1. Cache URLSafeSerializer for file_token generation - Previously, a new URLSafeSerializer was instantiated and conf.get_mandatory_value() was called for every DAG - Now uses @lru_cache to create the serializer once and reuse it 2. Eliminate redundant Pydantic validation in response construction - The original pattern used model_validate -> model_dump -> model_validate which caused triple serialization overhead per DAG - Now validates once with DAGResponse.model_validate(), then uses model_construct() to build DAGWithLatestDagRunsResponse Together, these changes reduced page load time from 12-16 seconds to ~130ms in our dev environment. (cherry picked from commit a915216) Co-authored-by: john-rodriguez-mgni <107643943+john-rodriguez-mgni@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> * [v3-1-test] Bump astroid>=4 (#63170) (#63174) (cherry picked from commit 0a985ea) Co-authored-by: Elad Kalif <45845474+eladkal@users.noreply.github.com> * chore(deps): bump the core-ui-package-updates group across 1 directory with 2 updates (#63153) Bumps the core-ui-package-updates group with 1 update in the /airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui directory: [@chakra-ui/react](https://github.com/chakra-ui/chakra-ui/tree/HEAD/packages/react). Updates `@chakra-ui/react` from 3.33.0 to 3.34.0 - [Release notes](https://github.com/chakra-ui/chakra-ui/releases) - [Changelog](https://github.com/chakra-ui/chakra-ui/blob/main/packages/react/CHANGELOG.md) - [Commits](https://github.com/chakra-ui/chakra-ui/commits/@chakra-ui/react@3.34.0/packages/react) Updates `happy-dom` from 20.7.0 to 20.8.3 - [Release notes](https://github.com/capricorn86/happy-dom/releases) - [Commits](capricorn86/happy-dom@v20.7.0...v20.8.3) --- updated-dependencies: - dependency-name: "@chakra-ui/react" dependency-version: 3.34.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: core-ui-package-updates - dependency-name: happy-dom dependency-version: 20.8.3 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: core-ui-package-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] Further limit setuptools after 82.0.1 is released (until redoc fixes it) (#63202) (#63203) (cherry picked from commit b528e50) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * Fix race condition in auth manager initialization (#62431) (#62995) FAB FastAPI routes call get_application_builder() on every request, which creates a new Flask app and invokes init_app(). Concurrent calls race on the singleton auth_manager's appbuilder and security_manager, causing KeyError: 'AUTH_USER_REGISTRATION' and AttributeError. Add _init_app_lock around the critical section in init_app() that mutates the singleton auth_manager state and registers views, so concurrent get_application_builder() calls are serialized. Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com> * chore(deps): bump github/codeql-action (#63223) Bumps the github-actions-updates group with 1 update: [github/codeql-action](https://github.com/github/codeql-action). Updates `github/codeql-action` from 3.29.0 to 4.32.6 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@ce28f5b...0d579ff) --- updated-dependencies: - dependency-name: github/codeql-action dependency-version: 4.32.6 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] Clarify docs on max_active_tasks parameter on a Dag (#63217) (#63228) (cherry picked from commit 9ae1875) Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com> * [v3-1-test] Fix undefined variable in install_from_external_spec error message (#63233) (#63237) The error message in install_from_external_spec() referenced ${INSTALLATION_METHOD} which does not exist — the correct variable is ${AIRFLOW_INSTALLATION_METHOD}. With set -u active, hitting this error path would crash with an "unbound variable" error instead of printing the intended user-friendly message. The typo was introduced in a1717a6 and carried forward into the inlined copies in both Dockerfiles. (cherry picked from commit 2ab6f94) Co-authored-by: Xiaodong DENG <xddeng@apache.org> * Fix grid view URL for dynamic task groups (#63205) Dynamic task groups with isMapped=true were getting /mapped appended to their URL in the grid view, producing URLs like /tasks/group/{groupId}/mapped which has no matching route (404). The graph view correctly handles this by not appending /mapped for groups. This fix adds the same guard to buildTaskInstanceUrl. closes: #63197 (cherry picked from commit 7bc23ef) --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Dev-iL <6509619+Dev-iL@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Wei Lee <weilee.rx@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Pradeep Kalluri <128097794+kalluripradeep@users.noreply.github.com> Co-authored-by: Xiaodong DENG <xddeng@apache.org> Co-authored-by: Bugra Ozturk <bugraoz93@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com> Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com> Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com> Co-authored-by: Brent Bovenzi <brent@astronomer.io> Co-authored-by: Lakshmi Sravya <38032391+LakshmiSravyaVedantham@users.noreply.github.com> Co-authored-by: john-rodriguez-mgni <107643943+john-rodriguez-mgni@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Elad Kalif <45845474+eladkal@users.noreply.github.com> Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com> Co-authored-by: Subham <subhamsangwan26@gmail.com>

This PR addresses a significant performance issue in the /ui/dags endpoint where page load times scaled poorly with the number of DAGs (12-16 seconds for just 25 DAGs in our testing). Two optimizations are implemented: 1. Cache URLSafeSerializer for file_token generation - Previously, a new URLSafeSerializer was instantiated and conf.get_mandatory_value() was called for every DAG - Now uses @lru_cache to create the serializer once and reuse it 2. Eliminate redundant Pydantic validation in response construction - The original pattern used model_validate -> model_dump -> model_validate which caused triple serialization overhead per DAG - Now validates once with DAGResponse.model_validate(), then uses model_construct() to build DAGWithLatestDagRunsResponse Together, these changes reduced page load time from 12-16 seconds to ~130ms in our dev environment. Co-authored-by: Cursor <cursoragent@cursor.com>

This PR addresses a significant performance issue in the /ui/dags endpoint where page load times scaled poorly with the number of DAGs (12-16 seconds for just 25 DAGs in our testing). Two optimizations are implemented: 1. Cache URLSafeSerializer for file_token generation - Previously, a new URLSafeSerializer was instantiated and conf.get_mandatory_value() was called for every DAG - Now uses @lru_cache to create the serializer once and reuse it 2. Eliminate redundant Pydantic validation in response construction - The original pattern used model_validate -> model_dump -> model_validate which caused triple serialization overhead per DAG - Now validates once with DAGResponse.model_validate(), then uses model_construct() to build DAGWithLatestDagRunsResponse Together, these changes reduced page load time from 12-16 seconds to ~130ms in our dev environment. (cherry picked from commit a915216) Co-authored-by: john-rodriguez-mgni <107643943+john-rodriguez-mgni@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com>

* fix: Unhandled Exception in remote logging if connection doesn't exist(#59801) (#62979) Cherry-picked from 3428dc9 with conflict resolution: - context.py: Added `import inspect` (skip `import functools` as `from functools import cache` already exists) - supervisor.py: Adopted early-return pattern and explicit `del` for GC, kept simpler env var handling (no `_AIRFLOW_PROCESS_CONTEXT` which doesn't exist in v3-1-test) - test_supervisor.py: Replaced `@pytest.mark.xfail` workaround with proper `use_real_secrets_backends` fixture Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> * CI: Upgrade important CI environment (#62981) * chore(deps-dev): bump the core-ui-package-updates group across 1 directory with 3 updates (#62968) Bumps the core-ui-package-updates group with 1 update in the /airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui directory: [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin). Updates `@typescript-eslint/eslint-plugin` from 8.50.0 to 8.56.1 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.56.1/packages/eslint-plugin) Updates `@typescript-eslint/parser` from 8.50.0 to 8.56.1 - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v8.56.1/packages/parser) Updates `ts-morph` from 23.0.0 to 27.0.2 - [Release notes](https://github.com/dsherret/ts-morph/releases) - [Commits](dsherret/ts-morph@23.0.0...27.0.2) --- updated-dependencies: - dependency-name: "@typescript-eslint/eslint-plugin" dependency-version: 8.56.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: core-ui-package-updates - dependency-name: "@typescript-eslint/parser" dependency-version: 8.56.1 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: core-ui-package-updates - dependency-name: ts-morph dependency-version: 27.0.2 dependency-type: direct:development update-type: version-update:semver-major dependency-group: core-ui-package-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] fix: gracefully handle 404 from worker log server for historical retry attempts (#62475) (#63000) * fix: gracefully handle 404 from worker log server for historical retry attempts (cherry picked from commit 25e9284) Co-authored-by: Pradeep Kalluri <128097794+kalluripradeep@users.noreply.github.com> * [v3-1-test] Remove issue_number option from newsfragment guidance in PR template (#63006) (#63012) The CI workflow added in #62975 validates that newsfragment filenames use the PR number, so allowing issue numbers would cause false CI failures. Align the PR template with the contributing docs and the new validation. (cherry picked from commit 41969f4) Co-authored-by: Xiaodong DENG <xddeng@apache.org> * [v3-1-test] CI: Upgrade important CI environment (#62989) (#63005) * [v3-1-test] CI: Upgrade important CI environment (#62989) (cherry picked from commit 60b52b7) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * Fix rebase --------- Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * [v3-1-test] Align integration LocalStack docker-compose with e2e LocalStack config (#62980) (#62993) (cherry picked from commit 6722c4b) Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com> * [v3-1-test] Fallback to no constraint builds for docker-context-files installation (#63051) (#63057) When building PROD from docker-context-files - i.e. when we run main build with providers built from sources, we should fall back to no constraints build when there is a conflict with constraints. This is a follow up after #62378 (cherry picked from commit fef2e62) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * Fill Turkish Translation Gap in v3-1-test (#63010) * Remove global from FastAPI app.py (#59772) (#62997) * Remove global from FastAPI app.py * Remove global from FastAPI app.py Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com> * chore(deps): bump actions/setup-java from 4.7.1 to 5.2.0 (#63102) Bumps [actions/setup-java](https://github.com/actions/setup-java) from 4.7.1 to 5.2.0. - [Release notes](https://github.com/actions/setup-java/releases) - [Commits](actions/setup-java@c5195ef...be666c2) --- updated-dependencies: - dependency-name: actions/setup-java dependency-version: 5.2.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/stale from 9.1.0 to 10.2.0 (#63099) Bumps [actions/stale](https://github.com/actions/stale) from 9.1.0 to 10.2.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@5bef64f...b5d41d4) --- updated-dependencies: - dependency-name: actions/stale dependency-version: 10.2.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/checkout from 4.2.2 to 6.0.2 (#63096) Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.2 to 6.0.2. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@11bd719...de0fac2) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: 6.0.2 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/setup-python from 5.6.0 to 6.2.0 (#63098) Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.6.0 to 6.2.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@a26af69...a309ff8) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: 6.2.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/github-script from 7.0.1 to 8.0.0 (#63090) Bumps [actions/github-script](https://github.com/actions/github-script) from 7.0.1 to 8.0.0. - [Release notes](https://github.com/actions/github-script/releases) - [Commits](actions/github-script@60a0d83...ed59741) --- updated-dependencies: - dependency-name: actions/github-script dependency-version: 8.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] Split npm dependabot minor+patch and major version PRs (#62889) (#63007) * Ignore major npm dependabot upgrades * Move major versions to different group (cherry picked from commit a2e3613) Co-authored-by: Brent Bovenzi <brent@astronomer.io> * chore(deps): bump the core-ui-package-updates group across 1 directory with 2 updates (#63069) Bumps the core-ui-package-updates group with 2 updates in the /airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui directory: [@chakra-ui/react](https://github.com/chakra-ui/chakra-ui/tree/HEAD/packages/react) and [happy-dom](https://github.com/capricorn86/happy-dom). Updates `@chakra-ui/react` from 3.33.0 to 3.34.0 - [Release notes](https://github.com/chakra-ui/chakra-ui/releases) - [Changelog](https://github.com/chakra-ui/chakra-ui/blob/main/packages/react/CHANGELOG.md) - [Commits](https://github.com/chakra-ui/chakra-ui/commits/@chakra-ui/react@3.34.0/packages/react) Updates `happy-dom` from 20.7.0 to 20.8.3 - [Release notes](https://github.com/capricorn86/happy-dom/releases) - [Commits](capricorn86/happy-dom@v20.7.0...v20.8.3) --- updated-dependencies: - dependency-name: "@chakra-ui/react" dependency-version: 3.34.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: core-ui-package-updates - dependency-name: happy-dom dependency-version: 20.8.3 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: core-ui-package-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/upload-artifact from 4.6.2 to 7.0.0 (#63128) Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.6.2 to 7.0.0. - [Release notes](https://github.com/actions/upload-artifact/releases) - [Commits](actions/upload-artifact@ea165f8...bbbca2d) --- updated-dependencies: - dependency-name: actions/upload-artifact dependency-version: 7.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/setup-go from 5.5.0 to 6.3.0 (#63124) Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.5.0 to 6.3.0. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@d35c59a...4b73464) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: 6.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump aws-actions/configure-aws-credentials (#63123) Bumps [aws-actions/configure-aws-credentials](https://github.com/aws-actions/configure-aws-credentials) from 4.0.1 to 6.0.0. - [Release notes](https://github.com/aws-actions/configure-aws-credentials/releases) - [Changelog](https://github.com/aws-actions/configure-aws-credentials/blob/main/CHANGELOG.md) - [Commits](aws-actions/configure-aws-credentials@010d0da...8df5847) --- updated-dependencies: - dependency-name: aws-actions/configure-aws-credentials dependency-version: 6.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/setup-node from 4.4.0 to 6.3.0 (#63119) Bumps [actions/setup-node](https://github.com/actions/setup-node) from 4.4.0 to 6.3.0. - [Release notes](https://github.com/actions/setup-node/releases) - [Commits](actions/setup-node@49933ea...53b8394) --- updated-dependencies: - dependency-name: actions/setup-node dependency-version: 6.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] chore(deps): bump actions/setup-go from 5.5.0 to 6.3.0 (#63133) (#63138) Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5.5.0 to 6.3.0. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@d35c59a...4b73464) (cherry picked from commit 274c2b0) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: 6.3.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] chore(deps): bump actions/download-artifact from 4.3.0 to 8.0.0 (#63065) (#63145) Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4.3.0 to 8.0.0. - [Release notes](https://github.com/actions/download-artifact/releases) - [Commits](actions/download-artifact@d3f86a1...70fc10c) (cherry picked from commit 2f4646c) --- updated-dependencies: - dependency-name: actions/download-artifact dependency-version: 8.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump the github-actions-updates group with 2 updates (#63157) Bumps the github-actions-updates group with 2 updates: [pnpm/action-setup](https://github.com/pnpm/action-setup) and [slackapi/slack-github-action](https://github.com/slackapi/slack-github-action). Updates `pnpm/action-setup` from 4.0.0 to 4.2.0 - [Release notes](https://github.com/pnpm/action-setup/releases) - [Commits](pnpm/action-setup@fe02b34...41ff726) Updates `slackapi/slack-github-action` from 2.0.0 to 2.1.1 - [Release notes](https://github.com/slackapi/slack-github-action/releases) - [Commits](slackapi/slack-github-action@485a9d4...91efab1) --- updated-dependencies: - dependency-name: pnpm/action-setup dependency-version: 4.2.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions-updates - dependency-name: slackapi/slack-github-action dependency-version: 2.1.1 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] perf: use load_only() in get_dag_runs eager loading to reduce data fetched per task instance (#62482) (#62996) * perf: use load_only() in eager_load_dag_run_for_validation to reduce data fetched The get_dag_runs API endpoint was slow on large deployments because eager_load_dag_run_for_validation() used selectinload on task_instances and task_instances_histories without restricting which columns were fetched. This caused SQLAlchemy to load all heavyweight columns (executor_config with pickled data, hostname, rendered fields, etc.) for every task instance across every DAG run in the result page — even though only dag_version_id is needed to traverse the association proxy to DagVersion. Add load_only(TaskInstance.dag_version_id) and load_only(TaskInstanceHistory.dag_version_id) to the selectinload chains so the SELECT for task instances fetches only the identity columns and the FK needed to resolve the dag_version relationship, significantly reducing the volume of data transferred from the database on busy deployments. Fixes #62025 * Fix static checks --------- (cherry picked from commit 13af96b) Co-authored-by: Lakshmi Sravya <38032391+LakshmiSravyaVedantham@users.noreply.github.com> Co-authored-by: pierrejeambrun <pierrejbrun@gmail.com> * perf(api): optimize /ui/dags endpoint serialization (#61483) (#63001) This PR addresses a significant performance issue in the /ui/dags endpoint where page load times scaled poorly with the number of DAGs (12-16 seconds for just 25 DAGs in our testing). Two optimizations are implemented: 1. Cache URLSafeSerializer for file_token generation - Previously, a new URLSafeSerializer was instantiated and conf.get_mandatory_value() was called for every DAG - Now uses @lru_cache to create the serializer once and reuse it 2. Eliminate redundant Pydantic validation in response construction - The original pattern used model_validate -> model_dump -> model_validate which caused triple serialization overhead per DAG - Now validates once with DAGResponse.model_validate(), then uses model_construct() to build DAGWithLatestDagRunsResponse Together, these changes reduced page load time from 12-16 seconds to ~130ms in our dev environment. (cherry picked from commit a915216) Co-authored-by: john-rodriguez-mgni <107643943+john-rodriguez-mgni@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> * [v3-1-test] Bump astroid>=4 (#63170) (#63174) (cherry picked from commit 0a985ea) Co-authored-by: Elad Kalif <45845474+eladkal@users.noreply.github.com> * chore(deps): bump the core-ui-package-updates group across 1 directory with 2 updates (#63153) Bumps the core-ui-package-updates group with 1 update in the /airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui directory: [@chakra-ui/react](https://github.com/chakra-ui/chakra-ui/tree/HEAD/packages/react). Updates `@chakra-ui/react` from 3.33.0 to 3.34.0 - [Release notes](https://github.com/chakra-ui/chakra-ui/releases) - [Changelog](https://github.com/chakra-ui/chakra-ui/blob/main/packages/react/CHANGELOG.md) - [Commits](https://github.com/chakra-ui/chakra-ui/commits/@chakra-ui/react@3.34.0/packages/react) Updates `happy-dom` from 20.7.0 to 20.8.3 - [Release notes](https://github.com/capricorn86/happy-dom/releases) - [Commits](capricorn86/happy-dom@v20.7.0...v20.8.3) --- updated-dependencies: - dependency-name: "@chakra-ui/react" dependency-version: 3.34.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: core-ui-package-updates - dependency-name: happy-dom dependency-version: 20.8.3 dependency-type: direct:development update-type: version-update:semver-minor dependency-group: core-ui-package-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] Further limit setuptools after 82.0.1 is released (until redoc fixes it) (#63202) (#63203) (cherry picked from commit b528e50) Co-authored-by: Jarek Potiuk <jarek@potiuk.com> * Fix race condition in auth manager initialization (#62431) (#62995) FAB FastAPI routes call get_application_builder() on every request, which creates a new Flask app and invokes init_app(). Concurrent calls race on the singleton auth_manager's appbuilder and security_manager, causing KeyError: 'AUTH_USER_REGISTRATION' and AttributeError. Add _init_app_lock around the critical section in init_app() that mutates the singleton auth_manager state and registers views, so concurrent get_application_builder() calls are serialized. Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com> * chore(deps): bump github/codeql-action (#63223) Bumps the github-actions-updates group with 1 update: [github/codeql-action](https://github.com/github/codeql-action). Updates `github/codeql-action` from 3.29.0 to 4.32.6 - [Release notes](https://github.com/github/codeql-action/releases) - [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md) - [Commits](github/codeql-action@ce28f5b...0d579ff) --- updated-dependencies: - dependency-name: github/codeql-action dependency-version: 4.32.6 dependency-type: direct:production update-type: version-update:semver-major dependency-group: github-actions-updates ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [v3-1-test] Clarify docs on max_active_tasks parameter on a Dag (#63217) (#63228) (cherry picked from commit 9ae1875) Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com> * [v3-1-test] Fix undefined variable in install_from_external_spec error message (#63233) (#63237) The error message in install_from_external_spec() referenced ${INSTALLATION_METHOD} which does not exist — the correct variable is ${AIRFLOW_INSTALLATION_METHOD}. With set -u active, hitting this error path would crash with an "unbound variable" error instead of printing the intended user-friendly message. The typo was introduced in a1717a6 and carried forward into the inlined copies in both Dockerfiles. (cherry picked from commit 2ab6f94) Co-authored-by: Xiaodong DENG <xddeng@apache.org> * Fix grid view URL for dynamic task groups (#63205) Dynamic task groups with isMapped=true were getting /mapped appended to their URL in the grid view, producing URLs like /tasks/group/{groupId}/mapped which has no matching route (404). The graph view correctly handles this by not appending /mapped for groups. This fix adds the same guard to buildTaskInstanceUrl. closes: #63197 (cherry picked from commit 7bc23ef) --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Dev-iL <6509619+Dev-iL@users.noreply.github.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Wei Lee <weilee.rx@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Pradeep Kalluri <128097794+kalluripradeep@users.noreply.github.com> Co-authored-by: Xiaodong DENG <xddeng@apache.org> Co-authored-by: Bugra Ozturk <bugraoz93@users.noreply.github.com> Co-authored-by: Jarek Potiuk <jarek@potiuk.com> Co-authored-by: Jason(Zhe-You) Liu <68415893+jason810496@users.noreply.github.com> Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com> Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com> Co-authored-by: Brent Bovenzi <brent@astronomer.io> Co-authored-by: Lakshmi Sravya <38032391+LakshmiSravyaVedantham@users.noreply.github.com> Co-authored-by: john-rodriguez-mgni <107643943+john-rodriguez-mgni@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: Elad Kalif <45845474+eladkal@users.noreply.github.com> Co-authored-by: Young-Ki Kim <kimyoungi99@naver.com> Co-authored-by: Subham <subhamsangwan26@gmail.com>

john-rodriguez-mgni requested review from bugraoz93, choo121600, ephraimbuddy, jason810496, pierrejeambrun, rawwar and shubhamraj-git as code owners February 5, 2026 05:40

boring-cyborg bot added the area:API Airflow's REST/HTTP API label Feb 5, 2026

tirkarthi reviewed Feb 5, 2026

View reviewed changes

airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py Show resolved Hide resolved

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py Outdated Show resolved Hide resolved

john-rodriguez-mgni force-pushed the fix/dags-endpoint-performance branch from b8ac2cf to 1350bfa Compare February 5, 2026 16:25

pierrejeambrun reviewed Feb 9, 2026

View reviewed changes

airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py Show resolved Hide resolved

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py Outdated Show resolved Hide resolved

pierrejeambrun added this to the Airflow 3.1.8 milestone Feb 9, 2026

pierrejeambrun changed the title ~~perf(api): optimize /ui/dags endpoint serialization~~ Optimize /ui/dags endpoint serialization Feb 9, 2026

pierrejeambrun reviewed Feb 9, 2026

View reviewed changes

john-rodriguez-mgni force-pushed the fix/dags-endpoint-performance branch from 1350bfa to f1a3a5c Compare February 10, 2026 04:55

uranusjr reviewed Feb 10, 2026

View reviewed changes

airflow-core/src/airflow/api_fastapi/core_api/datamodels/dags.py Outdated Show resolved Hide resolved

uranusjr reviewed Feb 10, 2026

View reviewed changes

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py Outdated Show resolved Hide resolved

pierrejeambrun approved these changes Feb 11, 2026

View reviewed changes

pierrejeambrun added the backport-to-v3-1-test label Feb 11, 2026

john-rodriguez-mgni force-pushed the fix/dags-endpoint-performance branch from f1a3a5c to 153d4c9 Compare February 11, 2026 21:22

jason810496 reviewed Feb 12, 2026

View reviewed changes

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py Outdated Show resolved Hide resolved

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py Show resolved Hide resolved

airflow-core/src/airflow/api_fastapi/core_api/routes/ui/dags.py Outdated Show resolved Hide resolved

pierrejeambrun reviewed Feb 16, 2026

View reviewed changes

pierrejeambrun mentioned this pull request Feb 16, 2026

API - UI Optimization #62020

Open

john-rodriguez-mgni force-pushed the fix/dags-endpoint-performance branch from 153d4c9 to 86dd884 Compare February 16, 2026 18:16

pierrejeambrun reviewed Feb 23, 2026

View reviewed changes

pierrejeambrun force-pushed the fix/dags-endpoint-performance branch from 86dd884 to 460629c Compare February 23, 2026 16:23

pierrejeambrun reviewed Feb 24, 2026

View reviewed changes

john-rodriguez-mgni force-pushed the fix/dags-endpoint-performance branch from 460629c to fa124e2 Compare February 27, 2026 06:50

john-rodriguez-mgni force-pushed the fix/dags-endpoint-performance branch from fa124e2 to ee50924 Compare March 2, 2026 17:25

vatsrahul1001 modified the milestones: Airflow 3.1.8, Airflow 3.1.9 Mar 3, 2026

pierrejeambrun approved these changes Mar 6, 2026

View reviewed changes

pierrejeambrun merged commit a915216 into apache:main Mar 6, 2026
129 checks passed

Conversation

john-rodriguez-mgni commented Feb 5, 2026

Summary

1. Cache URLSafeSerializer for file_token generation

2. Eliminate redundant Pydantic validation in response construction

Results

Test Plan

Uh oh!

boring-cyborg bot commented Feb 5, 2026

Uh oh!

Uh oh!

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

jason810496 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

john-rodriguez-mgni commented Feb 12, 2026

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

john-rodriguez-mgni commented Feb 16, 2026

Uh oh!

john-rodriguez-mgni commented Feb 18, 2026

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

john-rodriguez-mgni commented Feb 24, 2026

Uh oh!

john-rodriguez-mgni commented Feb 26, 2026

Uh oh!

john-rodriguez-mgni commented Feb 27, 2026

Uh oh!

john-rodriguez-mgni commented Mar 2, 2026

Uh oh!

john-rodriguez-mgni commented Mar 3, 2026

Uh oh!

pierrejeambrun left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

boring-cyborg bot commented Mar 6, 2026

Uh oh!

github-actions bot commented Mar 6, 2026

Backport failed to create: v3-1-test. View the failure log Run details

Uh oh!

pierrejeambrun commented Mar 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants