Skip to content

Skip asset-change registration for tasks with no outlets#68687

Merged
dstandish merged 1 commit into
apache:mainfrom
astronomer:astro/perf-skip-empty-asset-registration
Jun 18, 2026
Merged

Skip asset-change registration for tasks with no outlets#68687
dstandish merged 1 commit into
apache:mainfrom
astronomer:astro/perf-skip-empty-asset-registration

Conversation

@dstandish

@dstandish dstandish commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

Summary

TaskInstance.register_asset_changes_in_db runs on every task-success state transition (from the execution API PATCH .../state handler). For a task that declares no outlet assets and emits no outlet events — the common case — the method still executed its full body, including an AssetModel SELECT whose IN (...) clauses are all empty, before ultimately doing nothing.

This adds an early return when both task_outlets and outlet_events are empty, eliminating that database round-trip. It sits on the task-completion path, which gates how quickly the next task can be scheduled.

Detail

  • airflow-core/src/airflow/models/taskinstance.py: early-return guard at the top of register_asset_changes_in_db.
  • All downstream work (payloads_by_asset, the AssetModel lookup, the per-key _register calls, and the alias path) derives from task_outlets / outlet_events, so when both are empty the method is already a no-op — this just makes that explicit and cheap.

Tests

Added test_register_asset_changes_in_db_no_outlets_is_a_noop, which asserts (via assert_queries_count(0)) that no queries are issued for a task with no outlets. Existing asset-registration / partition tests continue to pass.

TaskInstance.register_asset_changes_in_db runs on every task-success
state transition. For a task that declares no outlet assets and emits no
outlet events -- the common case -- the method still executed its body
and issued an AssetModel SELECT with empty IN () clauses before doing
nothing useful. Return early when both task_outlets and outlet_events are
empty, avoiding that round-trip on the hot path that gates scheduling the
next task.

Signed-off-by: Daniel Standish <daniel.standish@astronomer.io>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dstandish
dstandish marked this pull request as ready for review June 18, 2026 15:39
@dstandish
dstandish requested review from XD-DENG and ashb as code owners June 18, 2026 15:39
@dstandish
dstandish merged commit 59fc9e2 into apache:main Jun 18, 2026
78 checks passed
@dstandish
dstandish deleted the astro/perf-skip-empty-asset-registration branch June 18, 2026 15:40
@vatsrahul1001 vatsrahul1001 added this to the Airflow 3.3.0 milestone Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants