Skip to content

Move BaseSensorOperator to TaskSDK definitions#48244

Merged
kaxil merged 1 commit into
apache:mainfrom
astronomer:move-basop-sdk
Mar 28, 2025
Merged

Move BaseSensorOperator to TaskSDK definitions#48244
kaxil merged 1 commit into
apache:mainfrom
astronomer:move-basop-sdk

Conversation

@kaxil

@kaxil kaxil commented Mar 25, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

closes #45743

I am not touching Providers (to change import Path for BaseSensorOperator) in this PR since I might move BaseOperator and BaseSensorOperator under airflow.sdk.bases module since they aren't really needed for DAG Authors.

Future Work:

kaxil
kaxil commented Mar 25, 2025
View reviewed changes
Comment thread airflow-core/src/airflow/sensors/README.md
kaxil
kaxil commented Mar 25, 2025
View reviewed changes
Comment thread airflow-core/src/airflow/serialization/serialized_objects.py
kaxil
kaxil commented Mar 25, 2025
View reviewed changes
Comment thread task-sdk/src/airflow/sdk/definitions/sensors/base.py
@kaxil
kaxil force-pushed the move-basop-sdk branch 7 times, most recently from 5f7eb7f to e33ea1b Compare March 26, 2025 22:26
@kaxil
kaxil marked this pull request as ready for review March 26, 2025 22:26
uranusjr
uranusjr reviewed Mar 26, 2025
View reviewed changes
Comment thread task-sdk/src/airflow/sdk/definitions/baseoperator.py Outdated
@kaxil
kaxil requested a review from eladkal as a code owner March 26, 2025 23:54
uranusjr
uranusjr reviewed Mar 26, 2025
View reviewed changes
Comment thread task-sdk/tests/task_sdk/execution_time/test_task_runner.py
@kaxil
kaxil force-pushed the move-basop-sdk branch 2 times, most recently from 03e1959 to a22d544 Compare March 27, 2025 12:20
kaxil added a commit to astronomer/airflow that referenced this pull request Mar 27, 2025
@kaxil
Add a fixture to easily replace ti.run usage
dc96404 
As we are replacing BaseOperator usage from Core to Task SDK, we are running into several issues, one of the common one being over-usage of `task.run()`.

While some cases can be easily replaced by `task.execute()` others needs execution of the tasks, sharing of XCom's in between, checking task state, correct exception etc.

To make this easier I have added `run_task` fixture which I have been using in apache#48244 and it has worked out well.

Example:
@kaxil kaxil mentioned this pull request Mar 27, 2025
Merged
kaxil added a commit that referenced this pull request Mar 27, 2025
@kaxil
Add a fixture to easily replace ti.run usage (#48439)
80840cb 
As we are replacing BaseOperator usage from Core to Task SDK, we are running into several issues, one of the common one being over-usage of `task.run()`.

While some cases can be easily replaced by `task.execute()` others needs execution of the tasks, sharing of XCom's in between, checking task state, correct exception etc.

To make this easier I have added `run_task` fixture which I have been using in #48244 and it has worked out well.

Example:
@kaxil
kaxil requested a review from o-nikolas as a code owner March 27, 2025 17:14
@kaxil
kaxil force-pushed the move-basop-sdk branch 3 times, most recently from 2c5ee02 to 465594b Compare March 27, 2025 19:53
kaxil
kaxil commented Mar 27, 2025
View reviewed changes
Comment thread task-sdk/tests/conftest.py
kaxil
kaxil commented Mar 27, 2025
View reviewed changes
Comment thread providers/databricks/tests/unit/databricks/sensors/test_databricks_partition.py
kaxil
kaxil commented Mar 27, 2025
View reviewed changes
Comment thread providers/databricks/tests/unit/databricks/sensors/test_databricks_sql.py
@kaxil
kaxil force-pushed the move-basop-sdk branch 2 times, most recently from 7cdfa3c to cfed275 Compare March 27, 2025 21:00
amoghrajesh
amoghrajesh reviewed Mar 28, 2025
View reviewed changes

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not have any major concerns the way things have been done, a few nits here are there

Comment thread airflow-core/tests/unit/serialization/test_dag_serialization.py
Comment thread providers/databricks/tests/unit/databricks/sensors/test_databricks_partition.py
Comment thread providers/http/tests/unit/http/sensors/test_http.py
Comment thread task-sdk/src/airflow/sdk/definitions/baseoperator.py
Comment thread task-sdk/src/airflow/sdk/definitions/baseoperator.py
Comment thread task-sdk/src/airflow/sdk/execution_time/task_runner.py
amoghrajesh
amoghrajesh approved these changes Mar 28, 2025
View reviewed changes

@amoghrajesh amoghrajesh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missed approving!

@kaxil
kaxil merged commit 347c98c into apache:main Mar 28, 2025
@kaxil
kaxil deleted the move-basop-sdk branch March 28, 2025 07:58
pankajkoti pushed a commit to astronomer/airflow that referenced this pull request Mar 28, 2025
@kaxil @pankajkoti
Add a fixture to easily replace ti.run usage (apache#48439)
e2967b8 
As we are replacing BaseOperator usage from Core to Task SDK, we are running into several issues, one of the common one being over-usage of `task.run()`.

While some cases can be easily replaced by `task.execute()` others needs execution of the tasks, sharing of XCom's in between, checking task state, correct exception etc.

To make this easier I have added `run_task` fixture which I have been using in apache#48244 and it has worked out well.

Example:
pankajkoti pushed a commit to astronomer/airflow that referenced this pull request Mar 28, 2025
@kaxil @pankajkoti
Move BaseSensorOperator to TaskSDK definitions (apache#48244)
81618c6 
closes apache#45743
shubham-pyc pushed a commit to shubham-pyc/airflow that referenced this pull request Apr 2, 2025
@kaxil @shubham-pyc
Add a fixture to easily replace ti.run usage (apache#48439)
e349e1a 
As we are replacing BaseOperator usage from Core to Task SDK, we are running into several issues, one of the common one being over-usage of `task.run()`.

While some cases can be easily replaced by `task.execute()` others needs execution of the tasks, sharing of XCom's in between, checking task state, correct exception etc.

To make this easier I have added `run_task` fixture which I have been using in apache#48244 and it has worked out well.

Example:
shubham-pyc pushed a commit to shubham-pyc/airflow that referenced this pull request Apr 2, 2025
@kaxil @shubham-pyc
Move BaseSensorOperator to TaskSDK definitions (apache#48244)
4e4f9cc 
closes apache#45743
aaron-y-chen pushed a commit to aaron-y-chen/airflow that referenced this pull request Apr 4, 2025
@kaxil @aaron-y-chen
Add a fixture to easily replace ti.run usage (apache#48439)
776ccb7 
As we are replacing BaseOperator usage from Core to Task SDK, we are running into several issues, one of the common one being over-usage of `task.run()`.

While some cases can be easily replaced by `task.execute()` others needs execution of the tasks, sharing of XCom's in between, checking task state, correct exception etc.

To make this easier I have added `run_task` fixture which I have been using in apache#48244 and it has worked out well.

Example:
aaron-y-chen pushed a commit to aaron-y-chen/airflow that referenced this pull request Apr 4, 2025
@kaxil @aaron-y-chen
Move BaseSensorOperator to TaskSDK definitions (apache#48244)
c334e07 
closes apache#45743
@eladkal eladkal mentioned this pull request Apr 6, 2025
Closed
@uranusjr uranusjr mentioned this pull request Apr 8, 2025
Merged
simonprydden pushed a commit to simonprydden/airflow that referenced this pull request Apr 8, 2025
@kaxil Simon Prydden
Add a fixture to easily replace ti.run usage (apache#48439)
d8aa52e 
As we are replacing BaseOperator usage from Core to Task SDK, we are running into several issues, one of the common one being over-usage of `task.run()`.

While some cases can be easily replaced by `task.execute()` others needs execution of the tasks, sharing of XCom's in between, checking task state, correct exception etc.

To make this easier I have added `run_task` fixture which I have been using in apache#48244 and it has worked out well.

Example:
simonprydden pushed a commit to simonprydden/airflow that referenced this pull request Apr 8, 2025
@kaxil Simon Prydden
Move BaseSensorOperator to TaskSDK definitions (apache#48244)
0e7b9a8 
closes apache#45743
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request May 28, 2025
@kaxil
Add a fixture to easily replace ti.run usage (#48439)
2777170 
As we are replacing BaseOperator usage from Core to Task SDK, we are running into several issues, one of the common one being over-usage of `task.run()`.

While some cases can be easily replaced by `task.execute()` others needs execution of the tasks, sharing of XCom's in between, checking task state, correct exception etc.

To make this easier I have added `run_task` fixture which I have been using in apache/airflow#48244 and it has worked out well.

Example:
GitOrigin-RevId: 80840cb5734630f95d831dd10ae103c3df3a4b2f
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Sep 24, 2025
@kaxil
Add a fixture to easily replace ti.run usage (#48439)
64d07d0 
As we are replacing BaseOperator usage from Core to Task SDK, we are running into several issues, one of the common one being over-usage of `task.run()`.

While some cases can be easily replaced by `task.execute()` others needs execution of the tasks, sharing of XCom's in between, checking task state, correct exception etc.

To make this easier I have added `run_task` fixture which I have been using in apache/airflow#48244 and it has worked out well.

Example:
GitOrigin-RevId: 80840cb5734630f95d831dd10ae103c3df3a4b2f
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Oct 22, 2025
@kaxil
Add a fixture to easily replace ti.run usage (#48439)
bf486f8 
As we are replacing BaseOperator usage from Core to Task SDK, we are running into several issues, one of the common one being over-usage of `task.run()`.

While some cases can be easily replaced by `task.execute()` others needs execution of the tasks, sharing of XCom's in between, checking task state, correct exception etc.

To make this easier I have added `run_task` fixture which I have been using in apache/airflow#48244 and it has worked out well.

Example:
GitOrigin-RevId: 80840cb5734630f95d831dd10ae103c3df3a4b2f
kosteev pushed a commit to GoogleCloudPlatform/composer-airflow that referenced this pull request Feb 27, 2026
@kaxil
Add a fixture to easily replace ti.run usage (#48439)
4237ec8 
As we are replacing BaseOperator usage from Core to Task SDK, we are running into several issues, one of the common one being over-usage of `task.run()`.

While some cases can be easily replaced by `task.execute()` others needs execution of the tasks, sharing of XCom's in between, checking task state, correct exception etc.

To make this easier I have added `run_task` fixture which I have been using in apache/airflow#48244 and it has worked out well.

Example:
GitOrigin-RevId: 80840cb5734630f95d831dd10ae103c3df3a4b2f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@uranusjr uranusjr uranusjr left review comments
@amoghrajesh amoghrajesh amoghrajesh approved these changes
@ashb ashb Awaiting requested review from ashb ashb is a code owner
@XD-DENG XD-DENG Awaiting requested review from XD-DENG XD-DENG is a code owner
@eladkal eladkal Awaiting requested review from eladkal
@o-nikolas o-nikolas Awaiting requested review from o-nikolas o-nikolas is a code owner

Assignees

No one assigned

Labels

area:serialization area:task-sdk

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Move BaseSensor out of core and in to TaskSDK definitions

3 participants

@kaxil @uranusjr @amoghrajesh