Skip to content

Backport Docs setup for the Task SDK (#51153)#52682

Merged
kaxil merged 1 commit intoapache:v3-0-testfrom
astronomer:backport-ebfc0de-v3-0-test
Jul 2, 2025
Merged

Backport Docs setup for the Task SDK (#51153)#52682
kaxil merged 1 commit intoapache:v3-0-testfrom
astronomer:backport-ebfc0de-v3-0-test

Conversation

@sunank200
Copy link
Copy Markdown
Collaborator

This PR is a continuation of #47357

To enhance the experience (i.e., rather than just being given an alphabetical list of classes/functions), I have opted not to use the AutoAPI extension's auto-generate feature, allowing me to precisely control the order and grouping of the classes.

I have organized Task-SDK API into these sections:

  • Defining DAGs: DAG, dag
  • Decorators: task, task_group, setup, teardown
  • Bases: BaseOperator, BaseSensorOperator, BaseNotifier, BaseOperatorLink, XComArg
  • Connections & Variables: Connection, Variable
  • Tasks & Operators: TaskGroup, get_current_context, get_parsing_context, Param, chain, chain_linear, cross_downstream
  • Assets: Asset, AssetAlias, AssetAll, AssetAny, AssetWatcher, Metadata

I also ensured that all documented references use the top-level SDK path (e.g., airflow.sdk.DAG) instead of exposing the underlying module paths (e.g., airflow.sdk.definitions.dag.DAG).

I have incorporated a test that compares the items in the generated objects.inv with the elements I re-export from airflow/sdk/__init__.py.

The airflow/sdk/__init__.pyi type stub is unfortunately necessary, as without it (and with the imports solely within the if TYPE_CHECKING block), the re-exported classes were not appearing where I wanted them. Specifically, I want users to import items directly from airflow.sdk, not the submodule.

    stubgen -o task-sdk/src -m airflow.sdk

Test this with

 breeze build-docs task-sdk

or

uv run --group docs build-docs task-sdk

Screenshots
Screenshot 2025-06-02 at 6 15 40 PM Screenshot 2025-06-02 at 7 04 10 PM Screenshot 2025-06-02 at 6 44 01 PM Screenshot 2025-06-11 at 1 06 26 AM

closes: #43010
closes: #51283
(cherry picked from commit ebfc0de)


^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

This PR is a continuation of apache#47357

To enhance the experience (i.e., rather than just being given an alphabetical list of classes/functions), I have opted not to use the AutoAPI extension's auto-generate feature, allowing me to precisely control the order and grouping of the classes.

I have organized Task-SDK API into these sections:
* **Defining DAGs**: `DAG`, `dag`
* **Decorators**: `task`, `task_group`, `setup`, `teardown`
* **Bases**: `BaseOperator`, `BaseSensorOperator`, `BaseNotifier`, `BaseOperatorLink`, `XComArg`
* **Connections & Variables**: `Connection`, `Variable`
* **Tasks & Operators**: `TaskGroup`, `get_current_context`, `get_parsing_context`, `Param`, `chain`, `chain_linear`, `cross_downstream`
* **Assets**: `Asset`, `AssetAlias`, `AssetAll`, `AssetAny`, `AssetWatcher`, `Metadata`

I also ensured that all documented references use the top-level SDK path (e.g., `airflow.sdk.DAG`) instead of exposing the underlying module paths (e.g., `airflow.sdk.definitions.dag.DAG`).

I have incorporated a test that compares the items in the generated `objects.inv` with the elements I re-export from `airflow/sdk/__init__.py`.

The `airflow/sdk/__init__.pyi` type stub is unfortunately necessary, as without it (and with the imports solely within the `if TYPE_CHECKING` block), the re-exported classes were not appearing where I wanted them. Specifically, I want users to import items directly from `airflow.sdk`, not the submodule.

        stubgen -o task-sdk/src -m airflow.sdk

Test this with

     breeze build-docs task-sdk

or

    uv run --group docs build-docs task-sdk

**Screenshots**
<img width="1720" alt="Screenshot 2025-06-02 at 6 15 40 PM" src="https://github.com/user-attachments/assets/7ce06d9f-1af3-4156-acf7-642d5f37f907" />
<img width="1726" alt="Screenshot 2025-06-02 at 7 04 10 PM" src="https://github.com/user-attachments/assets/adfa99b3-1c40-4c37-b523-4d6ee27381b2" />
<img width="1707" alt="Screenshot 2025-06-02 at 6 44 01 PM" src="https://github.com/user-attachments/assets/d5ccbabc-dfbd-465d-ae32-3697acdce827" />
<img width="1728" alt="Screenshot 2025-06-11 at 1 06 26 AM" src="https://github.com/user-attachments/assets/c8a5792b-59ad-4855-8937-4877a31a9a55" />

closes: apache#43010
closes: apache#51283
(cherry picked from commit ebfc0de)
@kaxil kaxil merged commit 9afb410 into apache:v3-0-test Jul 2, 2025
142 of 148 checks passed
@kaxil kaxil deleted the backport-ebfc0de-v3-0-test branch July 2, 2025 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants