AIP-84 Bootstrap the new Rest API

[pierrejeambrun]

Body

After we decide on our tech stack. Let's get the UI REST API started.

The REST API should also be plugged into the new UI full react app.

MVP PR needs:

• research tech stack, (probably fastapi, what else?) => going with fast api
• Make pydantic v2 and fastapi as core dependencies for airflow 3, remove all v1 / v2 / none code (utils, test, CI). Pydantic v2 and FastAPI as core dependency #41857
• migrate one custom endpoint from views.py Setup ui rest api #41798
  • Let's start with next_run_datasets, its relatively simple and we will still need it
  • We will keep the old UI connected to the old views.py endpoint for now, and delete both later. Then we're not doing extra work just for backward compatibility
• Integrate the new webserver to the airflow CLI (maybe with a warning to discourage usage for now, also in airflow standalone) UI API CLI and Breeze integration #41896
• Integrate the new webserver to the Breeze workflow UI API CLI and Breeze integration #41896
• Fix Wrong file descriptor with Uvicorn async workers for Gunicorn. Check workers refresh. UI API CLI and Breeze integration #41896
• Add TLS to the API (moved to separate issue, cf note bellow)
• Handle permissions and access control, POC here UI Rest API authentication #42019 (moved to separate issue, cf note bellow)
• Tests + CI integrations UI API Add views tests #41957
• develop dev tools for the new API and for the UI to consume the new API (testing, autogeneration) Add openapi generation hook for UI API #42098 Switch UI codegen to fastAPI spec #42222
• contributor documentation Update UI contributing docs #41903 (brent did it)

Committer

• I acknowledge that I am a maintainer/committer of the Apache Airflow project.

[pierrejeambrun]

Done.

HTTPS and Permissions are moved to separate issues. Cf project board.