Reapply "update AzureBaseHook to return credentials that supports get_token method"

[karunpoudel]

Reapply #52182

Databases like Snowflake and Azure Postgres Flex support connection using Azure OAUTH token. Newer Azure Identity credential objects like ClientSecretCredential and DefaultAzureCredential support get_token method which can be used to generate OAUTH token. SnowflakeHook and PostgresHook can leverage AzureBaseHook to retrieve required token.

In this PR:

• Expose the credential object directly using get_credential method and make sdk_client argument optional so that the credential object can be retrieved without needing sdk_client. eg. AzureBaseHook().get_credential()
• Added new extra use_azure_identity_object in connection for backward compatibility. When set to true, get_credential method will return newer credential object of type ClientSecretCredential or DefaultAzureCredential instead of current ServicePrincipalCredentials or AzureIdentityCredentialAdapter. Current credential object doesn't support get_token method. eg. AzureBaseHook().get_credential().get_token()

I am ok with not making sdk_client argument optional if there is any concern.

Usage:

Create base azure connection using either client_secret or default azure credential.

# Using client_secret
AIRFLOW_CONN_AZURE_SP=azure://client_id:client_secret@/?tenantId=xyz&use_azure_identity_object=true

# Using default azure credentials
AIRFLOW_CONN_AZURE_SP=azure:///?use_azure_identity_object=true

Use above connection to retrieve token to connect to specific resource (e.g., azure postgres flex server).

from airflow.models import Connection
conn = Connection.get_connection_from_secrets("azure_sp")
azure_base_hook= conn.get_hook()
scope = "https://ossrdbms-aad.database.windows.net/.default"
password = azure_base_hook.get_token(scope).token

PostgresHook could be updated to retrieve token using azure connection similar to how it currently uses aws connection.

---

^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in airflow-core/newsfragments.

[ashb]

Opening-and-closing to trigger a full test run

[ashb]

This probably needs rebasing to latest main

[karunpoudel]

the pr check failures doesn't seem to be related to the code changes

[jscheffl]

the pr check failures doesn't seem to be related to the code changes

Yeah seem un-realted. Let's wait a moment until main is green again and then have another re-base. Hope then it is working.

[karunpoudel]

most of the tests have passed (except for the one failing in main), so might be safe to merge this

[jscheffl]

Double-checked and tests on main are the same failing here as well. Therefore OK to merge in my view. Hope main is green asap again.