Cannot Read Logs from Azure Blob Storage

[ahipp13]

Apache Airflow Provider(s)

microsoft-azure

Versions of Apache Airflow Providers

apache-airflow-providers-microsoft-azure==5.0.0

Apache Airflow version

2.5.0

Operating System

Debian GNU/Linux 11 (bullseye)

Deployment

Other 3rd-party Helm chart

Deployment details

We are currently deploying Airflow 2.5.0 using the Community Helm Chart 8.6.1 (https://github.com/airflow-helm/charts/tree/main/charts/airflow) and are using an external postgres database.

What happened

We have remote logging setup to the Azure Blob by setting these values in our values.yaml file:

AIRFLOW__LOGGING__REMOTE_LOGGING: "True"
AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER: "wasb-airflow-logs"
AIRFLOW__LOGGING__REMOTE_LOG_CONN_ID: "az-conn"

When a DAG is run, the logs are getting sent to the blob storage successfully. But, when you try to look at them in the Airflow UI, this pops up:

*** Reading remote log from wasb-airflow-logs/dag_id=logcleanup_dag/run_id=manual__2023-01-14T00:37:43.847862+00:00/task_id=logcleanup_dag_logcleanup/attempt=1.log.
Could not read logs from wasb-airflow-logs/dag_id=logcleanup_dag/run_id=manual__2023-01-14T00:37:43.847862+00:00/task_id=logcleanup_dag_logcleanup/attempt=1.log

These are the logs of the web pod:

What you think should happen instead

The logs should display correctly in the web UI

How to reproduce

Using Airflow 2.5.0 and a helm chart, set up logging to an azure blob storage by setting these values in your values.yaml file:

AIRFLOW__LOGGING__REMOTE_LOGGING
AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER
AIRFLOW__LOGGING__REMOTE_LOG_CONN_ID

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Taragolis]

Reported: #28231
Fixed by: #28234
Provider Changelog: https://airflow.apache.org/docs/apache-airflow-providers-microsoft-azure/stable/index.html#changelog