Amazon Provider v8.23.0 `importlib_metadata` module import error

[aiell0]

Apache Airflow Provider(s)

amazon

Versions of Apache Airflow Providers

Amazon provider v8.23.0

Apache Airflow version

2.7.0

Operating System

Linux

Deployment

Other Docker-based deployment

Deployment details

We are deploying into ECS Fargate with the following Dockerfile and requirements.txt files:

Dockerfile gist: https://gist.github.com/aiell0/a8100691966186f09a5237a480ecf55e
requirements.txt gist: https://gist.github.com/aiell0/59c180a1c14afa736610e7b0261060e4

What happened

Container failed to deploy. When we looked into the logs we found the following error:

ValueError: Cannot resolve 'airflow.providers.amazon.aws.log.s3_task_handler.S3TaskHandler': No module named 'importlib_metadata'

What you think should happen instead

Container/Task should have stood up successfully.

How to reproduce

Attempt to deploy a container via ECS using Apache Airflow v2.7 and v8.23 of the Amazon Provider.

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[eladkal]

Versions of Apache Airflow Providers
Amazon provider v8.23.0

Apache Airflow version
2.7.0

You claim using Amazon provider v8.23.0 and Airflow 2.7.0 but the docker gist you attached shows:

RUN python -m pip install apache-airflow==2.6.3 --constraint "https://github.com/apache/airflow/constraints-2.6.3/constraints-3.11.txt"
COPY requirements.txt /requirements.txt

[eladkal]

Possibly due to #39500
which added import importlib_metadata in airflow/providers/amazon/aws/utils/__init__.py

airflow/airflow/providers/amazon/aws/utils/__init__.py

Line 25 in 9284dc5

import importlib_metadata

where we should have

import sys

if sys.version_info >= (3, 10):
    from importlib import metadata
else:
    import importlib_metadata as metadata  # type: ignore[no-redef]
    

cc @ferruzzi @Taragolis WDYT?

@aiell0 if you will downgrade provider amazon version to 8.22.0 the issue should be resolved