Skip to content

Resolving EMR deprecated warnings#39743

Merged
vincbeck merged 9 commits intoapache:mainfrom
dirrao:emr_deprecation_warning
May 23, 2024
Merged

Resolving EMR deprecated warnings#39743
vincbeck merged 9 commits intoapache:mainfrom
dirrao:emr_deprecation_warning

Conversation

@dirrao
Copy link
Copy Markdown
Contributor

@dirrao dirrao commented May 21, 2024

Resolving EMR deprecated warnings

related: #39485

@vincbeck
Copy link
Copy Markdown
Contributor

Static checks are failing

@ferruzzi
Copy link
Copy Markdown
Contributor

I'm not sure what this is meant to be fixing, but we definitely don't want to be adding waiter_delay and waiter_max_attempts to every operator in every system test...

Copy link
Copy Markdown
Contributor

@o-nikolas o-nikolas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also agree from the description and code I have no idea what issue is attempting to be fixed here. Can you provide more details to make your PR easier for others to review?

@dirrao
Copy link
Copy Markdown
Contributor Author

dirrao commented May 22, 2024

@o-nikolas and @ferruzzi
We have marked some of the fields as deprecated. To resolve warnings we have made these code changes.

As per the recommendation from @Taragolis, Fix the operator code or adjust the operator default values at the time of removing deprecated code.

[tests/system/providers/amazon/aws/example_emr.py](https://github.com/apache/airflow/tree/main/tests/system/providers/amazon/aws/example_emr.py)
tests/system/providers/amazon/aws/example_emr.py:143:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_check_interval_seconds has been deprecated to standardize naming conventions.  Please use waiter_delay instead.  In the future this will default to None and defer to the waiter's default value.
 [tests/system/providers/amazon/aws/example_emr_notebook_execution.py](https://github.com/apache/airflow/tree/main/tests/system/providers/amazon/aws/example_emr_notebook_execution.py)
tests/system/providers/amazon/aws/example_emr_notebook_execution.py:54:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_check_interval_seconds has been deprecated to standardize naming conventions.  Please use waiter_delay instead.  In the future this will default to None and defer to the waiter's default value.

tests/system/providers/amazon/aws/example_emr_notebook_execution.py:54:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_countdown has been deprecated to standardize naming conventions.  Please use waiter_max_attempts instead.  In the future this will default to None and defer to the waiter's default value.

tests/system/providers/amazon/aws/example_emr_notebook_execution.py:75:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_check_interval_seconds has been deprecated to standardize naming conventions.  Please use waiter_delay instead.  In the future this will default to None and defer to the waiter's default value.

tests/system/providers/amazon/aws/example_emr_notebook_execution.py:75:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_countdown has been deprecated to standardize naming conventions.  Please use waiter_max_attempts instead.  In the future this will default to None and defer to the waiter's default value.

tests/system/providers/amazon/aws/example_emr_notebook_execution.py:87:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_check_interval_seconds has been deprecated to standardize naming conventions.  Please use waiter_delay instead.  In the future this will default to None and defer to the waiter's default value.

tests/system/providers/amazon/aws/example_emr_notebook_execution.py:87:
   airflow.exceptions.AirflowProviderDeprecationWarning:The parameter waiter_countdown has been deprecated to standardize naming conventions.  Please use waiter_max_attempts instead.  In the future this will default to None and defer to the waiter's default value.

@dirrao dirrao force-pushed the emr_deprecation_warning branch from 4ed771e to 59866b0 Compare May 22, 2024 13:56
@dirrao dirrao force-pushed the emr_deprecation_warning branch from 96ede5a to 49b8855 Compare May 22, 2024 14:05
dirrao and others added 5 commits May 22, 2024 21:01
Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
Co-authored-by: Vincent <97131062+vincbeck@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@ferruzzi ferruzzi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, fixing the operator(s) makes much more sense than changing the tests.

@dirrao dirrao requested a review from Taragolis May 23, 2024 03:56
@vincbeck vincbeck merged commit e565cea into apache:main May 23, 2024
romsharon98 pushed a commit to romsharon98/airflow that referenced this pull request Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants