Use set instead of list for dags' tags#41695
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
|
shahar1
left a comment
There was a problem hiding this comment.
Overall, I agree with the idea that set is more appropriate in this case, as tags should be unique. I'd be happy for more opinions whether it's worth making it as a breaking chage.
jscheffl
left a comment
There was a problem hiding this comment.
I like this cleanup! Especially as is is "non-breaking" for existing DAGs, so no effect on DAG authors but better data types. Just small proposals to pin the typing to strings
jscheffl
left a comment
There was a problem hiding this comment.
Some tests fail, otherwise OK for me. Small nit as comments
Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
|
I do not like this, but feel free to merge this without me if others feel more strongly this i beneficial. |
…it raises the error: Subscripted generics cannot be used with class and instance checks.
|
@uranusjr why you don't like this? |
|
I mostly outlined the reasons above, but to summarise, you want to keep this mutable so you can call With that said, 3.0 is the exact time to break everyone, so I am still open for this to be merged if people feel fine about it. But I will not approve nor merge this change myself. |
Would it make you more happy if we keep the interface as a list (so: non breaking) but during setting/init it is temporarily converted to a set to ensure no duplicates are in the list? |
|
@uranusjr I am sorry to hear you don't like my change. I think the solution of @jscheffl is reasonable to handle your conflict, even though I think we should break the interface to a But can you (@uranusjr) at least close your conversations? As it blockes the merge request (that was approved by other people). |
|
Awesome work, congrats on your first merged pull request! You are invited to check our Issue Tracker for additional contributions. |
* Started working on dag tags, moved the tags to set, and added atest to check duplications. * Fixed more areas of the c'tor. * Fixed test of dag tags. * Added a check to see if the tags are mutable. * Added newsfragment. * Removed unecessary check. * Removed str specification at the type, for compatability with python 3.8. * Removed more type specification as part of compatability with python 3.8 * Fixed the newsfragment. * Added missing word. * Used `` for code segemnts at the rst file. * Reformatted the file. * Fixed wrong method for adding tag. * Added type hinting at the dag bag. * Deserialized the tags to set. * Adjusted the tests for the set type. * Added type hinting. * Sorting the tags by name. * Changed to typing. * Update newsfragments/41420.significant.rst Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com> * Update newsfragments/41420.significant.rst Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com> * Removed the generic specification at the dag args expected types, as it raises the error: Subscripted generics cannot be used with class and instance checks. * Added tags to the expected serialized DAG. * Added sorting the tags keys by the name key. * Fixed sorting tags by name to use `sorted` instead of `.sort` * Fixed tags comparesion, as it's now a set, and not a list. --------- Co-authored-by: Jens Scheffler <95105677+jscheffl@users.noreply.github.com>
Closes issue 41420.
Thank you for viewing this PR :)
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rstor{issue_number}.significant.rst, in newsfragments.