dbt==0.20.1 commands break on python 3.9.7

[tkoncz]

Many dbt commands (dbt run, dbt test, ...) are breaking on Python 3.9.7 released yesterday: https://docs.python.org/release/3.9.7/whatsnew/changelog.html.

Here is a traceback:

2021-09-01 14:55:10.133377 (MainThread): Running with dbt=0.20.1
2021-09-01 14:55:11.273707 (MainThread): Encountered an error:
2021-09-01 14:55:11.273998 (MainThread): Profile() takes no arguments
2021-09-01 14:55:11.275664 (MainThread): Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/dbt/main.py", line 125, in main
    results, succeeded = handle_and_check(args)
  File "/usr/local/lib/python3.9/site-packages/dbt/main.py", line 203, in handle_and_check
    task, res = run_from_args(parsed)
  File "/usr/local/lib/python3.9/site-packages/dbt/main.py", line 242, in run_from_args
    task = parsed.cls.from_args(args=parsed)
  File "/usr/local/lib/python3.9/site-packages/dbt/task/base.py", line 157, in from_args
    return super().from_args(args)
  File "/usr/local/lib/python3.9/site-packages/dbt/task/base.py", line 75, in from_args
    config = cls.ConfigType.from_args(args)
  File "/usr/local/lib/python3.9/site-packages/dbt/config/runtime.py", line 233, in from_args
    project, profile = cls.collect_parts(args)
  File "/usr/local/lib/python3.9/site-packages/dbt/config/runtime.py", line 211, in collect_parts
    profile = cls._get_rendered_profile(
  File "/usr/local/lib/python3.9/site-packages/dbt/config/runtime.py", line 190, in _get_rendered_profile
    return Profile.render_from_args(
  File "/usr/local/lib/python3.9/site-packages/dbt/config/profile.py", line 423, in render_from_args
    return cls.from_raw_profiles(
  File "/usr/local/lib/python3.9/site-packages/dbt/config/profile.py", line 388, in from_raw_profiles
    return cls.from_raw_profile_info(
  File "/usr/local/lib/python3.9/site-packages/dbt/config/profile.py", line 338, in from_raw_profile_info
    return cls.from_credentials(
  File "/usr/local/lib/python3.9/site-packages/dbt/config/profile.py", line 242, in from_credentials
    profile = cls(
TypeError: Profile() takes no arguments

For more details please see the following Slack thread: https://getdbt.slack.com/archives/C99SNSRTK/p1630506829031300

[shr1k]

Can confirm, I faced the same issue. I wonder if it's this change in the Python 3.9.7 release causing the breakage?

• bpo-44806: Non-protocol subclasses of typing.Protocol ignore now the init method inherited from protocol base classes. And the PR in question.

The Profile class in dbt ultimately inherits from a Protocol class, and I guess something's going wrong with the dataclass-generated __init__ there?

IANAPG (I Am Not A Python Guru) so I'm mostly just guessing though! 🤷‍♂️

[tnightengale]

Surprised there are not more people here... 👀

EDIT: Confirmed that pinning to Python 3.9.6 solved the issue. All credit to the linked Slack thread above!

[julianfortune]

I'm having the same issue in a different code base.

I feel confident the change linked in the issue body are the problem, but I don't understand the typing codebase enough to be certain.

I created this minimal example:

from dataclasses import dataclass
from typing import Protocol

class P(Protocol):
    pass

@dataclass
class B(P):
    value: str

print(B("test"))

In 3.9.7:

Traceback (most recent call last):
  File "test.py", line 11, in <module>
    print(B("test"))
TypeError: B() takes no arguments

In 3.9.6:

B(value='test')

[julianfortune]

Python Issue

[jtcohen6]

Thanks for jumping into this one, everyone! And thanks for opening the python issue, @julianfortune.

I'm not yet sure whether this represents:

• an unintended/unsupported mechanism of protocol inheritance/instantiation, in which case, we should refactor the places where we use Protocol (there's a lot of them)
• a regression in python 3.9.7 that will be fixed in a future patch release, at which point we should update python_requires to mark 3.9.7 as unsupported

[uriyyo]

As a temporary workaround you can patch dataclasses._set_new_attribute method:

if sys.version_info >= (3, 9, 7):
    import dataclasses

    original_set_new_attr = dataclasses._set_new_attribute


    def patched_set_new_attr(cls, name, value):
        if (
            name == '__init__'
            and any(getattr(c, '_is_protocol', False) for c in cls.__bases__)
            and cls.__init__ is object.__init__
        ):
            del cls.__init__

        return original_set_new_attr(cls, name, value)

    dataclasses._set_new_attribute = patched_set_new_attr

[julianfortune]

@uriyyo ’s cpython patch

[julianfortune]

@uriyyo Thank you for the workaround, the fix, and a new test case to prevent the regression in the future! 🎉