Method access of generic class has Unknown rather than generic type.

[draustin]

Describe the bug

In strict mode, method access on a generic class (not an instance) gives a type is partially unknown error. The problem arose for me calling __init__ of a generic superclass (example 1 below). But I found that it is not specific to __init__ - it happens with any method.

I tried supply the type as an argument i.e A[int].__init__. This passes type checking but doesn't actually call the method at runtime.

I would expect a (unbound) method to be treated similarly to a generic (non-method) function i.e. using the @ notation. My example 2 below illustrates the point.

Code or Screenshots

Example 1 - calling superclass __init__:

from typing import Generic, TypeVar

T = TypeVar("T")

class A(Generic[T]):
    def __init__(self, x:T):
        ...

class B(A[int]):
    def __init__(self, x:int):
        A.__init__(self, x) # Type of "__init__" is partially unknown

(Pyright playgrround)

Example 2 - inconsistency with (non-method) generic function:

from typing import Generic, TypeVar

T = TypeVar("T")

def g(a: T, x:T):
    ...

# Revealed type is "(variable) def g2(a: T@g, x: T@g) -> None".
g2 = g

class A(Generic[T]):
    def f(self, x: T):
        ...

# Partially unknown type.
# I would expect revealed type to be something like "f(self: A[T@f], x: T@f)".
f = A.f

Playground

VS Code extension or command-line
Pyright playgrround, strict mode, latest version.

[erictraut]

The current behavior is by design, so I don't consider this a bug.

The expression A.__init__ binds the __init__ method to the class A[Unknown]. If you want it to bind to A[int], you should use A[int].__init__.

The proper way to handle calls to a superclass __init__ — one that works with multiple inheritance — is to use a super() call, like this:

class B(A[int]):
    def __init__(self, x: int):
        super().__init__(x)

[draustin]

Thank you for the explanation. However I did not follow

The expression A.__init__ binds the __init__ method to the class A[Unknown].

What does "bind" mean here? (Clearly not the binding of a function to an instance to produce a bound method.) To my mind, the answer should follow from the runtime behavior. Accessing a regular (not @staticmethod or @classmethod) method of a class results in a regular (non-method) function. The signature of this function is the same as if it were declared at module scope rather than class scope, except with self annotated as the type of the enclosing class. Therefore, A.__init__ evaluates to a function with signature identical to:

def __init__(self: A[T], x: T):
    ...

If we accept the above, then why should Pyright treat a function defined in a generic class (i.e. A.__init__) differently from a generic function declared at module scope (e.g. g in my second example)? Both are generic functions. In isolation, the expressions A.__init__ and g have partially unknown type, but they both become known when their arguments are correctly given. More generally, any use of generics involves subexpressions of partially unknown type, but Pyright does not generally flag them.

Regarding your suggested workaround: I did try calling A[int].__init__(self, x), but found that the call does nothing at runtime. I searched around for a reference on the runtime behavior of subscripted generics but couldn't find anything except a cpython commit from 2017 suggesting that at runtime A[int] should evaluate to a proxy for the original class, and therefore my call should have worked as you suggested. Could this be a bug in Python?

Finally, the reason that I prefer to explicitly call the base class rather than use super() is that I wish to retain static type checking of __init__ arguments with multiple inheritance. The runtime result of super() is not statically determined. Pyright reports the type of super().__init__ as the type of the __init__ method of the first base class. The usual **kwargs practice for cooperative inheritance means forgoing type checks on the arguments of second and subsequent base classes. (The diamond inheritance problem that super() solves does not arise in my project because I use Protocol to define interface classes, but do not inherit from these interfaces.)

[erictraut]

"Binding" occurs when a method is accessed through a class or object via a . operator. At runtime, binding specifies the value of the self or cls parameter (for instance and class methods, respectively). At static analysis time, it specifies the type of self or cls which has the side effect of specifying the values of the class-scoped type variables.

For example:

Code sample in pyright playground

from typing import Generic, TypeVar

T = TypeVar("T")

class A(Generic[T]):
    def method(self, x: T) -> T:
        ...

reveal_type(A[int]().method)  # "(x: int) -> int"
reveal_type(A[str]().method)  # "(x: str) -> str"
reveal_type(A().method)  # "(x: Unknown) -> Unknown"
reveal_type(A[int].method)  # "(self: A[int], x: int) -> int"
reveal_type(A.method)  # "(self: A[Unknown], x: Unknown) -> Unknown"

[draustin]

At runtime, binding specifies the value of the self or cls parameter (for instance and class methods, respectively).

For an instance method accessed through an instance, or a class method accessed through an instance or a class, I agree. But this is not true for an instance method accessed through a class.

class A:
    # Define an instance method.
    def f(self):
        ...

# Access it through the class, and call it without arguments.
A.f() # TypeError: A.f() missing 1 required positional argument: 'self'

a = A()
# These 2 are equivalent.
A.f(a)
a.f()

I believe this is the crux of the issue. Unlike the other 3 cases, accessing an instance method through a class via the . operator does not bind the values of any arguments. The resulting function should therefore have the same static type as the original method.

[erictraut]

does not bind the values of any arguments

It doesn't bind any of the values to any arguments, but it does bind the type of the class. From a static typing standpoint, this is important as I show in my example above.

The typing spec is underspecified when it comes to binding rules. This is something I would like to see us standardize because there are differences across type checkers. I think that pyright's behavior is "more correct" than mypy's in this regard, so I will be pushing to make pyright's behavior the standard, but maybe someone can convince me otherwise. In any case, this will require a broader discussion to resolve. I don't plan to make any changes to pyright's current behavior until we reach a resolution in that discussion.

[draustin]

Thank you for the explanation. I completely understand and respect your decision to keep the current behavior until a standardization is achieved. I still don't quite follow what's meant by "bind the type of the class" but I'm happy to raise this in another forum. Thanks for your work on this fantastic tool.