Inconsistencies between `Type` and `type`

Typing
1

The types typing.Type and builtins.type are closely related. However, mypy and pyright treat them somewhat inconsistently (both internally to each type checker and across the two type checkers).

I’d like to fix the internal inconsistency in pyright, but before I make the change, I want to make sure my understanding of the spec is correct here.

PEP 484 says:

When Type is parameterized it requires exactly one parameter. Plain Type without brackets is equivalent to Type[Any] and this in turn is equivalent to type (the root of Python’s metaclass hierarchy). This equivalence also motivates the name, Type, as opposed to alternatives like Class or SubType, which were proposed while this feature was under discussion; this is similar to the relationship between e.g. List and list.

Regarding the behavior of Type[Any] (or Type or type), accessing attributes of a variable with this type only provides attributes and methods defined by type (for example, __repr__() and __mro__).

If I’m interpreting this correctly, type checkers should treat all of the following type annotations the same with regard to attribute access: Type, Type[Any], type, and type[Any].

Code sample in pyright playground
Code sample in mypy playground

from typing import Any, Type

def func1(t1: Type, t2: Type[Any], t3: type, t4: type[Any]):
    v1 = t1.x  # No Error
    v2 = t2.x  # Mypy: No Error, Pyright: Error *** Mismatch ***
    v3 = t3.x  # Error
    v4 = t4.x  # Mypy: No Error, Pyright: Error *** Mismatch ***

Pyright currently emits errors for v2, v3, v4
Mypy currently emits an error only for v3
Pyre currently emits errors for none of these cases

If I’m reading the spec correctly, an error should be emitted for all four. Does that make sense?

Mypy was the reference implementation for PEP 484, so I’m always cautious when my understanding of PEP 484 appears to differ from mypy’s behavior.

1 Like
2

The behavior here is indeed confusing and inconsistent. This would be a good area to address by the Typing Council as a spec clarification once the Council is constituted.

I think there are two things that type/type[Any] could mean:

  • An object that only allows operations that are valid on all types (analogous to object). With this interpretation, we should allow operations like t.__name__ that work on all types, but we should not allow accessing attributes like t.x that don’t exist on all types. Both mypy and pyright allow spelling this as type[object] or Type[object] (and so do pyanalyze and pytype, but pyre treats type and Type differently).
  • An object that is a type, but allows any operation that might be valid on some type (analogous to Any). This is how mypy treats type[Any]: for a variable t of type type[Any], t.__name__ is str (because that attribute is present for any type), but t.x is Any. I’ll call this “type[Any]”, though not all type checkers treat type[Any] this way.

For completeness, I’ll note that pyre, pytype, and pyanalyze all produce no errors on your code sample, though they differ in exactly what types they infer.

The text of PEP 484 seems to say that type/Type/Type[Any] should all be equivalent and mean type[object] as I defined it above. However, that’s not what any type checker has actually implemented.

Now, how should type checkers behave?

First, type and Type should mean exactly the same thing. That’s how all other pre-PEP 585 aliases behave, and it’s confusing for type to be different. That means both mypy and pyright have a bug in this area.

Next, type[Any] probably should have the behavior I outlined above for it (an object that is a type but also allows any operation that might be valid on some type). That’s how most type checkers except pyright treat it. I’m not sure how useful this behavior is, though, and it does create a weird entry in the type system, an “Any for types”.

Last, should plain type be equivalent to type[Any] or type[object]? The former is more consistent with the rest of the type system, and with the current text of PEP 484.

So I would say that your code sample should produce no errors: all of these annotations are equivalent to type[Any] and should allow arbitrary attributes.

4 Likes
3

Sorry for the mess. I believe I did the initial implementation of Type in mypy and it was beyond my understanding of the code base at the time.

I agree that type and Type should be equivalent to the extent possible – however, note that in expressions, type(x) and type(name, bases, dct) are allowed, while Type cannot be called/instantiated.

It would also be odd if type (used as a type annotation) didn’t mean the same as type[Any].

After t: type[C], the allowable attributes of t are the class attributes of C (and they have the corresponding types). This is a bit icky because some class attributes may not exist at runtime, e.g.:

class C:
    attr: int

Here, C.attr fails at runtime but mypy and pyright currently allow it. (There may be ickier examples using dataclasses.field?)

Apart from this corner case, it makes sense that after t: type[int], we can access t.numerator and it has the expected type, (int) -> int. So it also makes sense that type[Any] has all attributes.

IOW, it looks like I’m with Jelle, and none of these should be errors. I believe there’s still a use case for type (or type[Any]): an argument of that type requires passing a class object.

2 Likes
4

I largely agree but with one small difference. Any attribute that exists on all objects shouldn’t be treated as having a type of Any in this case. If it’s provided by type or object itself, that type info should remain in use, as the alternative would have required LSP-violating subclasses. Treating it this way allows type as an annotation to continue functioning as-is in stricter code bases that may have touched some of those attributes (such as __name__ as Jelle pointed out above) without losing the known type information.

__init__, __call__ and many other dunder methods intended for implementation should probably be excluded from being considered as inherited from object or type for the purposes of LSP as this is a consequence of implementation and not of the intent of the language to enforce a signature on creating all objects. (however, there are issues with ignoring these in user-provided classes), this is probably a good topic for the typing council to address.

5

Yeah, I agree. And that’s current behavior.

2 Likes
6

It seems that nobody disagrees with my comment above. Once the Typing Council/PEP 729 is in place, we should amend the spec to follow that comment (cross-reference Re-specify behavior of type/Type · Issue #9 · JelleZijlstra/typing-spec · GitHub).

1 Like
7

The proposal above makes sense to me. It’s internally consistent and defensible from a type perspective. However, I think there are alternative interpretations that are also consistent and defensible.

Before we move to amend the spec, I’d like to first implement the proposal in pyright and understand the implications for such a change for existing code bases. I haven’t had time to do this yet. It’s a pretty significant change, and I’ve been busy with more pressing issues.

1 Like
8

That makes sense, there’s no hurry. I think this will also be a good test case for the process for changing the spec (once there is a Typing Council). We can use the experience to figure out what kind of process works best.

2 Likes
9

Tangentially related, if we’re fleshing out the spec for type[T], could we also make it work for special types like unions and protocols? What the old TypeForm proposal was proposing. If memory serves, there was consensus that TypeForm would be unnecessary and type should just handle these, but it was some time ago.

1 Like
10

That’s a separate issue that deserves its own wider discussion. I don’t think there was consensus for what you’re saying, for what it’s worth, just a lack of energy to bring the TypeForm proposal to completion.

1 Like
11

Roger, makes sense. I brought it up since it’s an additional inconsistency between Pyright and Mypy (in Pyright this works from what I can tell, for unions at least).

12

I’ve implemented the proposal in pyright. It was thankfully easier to implement and had much less of an impact on type checking results than I expected. Here is the mypy_primer output, which shows that pyright users were using # type: ignore to work around its old behavior.

I agree with Jelle that the typing council should consider this proposal for formal inclusion in the typing spec.

Mypy would need to make a (hopefully minor) change to conform with the proposal.

3 Likes