PEP 742: Narrowing types with TypeIs

PEP 742 proposes to add a new special form to the type system to support user-defined type narrowing functions.

This proposal builds on PEP 724, which proposed to modify the semantics of the existing TypeGuard special form. While there was general agreement that a change to the type system was warranted, there was disagreement over the backwards compatibility implications of changing TypeGuard. As a result, the Typing Council failed to come to an agreement on PEP 724.

The alternative proposal in this new PEP avoids any backwards compatibility issues by instead creating a new special form and leaving TypeGuard unchanged.

The biggest open issue is the name. My hope is that the PEP’s new construct becomes the default tool used for user-defined type narrowing functions, leaving the existing TypeGuard as a more advanced tool used in rare cases. To make that happen, the new construct needs to have a clear, concise name. The PEP currently proposes the term TypeNarrower, which emphasizes that the new form always narrows types. Other names under consideration include:

  • TypeIs. Perhaps my favorite: short and reads well; def ...(...) -> TypeIs[int]: returns whether the type is int. But it’s unusual grammatically for a class.
  • TypeCheck. Guido likes this form. I feel it is too confusing with other uses of “type checking”.
  • Narrowed or NarrowedTo. Also links to *type narrowing" as a term, but more concise.
  • IsInstance. Emphasizes that the new construct has similar effects as isinstance. But the new construct is not exactly like isinstance.
  • StrictTypeGuard. This was in earlier drafts of PEP 724. It is too long and obscure; users would need an advanced understanding of typing to understand what “strict” means here.
  • Predicate or TypePredicate: Mirrors TypeScript’s terminology.

Other suggestions are welcome. I hope a clear consensus emerges; if so, I’ll update the PEP to use the new name.

There are a few areas that may need discussion:

  • I chose to give TypeNarrower the same specification as TypeGuard for all aspects that are not related to the type narrowing behavior, though some aspects of this specification are questionable (e.g., subtype relations with bool). I believe it is better for TypeGuard and TypeNarrower to behave as similarly as possible. Changes to these behaviors can be taken up separately by the Typing Council and applied to both TypeGuard and the new construct.
  • The new PEP, just like PEP 724, is intentionally vague about the exact semantics of type narrowing, since the theoretical answer relies on intersection types, which are not a formal part of the type system. Instead, the PEP recommends that type checkers use similar type narrowing techniques to what they use for isinstance(). There is ongoing work to prepare a PEP about intersection types. If and when such a PEP is accepted, it would provide a more precise definition for intersections.
14 Likes

I also like the name TypeIs, for the reasons you mentioned. I don’t think the usual objections to a verb name for a class really apply here, since it is a typing special form, not a class which is ever instantiated. There is already precedent for verb-named typing special forms (Unpack).

14 Likes

A post was split to a new topic: Today I learned bool is a subtype of int

From all this silence I’m concluding that TypeIs has consensus behind it. I will change the PEP to use this name.

I will still use “type narrowing function” as the technical term for a function returning TypeIs.

9 Likes

@Jelle Nice PEP, thank you for you work :slight_smile:

I have a comment re:

In the long run, most users should use TypeNarrower , and TypeGuard should be reserved for rare cases where its behavior is specifically desired.

What are those cases? How about including an example in the PEP itself?

1 Like

The name TypeIs sounds incompatible with the usual semantics of is. Adapted from an example from PEP 742 – Narrowing types with TypeNarrower | peps.python.org:

from typing import TypeIs, assert_type

def is_str(x: object) -> TypeIs[str]:
    return isinstance(x, str)

def f(x: str | int) -> None:
    if is_str(x):
        assert_type(x, str)
    else:
        assert_type(x, int)

class MyStr(str): pass

string_: MyStr = MyStr()
f(string_)  # No type-checker complaints, but what does this actually mean?

assert_type statically checks for type(string_) is str, not type(string_) <is a subtype of str>. So, while TypeIs[str] sounds compatible in this example, the fact that it works with isinstance(x, str) makes it not compatible.


To me, the name TypeIs in TypeIs[Typ] reads exactly as if the intention is to guard type(obj) is Typ or type(obj) in <sequence of types>. Type identity is important for any kind of code doing MRO analysis, so adopting the name TypeIs in PEP 742 makes it confusing in this context.

I’m more inclined towards IsInstance or something similar like IsSubtype.

6 Likes

As long as we’re still voting, I’m still in favor of TypeCheck, but I don’t care too much about the color of this bikeshed, and further discussion about what the meaning of “is” is doesn’t feel productive; if TypeCheck is rejected, TypeIs is my second choice.

I just posted a PEP update to greatly expand the “How to teach this” section, covering:

  • When to use TypeIs
  • How to write a safe TypeIs function
  • TypeIs versus TypeGuard (including more concrete guidance on when TypeGuard is the right tool to use)

The new section can be seen here:

I will merge it in to the main repo once people have had some time to give feedback.

6 Likes

For a function returning TypeIs[T] to be safe, it must return True if and only if the argument is compatible with type T , and False otherwise. If this condition is not met, the type checker may infer incorrect types.
[source]

Let’s consider the following type narrowing function:

def is_int(x: object) -> TypeIs[int]:
    return isinstance(x, float)

So replacing if isinstance(x, float): ... with if is_int(x): ... forces type checker to believe the is_int’s signature; which basically turns off an aspect of type checking. Two thoughts:

  1. How TypeIs works is somewhat similar to how typing.cast works. So how about calling it CastAs (or something in that vain) so it’s clear those are close? (I’m aware it’s already been renamed.)

  2. On the other hand, is there a way to force type checker to emit as error on incoherent narrowing functions that are simple enough, like is_int? For some other cases – like is_point (see here) – it would still be impossible. Not sure how to distinguish one from the other, though – type checker would probably not be able to on its own. Two forms: TypeIs and ForceTypeIs?


However, sometimes you want to reuse a more complicated check in multiple places, or you use a check that the type checker doesn’t understand.
[source]

So reusing “a more complicated check [that type checker does understand] in multiple places” with a single type narrowing function forces type checker not to not try to understand it any more. I don’t believe this is the cost I would be willing to pay in most cases.

I disagree, you could argue that TypeGuard works like typing.cast, but TypeIs is clearly different, since it can result in a more specific type.

The analogy for isinstance is much closer to the truth and TypeIs mirrors that, if you wanted to argue against TypeIs you could mention, that it could be mistaken for indicating type(foo) is T which would once again be closer to TypeGuard than what TypeIs does.

My comments are as (in)valid about TypeGuard as they are (in)valid about TypeIs. But since we are de facto fixing TypeGuard with TypeIs we can as well go further.

EDIT: I need to mention that I did not read any historical discussions about TypeGuard’s PEP, so those point might have been raised there. Sorry if they were.

They are not though, let me illustrate:

def is_float_guard(x: object) -> TypeGuard[int]:
    return isinstance(x, int)
    
def is_float(x: object) -> TypeIs[int]:
    return isinstance(x, int)
    
foo: int | str
if is_float(foo):
    reveal_type(foo)  # int
else:
    reveal_type(foo)  # str

if is_float_guard(foo):
    reveal_type(foo)  # float
else:
    reveal_type(foo)  # int | str
    
if some_complex_check:
    bar = cast(float, foo)
    reveal_type(bar)  # float
else:
    reveal_type(foo)  # int | str

As you can see TypeIs does something different from a cast, while TypeGuard does essentially the same. A cast sets the type exactly and has no effect on other code branches, whereas TypeIs can narrow to a more specific type than T through intersection and also narrow the other branch by excluding the type.

1 Like

I understand. I see why CastIs is not a good idea. Thank you for the thorough explanation.

Does TypeIs name convey the “overriding” behaviour? In this regard TypeNarrower (or even NarrowedTo, which also was proposed) might be a better name, after all.

I can agree with you on that, I would personally prefer TypeNarrower or Narrowed over TypeIs, since there is some ambiguity about type identity vs. A is a int, as in, it is some int (including subclasses), not THE int.

As for your 2nd point, If we were going to add a third type-guard-like construct I would prefer it to be this, which also guarantees safety:

With TypeIs vs ForceTypeIs it becomes just as easy to accidentally use one over the other as it would be to write an incorrect type narrower in the first place, so I don’t think it’s a big enough win personally.

2 Likes

The @inline_guards idea – interesting!

this is probably becoming OT, so hiding

With TypeIs vs ForceTypeIs it becomes just as easy to accidentally use one over the other as it would be to write an incorrect type narrower in the first place, so I don’t think it’s a big enough win personally.

TypeIs – which would make type checker to check the narrowing function’s internal consistency – would be the go-to (“default”) one. Only in cases where type checker cannot understand it the ForceTypeIs would be advised – “force” word would help to avoid the confusion, at least it would in my case.

Could type checker check if ForceTypeIs is redundant over TypeIs? In this case a warning could be emitted. It’s probably not that straightforward to implement, though.

Thanks for the responses.

As for the name, I’m going to stick with TypeIs unless a consensus for some other name emerges in this thread. Clearly lots of people have various preferences, but I’m not yet seeing any alternative that is as widely liked.

Could type checkers detect unsound TypeIs (and TypeGuard) functions? Maybe in a few cases, but I don’t expect this to be possible in general. If type checkers add the ability to detect some unsound narrowing functions, we could mention that in the user-facing documentation later. I certainly don’t want to prohibit type checkers from trying.

1 Like

I prefer IsInstance, but I can just write from typing import TypeIs as IsInstance so the name is not a big deal. The big deal is I really love this proposal, so thank you for making it.

I’m glad to see this successor to the “strict type guard” discussion being worked on again. :sparkles:

The Semantics

The proposed semantics I fully agree with. Specifically:

  • type NP should be narrowed to A∧R
  • type NN should be narrowed to A∧¬R

The Name

The biggest open issue is the name.

I’m leaning toward IsInstance[] as my first choice, since the semantics of the narrowing function are most similar to isinstance().

+1: This reading is also my primary hesitation around saying TypeIs[] or IsType[]. I’d rather say IsInstance[] (as I mentioned above) or IsSubtype[]. On the other hand I see the alignment with TypeScript’s type is T syntax, which seems fine when I read it in that context. Overall I’d say:

  • IsType[] (in question form) is my second choice and
  • TypeIs[] (in assertion form) is my third choice.

I’d hesitate to use TypeCheck as the name. I agree with Jelle’s comment that it’s too easy to confuse with “type checking” in general.

Although I personally like variations of “narrow” / “narrower” (such as TypeNarrower[]), I think it’s a bit too jargony for general use. Ditto for Predicate or TypePredicate.

1 Like

On IsType vs TypeIs: “foo is type Bar” does read more naturally than “foo type is Bar” but I don’t think that’s the “sentence” formed by the code: rather, to pick a specific example:

def is_all_ints(seq: Sequence[object])
  -> TypeIs[Sequence[int]]:
  ...

This would read to me more like “is_all_ints takes a sequence and returns (true if its) type is a sequence of ints”. In the rest of the code, the thing you see is the function name (if is all ints), so it doesn’t make sense to me for this new type to be phased as a question: it’s being used where I’d expect to see an answer. This leans me mildly towards TypeIs.

If we wanted to be super explicit we could call it TrueIfTypeIs but I don’t think Python has other instances of “partial sentence type names” like this.

3 Likes

I just released typing-extensions 4.10.0, which provides the TypeIs special form.

The PEP is also supported in the latest releases of pyright and pyanalyze, and I’m hoping to get support merged into mypy soon.

7 Likes