Proposal: Add one-line type narrowing similar to 'cast'

My suggestion is to add a way to coerce type narrowing without any runtime change,
by adding a new ensure_type.

def function(arg: Any):
    ensure_type isinstance(arg, int)
    reveal_type(arg)  # Revealed type: "builtins.int"

Or

def function(arg: Any):
    ensure_type(arg, int)
    reveal_type(arg)  # Revealed type: "builtins.int"

Or something like this.

Examples:

def function(arg: Union[int, str, list]):
    ensure_type not isinstance(arg, int)
    reveal_type(arg)  # Revealed type: "str | list"
def function(arg: Union[int, str, list]):
    ensure_type not isinstance(arg, int)
    ensure_type not isinstance(arg, str)
    ensure_type not isinstance(arg, list)
    reveal_type(arg)  # Error: Statement is unreachable

I explained in this issue why it’s useful (and how it can also help with Intersection types).

What do you think?

1 Like

You know isinstance accepts a tuple of types, so you needn’t split that last one over three lines?

A type-checker time assert function could be a useful complementary tool to the run time assert. I just think it needn’t be a new statement necessitating syntax changes. Nor should it piggy back on isinstance and require that built-in to do something in a new context, that it doesn’t already do. And if this is going to go anywhere, it really needs some examples that can’t be dismissed with “Why not simply re-annotate the function with a better description of the types it is intended to support?” such as:

def function(arg: Iterable):
    ensure_type_not(arg, str)

My immediate thought was: Why not use assert?

def function(arg: Any):
    assert isinstance(arg, int)
    reveal_type(arg)  # Revealed type: "builtins.int"

From your linked thread I see:

  1. assert makes it slower at runtime, and sometimes we care
  2. assert can break things when improving types of an old code base

I’ve yet to find the performance argument, #1, to actually matter in a code base I maintain.

Argument #2 appears to be that the types are actually wrong, failing the assert. In that case I think you’d want the failure.

4 Likes
  1. You may care about the performance in some cases
  2. No I don’t want the failure. changing an old code base is risky and sometimes I’m only allowed to improve the type-hints without any risk. it is not ideal but there is no choice.
  3. One big benefit is when using Protocol. Please see my example with CanFlyProtocol:
class Animal:
    def say_my_name(self) -> None:
        print("My name is Animal")

@runtime_checkable
class CanFlyProtocol(Protocol):
    def fly(self) -> None: ...

class Bird(Animal, CanFlyProtocol):
    def fly(self) -> None:
        print("Fly")

def let_it_fly(animal: Animal):
    assert isinstance(animal, CanFlyProtocol)
    animal.say_my_name()
    animal.fly()
    reveal_type(animal)  # Revealed type: "<subclass of "Animal" and "CanFlyProtocol">"

Problem 1: We must decorate CanFlyProtocol with @runtime_checkable to be able to use assert isinstance(animal, CanFlyProtocol). even when we don’t need it at runtime.

Problem 2: isinstance is very slow with Protocol. See mypy docs:

isinstance() with protocols can also be surprisingly slow. In many cases, you’re better served by using hasattr() to check for the presence of attributes.

With the suggested ensure_type we avoid these problems.

1 Like

Have you thought about using assert isinstance and/or cast inside if TYPE_CHECKING guards?

4 Likes

If our whole point is to avoid runtime effects, then instead of requiring new Python syntax which (IMO rightly) has a high bar of acceptance for such new features, perhaps we can “resurrect” the idea of type comments which has largely fallen out of favor since annotations have been the preferred way to add first-hand type hint syntax support.

To use OP’s example, the ‘ensure’ type comment might look something like this:

def function(arg: Any):
    # type: ensure arg: int
    reveal_type(arg)  # Revealed type: "builtins.int"

TYPE_CHECKING is interesting but it still requires the protocol to have @runtime_checkable, or mypy will throw Only @runtime_checkable protocols can be used with instance and class checks

1 Like

You can do this as a one-liner, if you really want it, using typing(_extensions).TypeIs. The following works with both mypy and pyright:

from typing_extensions import TypeIs
from typing import TypeVar

T = TypeVar("T")

def unsafe_isinstance(x: object, y: type[T]) -> TypeIs[T]:
    return True

class Spam: pass
class Eggs: pass

def bar(x: Spam):
    if unsafe_isinstance(x, Eggs):
        reveal_type(x)  # Revealed type is "<subclass of "Spam" and "Eggs">"
    assert unsafe_isinstance(x, Eggs)
    reveal_type(x)  # Revealed type is "<subclass of "Spam" and "Eggs">"

I would be opposed to adding something like this to the standard library, as I don’t believe encouraging unsafe idioms like this is a good idea. Moreover, my sense is that most people are not looking for new ways of doing unsafe narrowing. I feel like I’ve seen a lot more requests for a version of cast() that was a little safer.

4 Likes

As the author of all the merged PRs linked to this issue, I’m well aware of the performance issues around isinstance() checks against runtime-checkable protocols. (Although they’re much faster in Python 3.12 than they were previously, they’re still much slower than isinstance() checks against nominal classes.)

Nonetheless, I’d urge you to profile your code before deciding whether or not the performance issues here are relevant to your use case. If you have a hot loop where you’re repeatedly calling isinstance() against a protocol, then this can absolutely matter a great deal. Often, however, that won’t be the case. You may well find that whether or not you call isinstance() against a runtime-checkable protocol is irrelevant to the overall performance of your code due to it being outweighed by the performance of other components.

3 Likes

Thank you, unsafe_isinstance is a good idea. There is a runtime cost but it’s insignificant.
I still think there should be a native way of doing this, like how we use cast.

Maybe if I change my suggestion to something similar to cast:

def function(arg: Union[int, str, list]):
    x = ensure_type(int, arg)
    reveal_type(arg)  # Revealed type: "int | str | list"
    reveal_type(x)  # Revealed type: "int"
    y = ensure_type_not(int, arg)
    reveal_type(y)  # Revealed type:  "str | list"

Also, instead of the name ensure_type() we can use type_is() or narrow()

I feel that this would pretty much do what you are asking out of the box: assign to a new variable with a narrower type, with the first variable having the larger type set.

def function(arg: Union[int, str, list]):
    _arg: str | list = arg  # type: ignore[assignment]
    reveal_type(_arg)  # Revealed type: "str | list"

You could run python with the -O flag and it will ignore assert statements–so your code will not suffer a performance penalty, and old janky code won’t fail (except when testing, when you might want it to? I don’t really get this requirement).

Unless you are relying on other asserts being checked to ensure proper function (which would be pretty weird), it seems like you might want this anyway.

1 Like

Personally, I’m not fond of type comments. with the last version of my suggestion we can do assignments as we do with cast()

x = ensure_type(int, arg)
# or
arg = ensure_type(int, arg)

With # type: ensure arg: int we can do only one thing in that line, but with the cast style we can do more:

r1, r2 = get_square_root(ensure_type(int, arg1)), get_square_root(ensure_type(int, arg2))

You just ignore the error, this is not what I’m looking for.
Please see a comparison with cast in the linked issue

Thank you. It’s possible to rely on other assert-s being checked, so I can’t really accept this solution.
Also, with this solution we still need to decorate Protocol with @runtime_checkable

It looks to me like this thread has fallen into the familiar pattern of an “Ideas” post that isn’t ready for serious consideration by core devs. Several working alternatives have been proposed and rejected, but the language isn’t going to change to suit one person’s particular situation. I’m moving this to the general help/discussion category.

If a workable proposal is developed hereafter, a new post to Ideas can be made.

1 Like