Draft PEP: Matching Union Types

I finally got around doing a PEP-style write-up for my old PR https://github.com/python/cpython/pull/118644, allowing match-case against union types.

Looking for feedback and a PEP sponsor.

Abstract

This PEP proposes allowing union types to be used in structural pattern matching[1].

Motivation

PEP 604 introduced the | operator to create type unions, and allows its usage in isinstance and issubclass calls.

PEP 634 introduced structural pattern matching, which allows representing isinstance calls using the more powerful class-pattern syntax.

This PEP proposes to allow using union types in class patterns, mirroring their behavior as observed in isinstance calls.

As a motivating example, consider using the ast module to find all decorated functions in a Python source file. The corresponding ast node classes are FunctionDef and AsyncFunctionDef. To match both types of function definitions, one must currently either write two separate case clauses or use a union-pattern of class-patterns:

import ast

for node in ast.walk(tree):
    match node:
        case (
            ast.FunctionDef(name=name, decorator_list=decorators)
            | ast.AsyncFunctionDef(name=name, decorator_list=decorators)
        ):
            print(f"Found function {name} with decorators {decorators}")

With this PEP, the same logic can be expressed more elegantly as:

import ast

AnyFunctionDef = ast.FunctionDef | ast.AsyncFunctionDef

for node in ast.walk(tree):
    match node:
        case AnyFunctionDef(name=name, decorator_list=decorators):
            print(f"Found function {name} with decorators {decorators}")

This has four main advantages:

  1. It reduces the amount of boilerplate code needed to match multiple classes in a single case clause, making the code more readable and maintainable.
  2. It allows for easier extension and refactoring. If a class is to be added or removed from the selection of classes to match, instead of updating all existing case clauses across all files in the codebase, updating the union type definition is sufficient.
  3. It strengthens the consistency between the behavior of isinstance and class-pattern matching.
  4. It allows emulation of @sealed types proposed in[2].

Specification

This PEP modifies the logical flow of class-pattern matching[3]. When name_or_attr is a union, the pattern is sequentially checked against each member of the union, returning either the first match or raising on the first encountered exception.

This makes matching a class pattern with union type essentially equivalent to matching the union-pattern of the class-pattern of the individual members:

union = A | B | C
match arg:
    case union(foo=foo, bar=bar): ...
    # is essentially equivalent to:
    case A(foo=foo, bar=bar) | B(foo=foo, bar=bar) | C(foo=foo, bar=bar): ...

A small difference is that matching malformed union types such as int | list[int] may still succeed, whereas case int() | list[int](): would be an illegal pattern causing a SyntaxError.

Originally, PEP 604 specified that calls like isinstance(5, int | list[str]) should always raise a TypeError, since the second argument is not an instance of type (but of types.GenericAlias). While this is the case in Python 3.10, this behavior was changed in Python 3.11, and a TypeError is only raised if no earlier legal member of the union produced a match. This PEP adopts the same behavior as isinstance for consistency. See[4] and[5] for more details.

Examples

A simple example of matching a union type:

IntOrStr = int | str
match "Hello":
    case IntOrStr(x):
        # will match, since str is a valid member of the union
        print(f"Matched string {x!r}")

match 3.14:
    case IntOrStr(x):
        # will not match, since float value is neither int nor str.
        ...
    case _:
        # will match, since the wildcard pattern matches everything
        print("Matched something else")

Matching union types with positional or keyword arguments is straightforward:

import dataclasses

@dataclasses.dataclass
class Point2D:
    x: int
    y: int

@dataclasses.dataclass
class Point3D:
    x: int
    y: int
    z: int

AnyPoint = Point2D | Point3D
IntOrStr = int | str

match Point2D(1, 2):
    case AnyPoint(x, y):
        print(f"Matched Point({x}, {y})")  # Matched Point(1, 2)

match Point3D(1, 2, 3):
    case AnyPoint(x, y, z):
        # Will match the second member, since it is not an instance of Point2D
        print(f"Matched Point({x}, {y}, {z})")  # Matched Point(1, 2, 3)

match Point3D(1, 2, 3):
    case AnyPoint(x, y, z=IntOrStr(value)):
        # No restriction on the subpatterns.
        print(f"Matched Point({x}, {y}, {value})")  # Matched Point(1, 2, 3)

Invalid class patterns within a union only raise exceptions if evaluated:

Dims = int | tuple[int, ...]

match 1:
    case Dims() as one:
        # eagerly matches int, second union member is never tested
        print("Matched single dimension {one!r}.")

match (1, 2):
    case Dims() as many:
        # will produce a ``TypeError``, since tuple[int, ...] is not a valid class pattern
        print("Matched multiple dimensions {many!r}.")

Reference Implementation

  • A proposed implementation for CPython is available in[6].

How to Teach This

The reference implementation updates the documentation[7] to describe the new behavior of class patterns with union types. Because this behavior aligns class patterns with the existing behavior of isinstance calls, it should be intuitive for users already familiar with structural pattern matching and Python’s type system.

Moreover, since Python 3.14, typing.Union and types.UnionType have been unified[8], eliminating concerns about runtime differences between the two forms of union types, which would have been needed to explain otherwise.

Open Issues

Note that TypeAliasType introduced by PEP 695 is not supported in class patterns:

type IntOrStr = int | str  # TypeAliasType, not UnionType

match "Hello":
    case IntOrStr():  # raises TypeError
        ...

match "Hello":
    case IntOrStr.__value__(value):  # holds the original UnionType
        # This is technically legal, but discouraged and likely unsupported by static type checkers.
        print("Matched {value!r}")  # Matched 'Hello'

However, the legacy typing.TypeAlias works as expected, since at runtime it produces a standard union type:

from typing import TypeAlias

IntOrStr: TypeAlias = int | str
match "Hello":
    case IntOrStr(x):
        print(f"Matched {x!r}")  # Matched 'Hello'

Tools that automatically convert TypeAlias to 695 TypeAliasType may cause runtime failures in class patterns, although similar issues already exist when such aliases are used in isinstance calls.

Copyright

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.

References


  1. [match-case] Allow matching Union types
    https://github.com/python/cpython/issues/106246 ↩︎

  2. Draft PEP: Sealed decorator for static typing
    https://discuss.python.org/t/draft-pep-sealed-decorator-for-static-typing/49206 ↩︎

  3. Class Patterns
    https://docs.python.org/3/reference/compound_stmts.html#class-patterns ↩︎

  4. Speed up isinstance on union types
    https://github.com/python/cpython/issues/91603 ↩︎

  5. isinstance(object, classinfo)
    https://docs.python.org/3/library/functions.html#isinstance ↩︎

  6. gh-106246: Allow the use of unions as match patterns
    https://github.com/python/cpython/pull/118644 ↩︎

  7. Class Patterns
    https://docs.python.org/3/reference/compound_stmts.html#class-patterns ↩︎

  8. Merge typing.Union and types.UnionType
    https://github.com/python/cpython/issues/105499 ↩︎

7 Likes

For this example:

IntOrStr = int | str
match "Hello":
    case IntOrStr(x):
        # will match, since str is a valid member of the union
        print(f"Matched string {x!r}")

match 3.14:
    case IntOrStr(x):
        # will not match, since float value is neither int nor str.
        ...
    case _:
        # will match, since the wildcard pattern matches everything
        print("Matched something else")

Won’t the syntax IntOrStr(x) require the __call__ method in Unions? And would that then allow Union[int, str]()? As we can see, unions do not currently support it, and I don’t think they will anytime soon (see cpython/Include/genericaliasobject.h at main · python/cpython · GitHub).

I don’t know why you think __call__ is needed here. It’s treated as a class-pattern, and more or less equivalent to

match 3.14:
    case int(x) | str(x): ...

the x here matches the value itself, which is how certain builtins do the matching, see: PEP 636 – Structural Pattern Matching: Tutorial | peps.python.org

1 Like

Well obviously some case ... | ... is treated that way, but when we define IntOrStr, it has the type Union (see typeshed/stdlib/builtins.pyi at main · python/typeshed, where __or__ and __ror__ of type return such an Union). The type typing.Union(implemented in C) is not callable. Now when a variable is called, the __call__ method of the variables instance is (obviously) called. That means, that for

class MyClass:
    def __call__(self, *args, **kwargs) -> Any:
        ...

instance = MyClass()
x = instance() # We call `__call__ here`

the last line will invoke the __call__ method.
Now doing:

import types

U: types.UnionType = int | str

U(x) # Will also invoke `__call__` OR `__new__` + `__init__` (depending on wether `type().__or__(type())` returns an instance)

Now here we obviously call some methods defined in the class Union (implemented in C), which is not callable that way iirc.

The link you send says

You can combine several literals in a single pattern using | (“or”): […]

However i don’t know whether that actually includes function calls, which might be based on the pattern, like:

match x:
    case int(x) | str(x): ...

Where x might be determined at runtime, and might be no literal (unlike case 400 | 401: ...).
Appart from this, functions used in match-case contexts should return bool, whilst int.__new__ and str.__new__ obviously return int / str respectivly. This makes us need to use the __eq__ method of the instance created at int.__new__. That should mean that equivalent code should be something like:

from typing import overload, Literal, Any

class my_type:
    def __new__(self, arg) -> my_type: ...
    @overload
    def __eq__(self, other: my_type) -> bool: ...
    @overload
    def __eq__(self, other: Any) -> Literal[False]: ...


class my_other_type:
    def __new__(self, arg) -> my_other_type: ...
    @overload
    def __eq__(self, other: my_type) -> bool: ...
    @overload
    def __eq__(self, other: Any) -> Literal[False]: ...


x = ...

if my_type(x).__eq__(3.14) or my_other_type(x).__eq__(3.14): ... # Replace `my_type` and `my_other_type` with `int` and `str`.

Now obviously some match-case could also invoke some __contains__ like pattern, so we check if 3.14 is in {a, b, c} for some

match 3.14:
    case a | b | c: ...

which would again require some special dunder method, that cpython/Include/internal/pycore_unionobject.h at main · python/cpython does not inform about (sadly Union is just typed as Union: _SpecialForm in the typing stubs, which makes access to the information about dunder methods hard).

If i am wrong about anything please lmk.

It seems unfortunate that you wouldn’t be able to use TypeAliasType for this - there’s already a bunch of class-related places you can’t use it with, it’d be nice to not add more.

I think you mistakenly believe __call__ gets executed in a class-pattern. This is not the case.

def test(x):
    match x:
        case list(): ...

No __call__ gets executed in this code. We can look at the discompiler output:

  1           RESUME                   0

  2           LOAD_FAST                0 (x)

  3           LOAD_GLOBAL              0 (list)
              LOAD_CONST               1 (())
              MATCH_CLASS              0
              COPY                     1
              POP_JUMP_IF_NONE         3 (to L1)
              UNPACK_SEQUENCE          0
              RETURN_CONST             0 (None)
      L1:     POP_TOP
              RETURN_CONST             0 (None)

whereas a plain list() statement without case gets compiled to:


  1           LOAD_NAME                0 (list)
              PUSH_NULL
              CALL                     0
              POP_TOP

As you can see there is no CALL anywhere in the former, but the special MATCH_CLASS primitive.

Sounds like a good proposal. Sorry, I don’t think I could find the time to act as sponsor, but I hope you can find another core dev (maybe @brandtbucher, who implemented match/case originally; or a member of the Typing Council).

2 Likes

I would put it differently: I think it is a mistake to add this feature if it only works with the legacy way of declaring types.

I assume that it was a very conscious and deliberate choice that TypeAliasType is incompatible with various things coming under the banner of runtime typing and that that choice was made precisely with the expectation that the sort of idea in this PEP was intended not to work with static types in future.

There needs to be a more general resolution on where the boundary lies for reasonable use of static types at runtime. Wherever that boundary lies TypeAliasType should work for the things up to the boundary and general language runtime features should not be based on things that lie on the other side of the boundary.

2 Likes

I don’t see why this feature should wait for a resolution to the TypeAliasType runtime question. It makes sense to me that if TypeAliasType doesn’t work with isinstance, it also shouldn’t work with match. If, at some point, TypeAliasType is made to work with isinstance (which I’m kind of hoping for) then support for match can be added as well.

There are many other things that only work with (“legacy”) TypeAlias, so it doesn’t seem weird to me to add another:

  • subclassing
  • instantiating
  • accessing class methods and attributes
1 Like