PEP 747: TypeExpr: Type Hint for a Type Expression

This pep allows you to better type hint functions like that. But it does not change how easy/hard it is to implement them. So this pep does not give any answer in what those cases say and that enters into realm of how do you implement runtime type introspection logic and what cases do you support.

Edit: typeguard · PyPI you may want to look at this library including it’s implementation and issues it has to see various kinds of challenges for function like that and how they are handled.

1 Like

Thanks for clarification!


The following content is off-topic to this thread. Therefore I folded them to avoid distraction.

I ran the tests I wrote for rttc on typeguard, and surprised to find that it failed many of them … This indicates that typeguard might be only providing a subset of features that I am looking for, or at least it requires some extra syntax to work (i.e. not the most intuitive solution).

Test results for typeguard
  • Edit1: (Correction) typeguard allows a int value to match type float. I’ve updated my tests to use int and str to create type mismatch. The number of failed test increased by 1 (12 total).

  • Edit2: By standard list[T] only accepts one type parameter. I changed tests under 02-multiple to align with it. Number of failed test is down by 2 (10 total).

======================== 01-simple =========================
[ PASS ] check_type(1, <class 'int'>) is int => True
[ PASS ] False
[ PASS ] check_type([1, 2, 3], list[int]) is list => True
[ FAIL ] check_type([1, 2, ''], list[int]) is list => True (expected False)
[ PASS ] check_type(1, typing.Literal[1]) is int => True
[ PASS ] False
[ PASS ] check_type('alex', typing.Literal['alex', 'bob']) is str => True
[ PASS ] False
======================= 02-multiple ========================
[ PASS ] check_type((1, '2', 3.0), tuple[int, str, float]) is tuple => True
[ PASS ] False
[ PASS ] False
[ PASS ] check_type((1, '2', 3.0), tuple[int, str, float]) is tuple => True
[ PASS ] False
[ PASS ] check_type({3.0, 1, '2'}, set[int | str | float]) is set => True
[ PASS ] False
[ PASS ] check_type({1: '2', 3: 4.0}, dict[int, str | float]) is dict => True
[ FAIL ] check_type({1: '2', '3': 4.0}, dict[int, str | float]) is dict => True (expected False)
[ FAIL ] check_type({1: '2', 3: None}, dict[int, str | float]) is dict => True (expected False)
======================== 03-nested =========================
[ PASS ] check_type([[1, 2], [3, 4]], list[list[int]]) is list => True
[ FAIL ] check_type([[1, 2], [3, '4']], list[list[int]]) is list => True (expected False)
======================== 04-unions =========================
[ PASS ] check_type([[1, 2], ['3', '4']], list[list[int] | list[str]]) is list => True
[ FAIL ] check_type([[1, '2'], [3, '4']], list[list[int] | list[float]]) is list => True (expected False)
======================== 05-inherit ========================
[ PASS ] check_type([1, 2, 3], <class 'tests.05-inherit.A'>) is tests.05-inherit.A => True
[ FAIL ] check_type([1, 2, 0.0], <class 'tests.05-inherit.A'>) is tests.05-inherit.A => True (expected False)
[ PASS ] check_type(B(x=1, y='2'), tests.05-inherit.B[int, str]) is tests.05-inherit.B => True
[ FAIL ] check_type(B(x=1, y=2.0), tests.05-inherit.B[int, str]) is tests.05-inherit.B => True (expected False)
========================= 06-guard =========================
[ PASS ] A([1, 2, 3]) is tests.06-guard.A => True
[ FAIL ] A([1, 2, '3']) is tests.06-guard.A => True (expected False)
[ PASS ] B(x=1, y='2') is tests.06-guard.B => True
[ FAIL ] B(x='1', y=2) is tests.06-guard.B => True (expected False)
[ PASS ] C(x=1) is tests.06-guard.C[int] => True
[ FAIL ] C(x='1') is tests.06-guard.C[int] => True (expected False)
[ PASS ] add(1, 2) is int => True
[ PASS ] add('1', '2') is str => True
[ PASS ] False
[ FAIL ] 10/35 tests failed
Steps to reproduce
git clone git@github.com:zhangyx1998/rttc.git
cd rttc && python3 -m pip install -r tests/requirements.txt # termcolor

To test rttc:

python3 -m tests

To test typeguard:

pip3 install typeguard
TARGET=typegurad python3 -m tests

Should this program be accepted by type checkers?

class C:
  pass

def foo() -> type[C]:
  return C

def test(x: TypeForm[C]) -> None:
  pass

test(foo())

The argument foo() is not a valid type expression, but it is the type of the expression C. I’m trying to understand if implementing this feature requires contextual type information or if subtyping is enough.

Should this program be accepted by type checkers?

If we conclude that type[T] is a subtype of TypeForm[T], then this would be accepted by type checkers. I’m not yet convinced that this special case is justified. If you want test to accept type[C], you could make that explicit:

def test(x: TypeForm[C] | type[C]) -> None:
    pass

We seem to be at somewhat of an impasse on this aspect of the PEP. I’ve been exploring some ideas with @mdrissi in the pyright discussion forum about potential ways to move beyond the impasse.

1 Like

Much of recent comments are on that exact topic with arguments both ways. There’s also some extra discussion here on that. From that discussion with Eric 3 rules can be made on when type should be compatible with TypeForm,

  1. Let A be a variable. If all values possible in reveal_type(A) are valid for TypeForm directly then A should be fine.
  2. It applies only to classes that are officially documented and included in typeshed stubs. This excludes internal types like _LiteralGenericAlias.
  3. It applies only to types that can be used to solve T when the type is assigned to TypeForm[T]

2/3 come directly from Eric. The wording of 1 is different then discussion, but I’m unsure how best to word/explain 1. The original wording for 1 is longer and was,

Let A/B be valid expressions that can be used as type annotations in some context. So type[T], int, str, type[int], ClassVar[int], Literal[“a”], list[str], Annotated[“…”, int] would all be possible expressions.

Define function f with one argument X with annotation expression of A. Take space of all programs that are sound using that function. Call that space P1. Change A with A | B. Call space sound functions this time P2. If every program in P2 is also on P1 then B should be treated as a subtype of A.

Not confident the new 1 is best description, but that’s the idea.

Thanks for the responses. I apologize for not following the discussion more closely and forcing you to repeat what has already been discussed.

Personally, I don’t have an opinion one way or another. However, if it does become necessary to reject programs like mine above, then I think this is the first feature to require contextual typing for function calls.

Before this feature, given an expression f(x), it is valid for type checkers to first synthesize a type for x, then check that type against the corresponding argument type in f.

After this feature, type checkers must be able to first synthesize a type for f, get the type of it’s argument, and then decide how to check the expression x with respect to that type.

The PEP itself refers to extant “contextual rules” although I was not able to find any mention of these rules myself.

When a static type checker encounters an expression that follows all of the syntactic, semantic and contextual rules for a type expression as detailed in the typing spec, the evaluated type of this expression should be assignable to TypeForm[T] if the type it describes is assignable to T.

I’m not sure I understand what you mean by “synthesize a type for x”. Do you mean “evaluate the type of x”? And when you say “argument type in f”, I presume you mean the declared parameter type?

If I understand your point correctly, then it’s easy to demonstrate that it’s false. There are many situations today where bidirectional type inference (aka “context”) is required for correct evaluation of a call expression and the subexpressions that comprise its arguments. Here’s a trivial example of where a call to func1 and its arguments must be evaluated contextually.

def func1[T](x: T) -> list[T]:
    return [x]

def func2(v: list[float]) -> None:
    pass

v1: list[float]

v1 = func1(1)  # Function call is evaluated with context
func2(v1)

v2 = func1(1)  # Function call is evaluated without context
func2(v2)  # Error

Here’s another example involving a TypedDict:

class TD(TypedDict):
    x: str

def func1[T](x: T) -> T:
    return x

def func2(v: TD) -> None:
    pass

v1: TD

v1 = func1({"x": ""})  # Function call is evaluated with context
func2(v1)  # OK

v2 = func1({"x": ""})  # Function call is evaluated without context
func2(v2)  # Error

I could provide dozens of other real-world examples.

You are correct that the typing spec doesn’t mandate that type checkers implement bidirectional type inference, but practically speaking, this feature is required. Specifying how and where a type checker should use context is (at least for now) beyond the scope of the typing spec.

Yes, I am fairly certain we mean the same thing by synthesis and evaluate. I am borrowing the synthesis term from bidirectional literature (Dunfield and Krishnaswami), which draws a useful distinction between synthesizing and checking. I would also be happy with the language “infer a type for x.”

Yes, thank you.

I agree that bidirectional type inference is practically required, as your examples show. I think the spec should have something to say about bidirectional typing, either as part of the TypeForm PEP or to support it.

If a type checker existed which rejected your programs, I don’t think that it would be very popular, but it would conform to the spec. If a type checker existed which rejected my program above, it would ideally not conform to the spec, yes?

Bidirectional type inference is not applicable in your code sample above because the expression foo() evaluates (unambiguously) to type[C]. There is no context that could change this.

The answer to your question therefore hinges on whether the spec ultimately provides a special case such that type[C] is assignable to TypeForm[C]. That’s still an open question. The answer will dictate whether a conformant type checker is obliged to accept or reject your program above.

Pyright’s current (experimental) implementation of PEP 747 assumes that type[C] is not assignable to TypeForm[C], so it rejects your program.

I was checking up on this PEP to see what the status is, and saw no activity since November. But I’ve just now noticed that there’s more recent (interesting!) activity over in the aforementioned pyright github discussion.

I’m not sure if there’s a strict rule against it (?), but it seems to me like the PEP should link out to that thread as well as this one.

1 Like

Update: I’m currently working on the mypy implementation of the TypeForm PEP. I’ve drafted an initial PR and am working on improving it through feedback. Some of the feedback is complex and may take a few weeks to resolve, considering the very limited energy/time on my end.

Once there are 2 implementations of the PEP (in mypy and pyright), and any implementation issues that could affect the PEP itself are discussed, the PEP should be ready to submit to the Typing Council.

7 Likes

Exciting news: The mypy implementation of TypeForm is now ready for testing by library maintainers and other users! :partying_face:

To use mypy with TypeForm support:

# Install mypy with TypeForm support
pip install git+https://github.com/davidfstr/mypy@170a5e70dfb03425c98ba53b5ecb9b7def6e3f65  # >= 2025-03-25

# Install typing_extensions with TypeForm support
pip install 'typing_extensions>=4.13.0'

# Run mypy with TypeForm support enabled
mypy --enable-incomplete-feature=TypeForm

I’m continuing to shepherd the PR that merges this support into mypy’s main branch and eventually into a released version of mypy.

Additionally pyright >= 1.1.395 supports the latest draft of the TypeForm PEP, when you set “enableExperimentalFeatures” to true in the pyright configuration. So you can test how TypeForm behaves with pyright too.


One rough edge was found during the implementation: mypy cannot recognize quoted type expressions (like 'str | None' or list['str']) as TypeForm literals implicitly in all syntactic locations. Only 3 presumably-common locations (assignment r-values, callable arguments, and returned expressions) are currently supported:

# 1. Assignment r-value
typx: TypeForm = 'str | None'  # OK

# 2. Callable argument
def accept_typeform(typx: TypeForm) -> None: ...
accept_typeform('str | None')  # OK

# 3. Returned expression
def return_typeform() -> TypeForm:
    return 'str | None'  # OK

# Everywhere else...
list_of_typx: list[TypeForm] = ['str | None']  # ERROR
dict_of_typx: dict[str, TypeForm] = {'str_or_none': 'str | None'}  # ERROR

When using a quoted type expression in other locations, mypy gives an error that reads:

TypeForm containing a string annotation cannot be recognized here. Surround with TypeForm(...) to recognize.

To avoid errors you must tweak the code to use the explicit TypeForm(...) syntax:

# Everywhere else... (Take 2)
list_of_typx: list[TypeForm] = [TypeForm('str | None')]  # OK
dict_of_typx: dict[str, TypeForm] = {'str_or_none': TypeForm('str | None')}  # OK

I view this current limitation in the mypy implementation as a small deviation from the PEP, which implies a string literal expression containing a valid type expression should be assignable to TypeForm regardless of syntactic position.

Any comments on this limitation in general? Any comments on the current error message?

7 Likes

I think this should be treated as a bug in mypy, and ideally it should get fixed eventually. However, it doesn’t seem like this limitation would be all that onerous, and your error message is clear.

2 Likes

I have a couple questions about this rule, from the “Implicit TypeForm Evaluation” section:

Expressions that violate one or more of the syntactic, semantic, or contextual rules for type expressions should not evaluate to a TypeForm type

  1. Does this mean that whether a type expression should evaluate to a TypeForm type depends on where it appears? Two of the given examples are:
bad4: TypeForm = Self  # Error: Self not allowed outside of a class
bad12: TypeForm = T  # Error if T is an out-of-scope TypeVar

which seem to imply that these would be okay if we were inside a class and T were in-scope:

class C(Generic[T]):
    x: TypeForm = Self  # is this allowed now that we're inside a class?
    y: TypeForm = T  # is this allowed now that T is in-scope?
  1. When the PEP says that the type argument to TypeForm[T] and the argument to TypeForm(...) must be a “valid type expression,” is validity being defined using the same rule as above? In particular, I’m wondering again about type expressions that are valid only in some contexts:
# always allowed, never allowed, allowed only inside a class?
x: TypeForm[Self]
y = TypeForm(Self)

Yes, context matters. Self and T both comply with the syntactic rules for a type expression but do not describe a valid type expression in these examples because of the context in which they appear. Static type checkers already validate these semantic and contextual rules for type annotations. This PEP extends this validation to value expressions that are intended to be interpreted as type forms.

Yes, it’s using the same syntactic, semantic, and contextual rules spelled out in the typing spec today for type expression validity. The syntactic rules are described with a sub-grammar in the type annotations chapter of the spec. The semantic rules are provided as parenthetical comments in the sub-grammar definition (e.g. (where name must refer to a valid in-scope TypeVar) or (valid only in some contexts)). The specific contextual rules are distributed throughout other parts of the spec. For example, the subchapter on Self describes where Self can be used in type expressions and where it is invalid.

1 Like

Thanks for the explanation! I asked about the contextual rule because it surprised me that where an expression can appear in a type annotation should affect its use as a value, but if it’s intentional and self-consistent, I’m fine with it.

I opened a PR to try to clarify what confused me in (2).