Idea: decorator for functions that should be usable as types

Ideas
1

In the standard library, there are a number of cases where a function is exposed to construct a type, and the documentation pretends that the function is that type. In typeshed, this often means that we compromise and pretend that it’s not a function in the stubs as well. Otherwise, it’s a worse experience for typing users. The function can’t be used as a type, and the actual type isn’t available at the name that users expect, if the stubs are accurate.

One way to resolve this could be a decorator to indicate that using this function as a type in a typing context indicates that the return type of the function should be used. I think this would be safe because currently it’s just an error to use a function as a type. I don’t have an idea for what a good name would be. @type_constructor maybe?

This is a pretty niche use case, but I figured I’d see what people think. It came up most recently for select.poll.

2

Shouldn’t be allowed and shouldn’t be lied about in the typeshed.

This has some safety issues because functions are not a drop-in replacement for types.

If things accept anything that can be called to get an instance of a type, and only use it for construction, they should accept Callable[..., T] rather than type[T]

1 Like
3

I think you misunderstand. I don’t mean using it as a callable type.

I mentioned select.poll. It’s a function, but we type it as a class to meet expectations:

class poll: ...

Using that in a type context doesn’t mean using it as a callable, most of the time, it means using it to annotate an instance of that class. Similarly if we could do this instead:

@type_check_only
class _poll: ...

@type_constructor
def poll() -> _poll: ...

to allow typing users to continue to use select.poll as an annotation for an instance of that type, where the type checker will interpret var: select.poll | None = select.poll() as equivalent to var: select._poll | None = select.poll().

I’m sure there’s a variety of things that would need to be disallowed, and if type[select.poll] is one of them, then that’s fine.

A more common example of the problem is the situation with multiprocessing.Queue, which is commonly used as a type annotation but is actually a bound method of multiprocessing.context._default_context. The actual class is multiprocessing.queue.Queue. Taking a hard line against the compromise in typeshed isn’t very practical without changes to type checkers to support the transition, one way or another.

1 Like
4

I didn’t misunderstand. These should be typed as functions, and the appropriate return type should also be typed. Fixing the “incompatibility” should be done by people accepting these who don’t rely on it being a type, just that calling it returns an instance. type[T] implies things that are not true for Callable[…, T]. This might need it’s own construct to represent, including perhaps the decorators you suggested, but those decorators should not be seen as equivalent to type[T]

It’s not a fun situation, but lying about the type is worse than leaving it typed as Any, as people get a false sense of what is actually expressed and type-safe.

1 Like
5

In that case maybe I’m not understanding you? I don’t understand where type[T] is coming from, that’s not the use case that I’m talking about, and I’m not suggesting that they be equivalent to type[T]. I’m suggesting that they be equivalent to T, only when used as a type annotation.

6

You can’t make them equivalent to T without the difference between type[T] and Callable[…, T] being relevant here.

(x: T) implies that type(x) → type[T], among other things. What actually happens here is that there is a public function and an implementation class. Typing both accurately should be fine here.

1 Like
7

type[anything_at_all] is already hilariously unsafe due to LSP exceptions on constructors. I don’t see this as much worse individually, but it isn’t making the job of anyone who wants to fix that easier either.

I think to avoid making that worse, what you’d want here is a decorator that takes a type, and marks the function it decorates as another constructor of that type. It can’t be compatible with the type itself, but for normal uses this could have well-defined behavior.

1 Like
8

I think that would make this:

@type_constructor(_poll)  # we could specify this is automatic by return type?
def poll() -> _poll:

So the meaning of this to be safe would be that poll is still a function when called, has the “shape” Callable[[], _poll], and returns a functiontype when type() is used on it, but that when used as an annotation we’re expecting an instance of _poll?

(to be clear, I want this to be expressible, just want to avoid it being expressed in a way that makes other existing problems more complex to solve)

9

That looks right to me, but I don’t know how that should work for things that are generic.

10

Yeah, I was originally thinking that at runtime it’s just a no-op:

def type_constructor(func):
    return func

Or even that, like @type_check_only, it’s a stub-only construct, at least until we have a use case for it at runtime. Maybe I’m wrong, but I suspect it wouldn’t find a lot of use outside of the standard library, and that’s all in typeshed.

I hadn’t considered generics yet. I think that for some situation where:

class _RealClass(Generic[_T]): ...

@type_constructor
def FakeClass(arg: _T) -> _RealClass[_T]: ...

It might be nice to be able to have FakeClass[T] be something that can exist at runtime, but it’s not strictly necessary? Any case like that in typeshed right now can’t be made generic this way in a runtime context already. So we wouldn’t lose anything versus the status quo.

def type_constructor(cls):
    def decorator(func):
        if getattr(cls, "__class_getitem__"):
            func.__getitem__ = cls.__class_getitem__
        return func
    return decorator

Something like this could be cute in theory but it doesn’t work right now and making something like that work is a change larger than this benefit is probably worth.

11

As for this part, that’s true. I think the type analysis should probably treat it as an alias, and internally just immediately swap it out for real type. I’d expect:

class _RealClass: ...

@type_constructor
def FakeClass() -> _RealClass: ...

x: FakeClass = FakeClass()
reveal_type(x)  # _RealClass
reveal_type(type(x))  # type[_RealClass]

# Still just a function outside of a type context
reveal_type(FakeClass)  # def () -> RealClass
print(type(FakeClass))  # <class 'function'>

y: type[FakeClass] = FakeClass  # invalid; Incompatible types in assignment
z: type[FakeClass] = _RealClass  # valid

reveal_type(z)  # type[_RealClass]

And so on.

1 Like
12

Whatever the function is at runtime should also support type unions.
Perhaps the function could be a GenericAlias at runtime? It naturally supports generics/unions and forwards calls to the underlying thing [1].

  1. In this case, the callable; not sure if GenericAlias relies on the proxied thing being a type ↩︎

13

I think this situation is too rare and specialized to justify a new type system feature. Instead, we should fix CPython to avoid this situation. For example, we could replace the select.poll function with a class constructor, or perhaps add a separate select.PollType.

The current situation is unsatisfactory because while the class claims to be select.poll, isinstance(..., select.poll) won’t work.

6 Likes