I’d like to propose implementing function.__getitem__ which returns a GenericAlias which should allow generic functions to be subscripted at runtime.
def construct[T]() -> list[T]: ...
construct[int]() # currently would raise a TypeError
It’s probably worth mentioning that would require a small change to GenericAlias.__call__ to avoid setting __orig_class__ on the returned object.
Thoughts?
I like the idea. What use cases do you have in mind?
At runtime this would do pretty much nothing, it just makes functions consistent with every other builtin class that is technically generic at type time.
+1 from me. This is the same as the change for collection types in 3.9, so we should probably add this in sooner rather than later.
Is this not part of PEP695? It should’ve been IMO.
No, I don’t think it should have been included as it’s not really related to type-parameter syntax.
What is the difference between construct[int]() and construct[str]()? How do you pass a type to a function which does not have parameters?
For all intents and purposes there is no difference and you’re right you can’t (for now, one day I’d like to be able to access the type parameter at runtime similarly to a normal parameter similar to how some other languages handle type parameters)
Nothing really, but the type hint here is important for type checkers to work.
It’s the same as how x = list[str]() was allowed in Python 3.9, the str does nothing at runtime but is important for the type system to be nice to work with.
The proposal should have started with a use case and explain what the proposed construct means.
«Consistent» does not say anything, and also can’t be a goal in itself.
«the type hint here is important for type checkers» in what way? what does the information mean / what do the type checkers do with it?
list[int] documents something as a list of ints, that’s more specific than list.
SomeClass[str] adds a parameter that can be used in method signatures for example.
What does function[str] mean? Is it setting a local type variable that can be used in the return signature of the function? Or do you mean to annotate function calls like function[str]() and then what would it mean?
Funny, I was thinking of this same idea this morning. It makes sense to me, regardless of PEP 695. Say we have a generic function:
T = TypeVar("T")
def foo(a: list[T]) -> T:
...
Now suppose we have another function that takes a callback like this:
def bar(callback: Callable[[list[int]], int]):
...
If we want to call bar(foo) but express the type correctly we might want to write bar(foo[int]). (In this case I think most type checkers can infer that, but there are likely cases where they can’t.)
We might also want to be explicit about the type when calling bar([]), to indicate that we want a specific list type: bar[int]([]).
I think we can rewrite all these use cases easily enough without adding this feature, but in terms of types, I do think that just as we can have a generic class C that we can make specific by writing C[int], we could have a generic function f that can be made specific by writing f[int].
I don’t feel strongly about it though.
Explicitly specialising a generic function call is often not necessary for a type checker since they can usually infer the types of the generic type parameters from the given arguments. Unlike functions, classes often need their type parameters explicitly written since their constructors are essentially functions that return generic objects but don’t take generic parameters in which the type parameters can be inferred from.
In the given example, there are no generic parameters for construct() in which the type parameter can be inferred, so explicit specialisation is actually required here. But then it’s hard to write a meaningful implementation for some method if there are no generic arguments and you can’t access the type of the type parameters at runtime.
It’s usually a mistake to use a type parameter in a function signature if it only appears once, unless the type parameter is constrained, in which case you’re probably using the type parameter to help work around the fact that Python doesn’t have proper overloading. And if you’re doing overloading in Python, the ability to inspect the type parameter type at runtime is important.
I was just recently looking into how the the look of generic functions can be faked in Python in order to overload a method by return type. It can already be done nicely.
from typing import TypeVar, Protocol, cast
Y = TypeVar('Y', int, str)
class Foo:
class GenericOverload(Protocol[Y]):
def __call__(self) -> list[Y]: ...
def __call__(self) -> float:
return 1.5
def __getitem__(self, key: type[Y]) -> GenericOverload[Y]:
try:
v = {
int: self.y_int,
str: self.y_str,
}[key]
return cast("__class__.GenericOverload[Y]", cast(object, v))
except KeyError as e:
raise TypeError from e
def y_int(self) -> list[int]:
return [1, 2, 3]
def y_str(self) -> list[str]:
return ['a', 'b', 'c']
foo = Foo()
print(foo()) # 1.5
print(foo[int]()) # [1, 2, 3]
print(foo[str]()) # ['a', 'b', 'c']
I don’t mind the suggestion but I doubt it’s going to be used very much. It could be useful if you for some reason wanted to indicate that the concrete type parameter type for a particular function call is important and should not be changed, i.e., accidentally.
This would also be nice for functions where the return type depends on the value of a passed argument. Without PEP 695, I imagine it could be explicitly genericised with an @generic[T] or @genericfunction[T] decorator.
Imagine a fictional array library:
@generic[T]
def zeros(shape: tuple[int, ...], dtype: DType) -> array[T]:
...
a = zeros[int]((4, 4), DType.INT)
There’s no need to make up new syntax to define a generic function. You can already write this:
T = TypeVar("T")
def zeros(shape: tuple[int, ...], dtype: DType) -> array[T]:
...
You just can’t write the following yet, and it would be nice if you could – that’s the proposal here:
a = zeros[int]((4, 4), DType.INT)
(There’s a question about generic functions using their type variable in only one position – that applies to argument types, not to retun types.)
So it seems no one is particularly opposed to this. Would this need a PEP or could it just be a standard change with an issue?
Asking the question, alas, is answering it – you probably need a PEP. That PEP should also answer the question “why wasn’t this done at the time PEP 585 was written” and “why are we no longer satisfied with that decision”.
I’m gonna guess the answer is no but do you have any ideas or discussion on the former or should I get in touch with @ambv?
I don’t recall, there may be traces of this in the public discussion around PEP 585 but they may be hard to find (some of it on typing-sig for sure). So asking if @ambv recalls is totally fair.
My own guess is that we probably didn’t have much of a use case for this. While it’s essential to be able to write a: list[int], you can’t write f: foo[int] (since foo isn’t a type, it’s a specific generic function). And writing list[int](a), while allowed, is very rare – similarly, foo[int](args) would be very rare, and more likely resolved by forcing the type of one of the arguments.
There isn’t any particular reason why PEP 585 doesn’t propose making function objects runtime-subscriptable, too. The use case for this didn’t appear to me at the time as a function isn’t a type.
Funnily enough, at some point in the past, I did suggest that treating functions like types would make for very nice typing of complex callables. The arguments that killed the idea at the time were:
CallableLike[some_func] construct to solve the problem above, we would put possibly irrelevant details about the function signature into the required callable type; for example: what if we don’t care about the names of the positional arguments?As for my current thoughts: given Guido’s concrete example it makes sense for me to have this.
Something like this would be pretty useful for eliminating wordy protocols.
Why then generic classes need to be inherited from Generic[]? I think that a special decorator is needed to distinguish free variables from bound variables.
@genericfunction[T]
def f(a: str, b: U) -> dict[T, U]:
...
x: dict[int, str] = f[str]('spam', 42)
No need to make functions subscriptable. It only needed to add genericfunction, so that genericfunction[]() returns a subscriptable object. I think it is easy.