types like Callable and Awaitable are completely unrelated to “collections”, therefore collections.abc is clearly the wrong place for them, so the fact that typing.Callable and typing.Awaitable are deprecated in favor of these aliases is very confusing.
imo they should either be moved to a different module, or the versions in typing should be un-deprecated until a more suitable location for them is decided
Note: The Python runtime does not enforce function and variable type annotations. They can be used by third party tools such as type checkers, IDEs, linters, etc.
Thanks for the link. As much as it feels initially awkward importing ABCs that are at the first glance unrelated to collections from collections.abc, I have to agree 100% with @guido here:
And coroutines are closely related to generators, and awaitables are closely related to coroutines, and many of these are returned by callables, so it all makes sense eventually following the slippery slope.
On the other hand, since it becomes apparent that collections.abc isn’t just about collections anymore, I think if we are willing to go through another round of deprecations to move things around, we should deprecate collections.abc altogether and move all ABCs/protocols into a top-level module named something like abcs, e.g. from abcs import Callable, or we can name it protocols if people find it too similar-looking to the existing module of abc.
Given that _collections_abc gets pulled in at startup anyway [1], there’s a decent argument to be made to just make all the common ABCs available as builtins (as @zhangyx suggested).
It would also make the more permissive (and usually more accurate) ABC based type hints just as easy to write as the overly narrow ones based on the concrete builtin types.
It’s technically omitted by the -S option (since that skips importing the site module), but anything else that imports os will bring it right back. ↩︎
while adding builtins is less potentially breaking than adding keywords, I’m not convinced this is a good idea at the current point in time.
I don’t think the collection ABCs are more accurate from a typing perspective, at least not currently. The mixing of nominal subtyping with structural subtyping leaves a few undesirable effects. If we want to make typing the correct thing for duck-typing easier, I think we’d be better served long-term by non-inheritable protocols with intersections. [1] While I’m not strictly opposed to just placing these in builtins, I don’t think that it should be done currently and should happen only at a point in time where there is better guidance about when not to use collection ABCs in type hints, as their intended runtime behavior predating the type system varies from how they end up within the type system in subtle ways that can be surprising.
There have been recent things in motion to work on making this feasible and intuitive. I have been trying to find some consensus on ways to improve a few edge cases that look to be a bigger problem with intersections but are also a problem independently. ↩︎
I actually was not aware of the difference. I assumed typing ABCs are just an alias of collection.abc. Is there a reason they are implemented differently?
I bet many users out there could have the same assumption on this.
The typing versions are effectively the same, this isn’t about which module imported from, the collection abcs (and the existing exports equivalent to them in typing) have a few fundamental problems from a typing perspective. While the issues one can encounter with them are not entirely common, any improvements for this have to be very narrowly tailored to avoid breaking a lot of people’s workflows because of how commonly used these collections are. With Sequence, MutableSequence, AbstractSet, MutableSet, Mapping, and Mutable mapping there are issues where adding something in a subclass both changes variance and removes a capability. [1]
Coroutine has an issue where the way it works is based on the function body being observable by type checkers, as there’s some syntactic wrapping that happens automatically with async def that isn’t defined completely unambiguously.[2]
A proposal for reducing the impact of this is already in progress. ↩︎
Previously discussed around here somewhere, but I don’t think there’s in progress work? ↩︎
This (exposing all the protocols via a single shared module) is pretty much the only workable idea from the perspective of teams with a lot of data scientists. They grudgingly accept typing because they understand it helps keep things workable as the project gets larger, but the cognitive load is high already. If we start splitting all the various protocols (Callable is used incredibly frequently) out into 3 or 4 different modules, they’ll (rightly) revolt.
Whereas if everything ended up in a 3-letter namespace abc, then we’d be able to keep going the way we do now - import typing as ty would become import abc - and then people can keep working in Databricks notebooks and all sorts of other places that can’t keep up with the latest auto-import IDE wizardry, but still be productive based solely on muscle memory.
Practicality beats purity. It is convenient to have these types in one place. It is inconvenient (disruptive) to move things once people have learned where they are. IMO creating new modules and packages to hold these won’t make the world better.
i disagree. i mean you could also argue that everything in the standard library should be moved into a single massive module just because it’s more convenient. but that doesnt mean its a good idea to do so.
imo the confusion of having these unrelated types shoved into this module outweighs any possible practical benefit. there are also other issues with the collections.abc module that only add to the confusion, like the fact that they’re defined in this wacky way that prevents IDEs from being able to navigate to their original definition:
