Currently, using a type[Something] annotation requires the given type to be a concrete class, which is often useful, since it means that the class can be safely instantiated within the code using the annotation. However there are cases where a class does not need to be instantiated in order to operate on it.
Here is a (somewhat contrived) example:
from abc import abstractmethod
class Abstract:
@abstractmethod
def foo(self):
pass
class Concrete(Abstract):
def foo(self):
print("bar")
def my_special_isinstance(o: object, t: type[Abstract]) -> bool:
return isinstance(o, t)
my_special_isinstance(Concrete(), Abstract)
# Only concrete class can be given where "Type[Abstract]" is expected [type-abstract]
Note that in reality, my code that reproduces this would be far more complex (rather than just being a wrapper around isinstance). Some other examples can be found in this issue.
It’s clear that unless type safety is reduced for other cases, this case will never satisfy mypy, and as such I believe the best solution is to implement a separate type annotation for this case: AbstractType. This could come with a few benefits:
Functions using it as an annotation could be prevented from attempting to instantiate the abstract class
The type annotation could give users of libraries that utilise it a bit more information on how the function behaves (ie knowing the class isn’t instantiated)
It would be possible to make the above example type-safe with relatively trivial edits
As far as I can tell, this is likely the best compromise to allow for code such as the above to be type-safe without reducing type safety for other code. What does everyone think?
I like this! I’ve had the exact issue before where I wanted to call isinstance(x, T) in a function where T was an argument, but then I couldn’t pass my abstract base class into that function.
I also see potential value in specializing AbstractType like AbstractType[BaseClass] so that only subclasses of BaseClass are allowed (which may be abstract!).
The reason is that mypy thinks the function my_special_isinstance might instantiate the given type, which is not possible with an abstract class, so mypy doesn’t allow passing abstract type to my_special_isinstance.
Cool, I didn’t see those in the thread, but it’s cool to see others have the same idea! I’ve read through the instructions for creating a PEP and think I’d be able to write one. I’m on holidays from university right now, so I have time to do some work on it! If anyone else wants to help out, I’d love your help too! I’ll post something on the Typing-sig mailing list to see if I can get any other feedback from there before I start.
Is this in fact a defect with mypy? Is there good rationale that it doesn’t allow an abstract class to be passed as argument annotated with type[Abstract]? I don’t find any doc that states type[C] implies a concrete class.
As far as I am aware, it is by design, and in a lot of cases it is actually very helpful. For example, it means that in one of my projects, Plugins derived from BasePlugin cannot be registered to the PluginManager unless they are concrete classes, thereby avoiding all potential bugs due to unimplemented methods.
I think it would be better to add a new annotation rather than changing the behaviour of Type, even if the existing behaviour isn’t necessarily correct. This is especially true, as if the behaviour of Type were changed to match what I have described for AbstractType, then there would be a need to also introduce a ConcreteType annotation to allow for the existing behaviour to continue, which would be a breaking change for many code bases.
I’d lean it’s at least very ambiguous. I often have functions that accept a type which include abstract types. With amount of runtime type introspection python has there’s a lot of things you can do passed an abstract type. There are also some very basic functions that work fine with abstract types (is instance/issubclass). While mypy considers that to be an error by default, pyright considers that to be fine always. In mypy’s case after a lot of discussion on this exact issue, the type error for passing abstract type to a function that accepts a type became it’s own separate error code so you could disable/enable it as your own choice. I think that’s a very convenient choice of just letting author choose whether it should be an error or not.
I believe that’s the point of wanting a separate annotation for this. When your function can accept an abstract type, it can annotate it as such. When it needs concreteness, it can annotate that instead.
In some places, you want to promise concreteness and in others you don’t. So you would need to disable errors within the code. I don’t think being able to disable errors for particular tools in the code is a good solution for a few reasons:
The disabling happens at the call site, but it’s the call target that knows whether it accepts concrete or abstract types,
the disabling has to be done for every type checker,
disabling is a lot noisier that choosing a slightly different annotation,
disabling would typically require a comment to explain why you’re disabling the error, and
you need to add more disable commands whenever you add type checking tools.
If we take it for granted that Type[A] implies a concrete type, then indeed we need a way to indicate non-concrete type.
For the naming of ‘AbstractType’, does it require the type to be abstract, or does it allow the type to be abstract? The naming seems to suggest the former while the use case seems to suggest the latter.
The fundamental issue, as far as I understand, is that unlike in C++ or Java etc where class abstractness is defined at language level, abstractness is not a native construct of Python classes but a runtime behavior of ABCMeta.
For example, if you code your own custom meta class that achieves the same result as ABCMeta, is a class constructed by it considered abstract by the programmer / typechecker?
Therefore I do find mypy’s treatment of requiring Type[A] annotated value to be concrete class to be wrong. Maybe it’s so done as there’s not a way to indicate to the typechecker that A should be constructible. Maybe an enhancement in this regard is appropriate, i.e. to support mock code like the following:
T = typing.TypeVar(bounds=[A, typing.Concrete])
def f(c: typing.Type[T]): …
You make a valid point about the naming suggesting behaviour that doesn’t line up with what I’m describing. On second thoughts, using ConcreteType and Type actually makes much more sense as the naming is more clear.
I initially didn’t suggest this as I was concerned that the changes would cause a decrease in code safety, since unsafe behaviour (instantiating an abstract class) would then be possible in code, since the existing error would no-longer apply. However, I didn’t realise that this would be addressed by the new error this could introduce (attempt to instantiate an abstract class), meaning that the locations where this reduced safety affects the code would be more apparent.
My only concern with this would be that this would be bad for backwards compatibility for Mypy, as unmaintained libraries would lose some degree of type safety for users (who in future would assume that anything annotated with Type won’t be instantiated, as proposed by this change) of libraries that didn’t change all their definitions.
So essentially, using ConcreteType for types that can be instantiated would be much more clear, but may cause problems for backwards compatibility in Mypy (although given that this is an inconsistency compared with other type checkers, this might not be a huge issue). Using AbstractType for types that cannot be assumed to be safe to instantiate is less clear, but will cause fewer issues for users of Mypy, potentially at the expense of users of other type checkers.
I’d love to hear more opinions on which of these makes the most sense to everyone. In particular I’d like to hear from some people that use Pyright a bit more, since I’m less familiar with it. How would each option affect your code?
