An alternative could be to make TYPE_CHECKING a built-in, as discussed in Moving TYPE_CHECKING out of typing and into the built-in namespace. I don’t personally love making it a builtin, since import typing is cheap enough for the vast majority of users, runtime typing is a real part of the language and TYPE_CHECKING can get very sketchy.
Yeah, I object to this. I don’t think that any symbol named TYPE_CHECKING should special semantics. This should apply only to typing. TYPE_CHECKING.
Both mypy and pyright already have mechanisms to specify that specific symbols should have hard-coded values at type checking time, so if someone wants to specify that a symbol named TYPE_CHECKING has the value of True or False, they can already configure their type checker as such. This shouldn’t be imposed on everyone unconditionally.
Both mypy and pyright already have mechanisms to specify that specific symbols should have hard-coded values at type checking time, so if someone wants to specify that a symbol named TYPE_CHECKING has the value of True or False , they can already configure their type checker as such. This shouldn’t be imposed on everyone unconditionally.
This doesn’t work for libraries. A library cannot know what type checker its users are going to use. Having a universal way of deferring typing imports without importing the typing module is very useful for libraries used in command line applications. I saw 35% improvement in import time in my library using this trick, I couldn’t do this if it didn’t work with either mypy or pyright.
Will there be any need for TYPE_CHECKING when PEP 749 is guaranteed (after Python 3.13 is EOL)?
Would it be possible to cache the interpreter state after the typing import and save it to disk? (Like a pyc file, but better). This might be possible if typing doesn’t really interact with anything the user has done to the interpreter, which may be the case?
There is nothing niche about wanting to avoid expensive imports that are not necessarily needed at runtime. The modules to be imported from may not even be installed in the case of optional dependencies.
The problem is that import typing has a real runtime overhead and it isn’t possible to avoid that overhead at runtime if the only way to check TYPE_CHECKING is by importing it from the typing module. From my measurements here a simple hello world program is made about 10% slower by from typing import TYPE_CHECKING.
I don’t think so, or at least not without more consideration; The typing module is stateful thanks to the overload decorator storing overloads for runtime retrieval. I also wouldn’t want to special-case the typing module here, if we’re looking to make something better than .pyc files, it should be something more generally useful.
As I mentioned in one of the other recent import time related threads though, anything we can do to speed up imports as well as defer expensive imports is welcome and these shouldn’t be mutually exclusive. Deferring imports where reasonable improves the startup cost for code that may not use everything it imports each time, making imports faster helps everyone who can’t defer or actually does use it.
It’s very unlikely that a library could completely avoid importing typing if it is typed, so in practice I don’t think this would help anyone.
In rare cases, a library could use a stub to provide its type information and completely avoid importing from typing. Although if module load time is such a concern, it would be best to implement such a library in a compiled language.
This would also set a terrible precedent in ascribing meaning to a non-builtin symbol based only on its name. I don’t think we should go there.
I also think that typing.TYPE_CHECKING is a significant footgun. It provides a way to lie to the type checker, and this makes code fragile. In my experience, the use of TYPE_CHECKING is a way bigger source of uncaught bugs than most of the esoteric type theory concerns that are frequently brought up in this forum. I advise developers to avoid using TYPE_CHECKING if they want to write robust and maintainable code.
As I said above, this leads to fragile code, so I strongly recommend against it. When you lie to type checkers and linters, you’re going to mask bugs that they would otherwise find. There are almost always ways to avoid using typing.TYPE_CHECKING. I use it only as a last resort. It’s a footgun that many developers are too eager to reach for.
If you’re insistent on using this approach in your own code (as opposed to a library), you have the option of telling mypy or pyright to treat a specified global symbol as “always True”. For example, if you’re using pyright you can add the following line to its config file:
defineConstant = { TYPE_CHECKING = true }
I’ll also note that if your program is so sensitive to startup times that you are trying to shave tens of milliseconds from it, then Python is probably not a great language choice to begin with.
Unfortunately, while I agree with you, the people dealing with this don’t always have the freedom to make a different choice here for various non-technical reasons.
I’ll have some actual numbers from a staging environment in about a week from now, but it was a noticeable delay in something that creates multiple interpreters.
Unfortunately, you specifically have also held that other means of deferring the import that ensure the runtime symbol is identical aren’t supported. (module getattr, and deffering the import through indirection, linked at top)
I don’t see how putting the typing imports into a TYPE_CHECKING block is lying about anything.
I see this sort of attitude a lot whenever anyone talks about speed in Python. I guess there are two different approaches that people take to the idea that Python is slow:
Maybe we should make things faster or reduce unnecessary overheads.
Maybe we should just give up and let everything become slower and slower.
You can use Python to wrap up C/Rust etc code but whatever you do to accelerate the heavy lifting a simple CLI program is very often limited by startup overhead. Every time someone decides that it’s okay to waste a bit more time here and there by making different parts of startup slower it all adds up.
If people took the attitude that runtime performance does actually matter then I think a lot of things would be designed differently. Instead every time a discussion about performance comes up people will call into question even the notion that anyone should care about it because “use a different language if you care about that”. If you don’t care about performance then it is inevitable that things will end up being slow.
This is representative of things I’ve done to ensure import costs are minimal in library code. Youll find zero typing imports that are eager here, and the library is fully typed including complex type signatures: GitHub - mikeshardmind/async-utils: Collection of async utillities It’s a design goal that things using this can import and only incur the cost of what is used, and the only thing I had to do that I wasn’t happy about was remove use of typing.Final, not because of it having any runtime effect, but because type checkers don’t understand Final as a deferrable re-export.
Eric, I respect your goal of making type checking more inline with runtime. I think it definitely a good rule to write code that is type checked the way it is run by the interpreter. What is proposed here however doesn’t encourage people to have more divergence between type checking and runtime, it merely gives people a way to not incur the typing module overhead. What you are suggesting here fits more in a style guide or a linter rule or maybe a super strict type checker setting, it shouldn’t be up for type checkers to prevent people who want faster import from using typing.
As I said before, having type checker-specific setting to define this won’t help libraries that need to support many users with different type-checkers/IDEs. I don’t think telling these libraries to write their code in a more performant language is fair. I also don’t think that pushing them to use stub files and leave their code un-annotated is a good idea. It makes it harder for them to maintain their code and makes them lose the benefit of type checking their own code with inline annotations.
This is needed to have lazy importing work properly. Rather than treat it as a footgun and tell people not to use it, it would be beneficial to look at why people need it and teach how to make sure it is only used sparingly, and in a way that runtime matches type time.
The alternative requires type checkers to understand and support module getattr and custom module loaders, both of which are in active use. This seems far too dynamic to be worth putting into the type system.
My experience is that it’s only a footgun when there’s no appropriate matching runtime symbol in the else branch.