I wish to know whether type checkers will fetch information from file.py if there also is a corresponding file.pyi.
To be more precise, I have a file.py definining a heavily overloaded function heavy_fn, as well as a simple type, say ModeLike = ModeEnum | Literal["mode1", "mode2"]. So I naturally isolate all the overloads of heavy_fn in file.pyi, but I keep ModeLike in file.py since I need it to be in the exposed API.
Now imagine that my typed package is used as third party. Will type checkers still fetch information about Mode or do I need to also redundantly define ModeLike = ModeEnum | Literal["mode1", "mode2"] in file.pyi?
If the answer is that I need to define it in both files, it also requires redefining ModeEnum if it’s also defined in file.py, which feels quite wrong again.
By the way, this approach to defining ModeLike is to ensure both static AND runtime checks as exhaustive as possible, while authorizing simple str calls in the API (simpler for users).
Any answer or advice is welcome, because I don’t think I’m going around this in the right way.
I am pretty sure that, if a .pyi file is present, the .py file is not consulted at all. That is at least how mypy behaves, as adding a stub file eliminates all errors on the corresponding Python file.
If I were defining types that need to be available at runtime and in stubs, I might split them into this structure:
mypkg/
types/
__init__.py
_stubs.pyi
_runtime.py
_runtime.py would contain types that you want to export or check at runtime, and _stubs.py would contain types that are only needed for type-checking. __init__.py would look like:
from ._runtime import [...]
TYPE_CHECKING = False
if TYPE_CHECKING:
from ._stubs import [...]
__all__ = [
...
]
Then in the rest of your package:
from . import types as t
Obviously, you don’t have to centralize all your types, but this could help resolve cases like you’ve described.
Note, I am not in any way a typing expert, and someone more informed may come around and say that this is a terrible idea.
Incidentally, I have also had the experience that overloads damage the readability of my Python files and it would be nice to cordon them off into .pyi files without losing type checking of the actual contents of my .py files. I suppose I would like them to act a bit like header files instead of replacements.
Thank you for the time you took to provide a possible solution. I think it is a bit too involved for me in terms of restructuring, but it’s worth considering it, clearly.
I personally feel that splitting off type information into a separate file for readability only makes the problem worse, hiding the problem doesn’t get rid of it. It does improve the situation for users that don’t care about typing, so I have sympathy for that particular use-case, even if it comes at a small additional cost for the people that do care about it.
There are however some proposals to reduce the verbosity of overloads, which seems like a better place to invest our efforts, rather than trying to unify type information between stubs and implementation. There are a lot of subtleties that make this a non-trivial exercise, since stubs were never designed to be used that way.
