Where would we put such a thing? typing.py? Make our own module?
Also, we don’t generally like adding syntax; adding decorators and functions is all fun and games compared to modifying the syntax because
New syntax = new issues
People will have to relearn rather than just say "Oh there is a new decorator. Yahoo!:
I may be interpreting this incorrectly though: @nemocpp Could you please elaborate on what you meant by
abstract TypeEnforcer class with a logging-like syntax (root TypeEnforcer or per-file TypeEnforcer) in an stdlib module that the 3rd party libraries can incorporate.
Then, using that same module, we’d have an official way of enforcing (or disabling) type checking, assuming you have set up a valid TypeEnforcer.
What would the ‘official way’ be?
(I’m not trying to shoot this down; just generally interested )
This is a good point, because I think the docs could be more clear about this. All I see right now is a bit in the function syntax docs that says “The presence of annotations does not change the semantics of a function.” and a similar brief note in the tutorial that says “Annotations are stored in the __annotations__ attribute of the function as a dictionary and have no effect on any other part of the function.” And those are true, but rather unobtrusively placed for such an important point. In a sense most of the weight is on what goes unsaid — the docs never say they are enforced, because they’re not.
In general I tend to think that when people are mistaken about how Python works the solution should be for them to read the docs. But this is harder when the docs are not so up-front about this.
Could you envisage a world (or future) where typing is mandatory for libraries/packages, but optional for scripts. And thus typing is gradual in the sense that you can use python perfectly fine without ever typing something. (There would also be no enforcement of typing in jupyter.)
Yes this would impose a cost that you have to pay when you convert some code you’ve been writing into a package that can be imported.
But with the guarantee that all packages are typed, typing could potentially be simpler, so it could still require less work overall to develop a typed package.
This is already the de facto situation right now. Typing is effectively not optional for libraries but is optional for end users.
Those end users don’t have to write type annotations themselves but their editor likely consumes the annotations from the libraries they use. The editor gives them warnings if they do something seemingly incorrect or it uses the annotations for autocomplete suggestions. The majority consumers of type annotations are editor plugins like pylance rather than command line checkers like pyright.
It’s not. In the last month I’ve had to work with 3 libraries that are untyped, with type stubs being unavailable. The biggest of those is plotly dash. A small one is a library we pay for.
Furthermore, polars, a modern library which does do it’s best to type everything, can’t type everything thoroughly. Because of the limitations of the type system.
And again, those limitations exist because we have support for gradual typing.
No. I’m talking about Rust-style typing. If your editor knows what type goes into a function, it needs to be able to deduce the type returned by the function. (That’s not strictly a correct description of Rust style typing, but hopefully close enough.)
Given any object, it returns a str. Would that count as being able to deduce the type?
In any case - none of this is at all enforced, and that can’t be changed for packages separately from scripts. If you want to enforce type annotations, do it for your projects, or for whichever projects you like, but it’s best done on a per-project level.
please don’t try to rules-lawyer me. That’s not productive. I am indeed aware that you can misinterpret what I wrote. As I wrote in the post you replied to:
What you’re pointing out here is actually a strength of Rust-style typing. Indeed you don’t need to think about what your function will return, your editor can already deduce that from what you’ve written. So an autocomplete (reliable, not AI) can do most of the work. That’s only possible because the editor can rely on the sub-functions also being typed.
It is still useful(/required?) to type-annotate the functions, but you can see that as a form of caching if you wish. The type analysis software caches its result as a type annotation, so that when another function uses describe it can use that cached result without having to analyse describe all over again.
I have no idea what you were intending. I have never written any Rust code. You can’t assume that we have the same background knowledge you do. What IS “Rust-style typing”? What is the benefit of it?
Python type checkers are expected to deduce things about the functions too. I genuinely do not understand what you’re saying here, probably because I don’t have the background that you do.
Yes, and as a result you and/or a bunch of people like you are pressurising those libraries to provide the annotations. The maintainers will be well aware that many downstream libraries and end users want the annotations.
If you are asking for some sort of system that would literally force them to provide the annotations then that is pointless because if they are not working on it then it is likely down to not having the resource to do so. What would the system do apart from refuse to import the library? How would that benefit you? How would that help the library with the enormous task of adding the annotations to an existing untyped codebase?
no, I didn’t use @wraps(func) while you did use it
on a side note, you evaluate the function which leads to some weird errors from python itselft, and not enforce_types
I didn’t think so far, ultimately since my idea was rather generic, it doesn’t quite matter much so long as it makes sense. Let’s call it “placeholder_module”.
Bad choice of words for which I sincerely apologize, I meant with a workflow which can be either global or per file (similar to the logging library, where you can set up your logging in the main file and it is set as the ‘root’ config). My idea was merely to provide in the stdlib an interface for the libraries to provide an unified workflow
My idea was something like
from placeholder_module import statically_typed
from mypy import config_type_enforcer
config_type_enforcer(enforce_by_default=True)
class MyClass:
code: int
def __init__(self, code: int):
self.code = code
@statically_typed(always = True)
def update_statically_typed(new_code: int):
self.code = new_code # Fails if new_code is not int
def update_not_statically_typed(new_code: int):
self.code = new_code # Default Python behaviour
@statically_typed
def update(new_code: int):
self.code = new_code # Default Python behaviour unless a type enforcer is configured with enforcing by default
Where each library would implement their own version of the type enforcer by inheriting by the abstract class TypeEnforcer which serves as a common interface for all libraries.
As so, the only additions to stdlib would be:
An abstract class TypeEnforcer
A decorator “statically_typed” which loads the current type enforcer and runs the type checking
Optionally, an assertion function to trigger the type checking manually
It is to note that the main goal of my idea is to have an unified decorator, so all other aspects (log structure, what to allow and what not, edge cases, …) would be library specific, so it wouldn’t be disruptive
In the instances you listed, your use of list[str] and dict are both probably more specific than necessary, you’re likely using the Sequence[int] and (Mutable)Mapping interfaces respectively. I do largely agree though that even if this were desired, the Python type system is inadequate for runtime interpreter enforcement. Anything involving metaclasses would almost certainly break down.
Sorry, I thought you were being sarcastic. Because in my mind it is implicitly understood that typing is part of the function declaration, and so a function that doesn’t declare it’s return type is obviously not typed. Even if it does satisfy the criterion that you can deduce the return type from the function body.
And also seeing the replies of the others, I do need to explain my experience of typing in Rust.
As an overview: Typing in Rust is less effort than in Python, and it gets you a lot more. However, mandatory typing can be unbelievably frustrating when (as in Rust) it prevents you from running half-finished algorithms.
It may sound strange that typing is easier when you don’t have the option to type something as dict[str, Any]. I’m finding that I can’t explain it very well. I know there are strongly typed languages where you have to declare the type of all variables. But in Rust, you only need to declare things that can’t be deduced by the compiler.[1] In practice this means that often you only need to type hint the arguments of a function and the return type. Due to multiple dispatch situations you sometimes need type hints in the middle of a function, but those can often be partial type hints, which reduces their burden considerably. In exchange, your IDE can tell you the type of all variables in all functions, which is incredibly freeing.
The function return types are a pain point. These generally can be deduced by the compiler, but you still have to write them, and if you’ve written the wrong one the program won’t run, not even in debug mode. So whilst developing a function you keep having to update the return type each time you want to test what the code is doing. When it’s hard to run half-finished function, it’s hard to develop code. I think it is possible to do better. [2]
I don’t know how I’d define Rust-style typing. I just know that the Rust typing system feels incredibly well designed, and nice. And when I’m struggling with Python typing, I wish that the Python typing system felt just as well-designed and nice.
A pedantic/petty reply would be that strongly typed languages (like Rust) demonstrate that you can enforce typing.
But that’s exactly the type of reply I protested against.
However, I don’t get what the proper interpretation of your statement is. That Python can’t enforce typing whilst still being Python? That it’s not possible for Python to enforce typing because Python is too flexible to be compatible with a complete typing system? Just that you don’t think Python should enforce typing?
That you can’t ignore or break the millions, probably billions of lines of existing Python code that isn’t typed. A substantial amount of that code will never be typed, because it’s legacy code that “just works”, was developed and is used in a commercial context where cost matters, and there’s no cost justification for adding types.
Rust included strict typing when it was first developed. I know Rust, and I know both the benefits and problems of its type system. Overall I think it’s fantastic. But it isn’t something that can be added after the fact to an existing language, especially not one that’s as popular as Python.
)