Fall back to the typing.* namespace in lazy hint contexts

Typing
22

Don’t the globals and locals parameters to typing.get_type_hints() already allow for this?

[edited to add locals]

23

Imports are a fundamental part of Python, but so are builtins. The usefulness of these two concepts are complementery, not mutually exclusive. Imports allow us to use an unboundedly large amount of code, without having a single large namespace for everything, and builtins allow us to use a small curated set of fundamental tools, without having to use imports.

24

They allow modifying the name resolution yes, but if as proposed here, the default fallback is specified to be the typing module (or some other set of typing fundamentals), these tools would need to be updated with this default but possibly overridable set. I don’t think it would make sense to have static type checking be specified with this rule without also implementing it as a change in these functions.

25

This seems to argue that builtins are never useful. I think the prevalence of them in Python, together with the popularity of the Python language, is a much stronger argument in the opposite direction. Additionally, I could make a long list of other languages where a “const” keyword or similar exists as a language construct, not requiring any imports.

Of course a language could be constructed where every language construct other than imports themselves has to be imported, but I don’t see the value of this, and this language is clearly not Python.

26

Not exactly. I would say that I’m arguing against adding new builtins—not for removing them.

Is it not possible for your editor to add the imports for you?

27

Is it not possible for your editor to add the imports for you?

I’m not very pleased about making this thread about the capabilities of my IDE.

The points I’ve brought up is about simplifying the language, and the capabilities of advanced IDEs is not really very relevant towards that.

When your IDE automatically adds imports for you, how does it know to pick the right one? Does it really allow you to not think about imports?

1 Like
28

“We should not strive to make the language simpler, because we have sufficient tools to handle the complexities we’ve created so far”. This is just not an attractive status quo to me.

29

Why does this seem any more silly than the fact that this code requires an import?

from datetime import datetime

current_time: datetime = datetime.now()

(Edited to make the similarity clearer).

1 Like
30

It creates a drop-down widget and I select it the import target. It takes only a few keystrokes.

I understand that that’s how you see things. However, I think that adding implicit imports makes the language more complicated. It’s an implicit source of names, which we don’t have. The names are not in local or globals or builtins, but they resolve anyway? Seems like a surprising source of problems to me.

By all means, simplify the language. We just disagree that this is a simplification.

The reason that I’m suggesting that you improve your IDE is that that is one alternative to solving your problem. You’ve introduced a problem and proposed a solution. I’m suggesting another solution that solves the same problem with a different set of tradeoffs.

This is pretty common on the ideas list. One idea that’s come up many times is multi-line comments. One of the most popular responses is to improve your IDE so that it can comment and uncomment multiple lines at a time.

32

That’s fine, but I’ve iterated many times now that I am not coming here asking for support how to improve my workflow, so let’s stop talking about how to improve my IDE. Newcomers to Python or to programming overall do not have access to my IDE.

I think that adding implicit imports makes the language more complicated. It’s an implicit source of names, which we don’t have. The names are not in local or globals or builtins , but they resolve anyway? Seems like a surprising source of problems to me.

This is a valid concern, and moves the discussion forward :+1:

In many other languages, a similar feature is implemented with language-level constructs such as a const keyword or similar. The keyword is not an object that’s available for users of those languages to interact with, and I don’t think that’s common source of confusion. Similarly in Python, we have both keywords such as class and def that are not objects that can be interacted with in runtime code, as well as builtin operators such as |. Trying to interact with those raises SyntaxError.

Please also note that “implicit imports” is a bit of a weird way to phrase this feature, as at runtime with the proposed feature there would be no import.

Yes, the proposal clearly changes the semantics of type hints, but all changes are not bad. And there are plenty of other special contexts within the language with special semantics. I don’t think anyone would argue that that the formatting mini language, another special niche of the language, would have been any better if all of its features would have been required to explicitly import. Why do you think this reasoning doesn’t apply similarly to the special context of type hints? Why does it not make sense to similarly have small set of fundamental tooling ergonomically working out-of-the-box here?

33

Yeah, that makes sense, but I guess I find it confusing.

What happens if a typing construct is shadowed?

Sequence = Z  # I agree that this i a bad idea.

x: Sequnce  # does this come from typing or from the shadowed value?

What happens if a typing construct is deprecated? It’s a bit weird to figure out where you’re using it?

And what’s the future of annotations? Aren’t they going to be actual live objects (not strings). That means that you do actually need to implicitly import typing to implement this feature to produce the appropriate objects.

34

What happens if a typing construct is shadowed?

At least my initial idea here is that this valid, and that the shadowed name will be used. That is, in your example, the type of x would be equal to Z. If this wasn’t the case, I think the compatibility cliff with existing code would make the introduction infeasible. As proposed, the only breakage would be if you have code that somehow relies on raising NameError during evaluation of type hints.

And what’s the future of annotations? Aren’t they going to be actual live objects (not strings).

By “lazy hint contexts” in the title, I’m referring to type hints defined under PEP 649 and PEP 563 contexts, both of which are lazily evaluated.

That means that you do actually need to implicitly import typing to implement this feature to produce the appropriate objects.

Yes, you would need to have the typing module available to evaluate type hints. The two existing tools to do this live in the inspect and typing modules. For typing.get_type_hints(), typing is obviously already imported when you use it. inspect.get_annotations() would need to be modified to lazily import the typing module. It seems it would make sense to defer the import of typing until the first NameError occurs that would trigger a fallback lookup.

What happens if a typing construct is deprecated? It’s a bit weird to figure out where you’re using it?

Hmm, yes maybe, but would it actually become harder than current situation? This touches upon a topic brought up earlier in the thread that perhaps something more narrowly curated than the full typing module should be used (as to minimize the risk of available names becoming deprecated – fewer names less chance of deprecation). I think from the arguments I’ve made so far here, the strongest case lies in making the special forms available, as those are the closest to typing-level “language constructs”.

35

I think it’s important to take the history of typing in Python into consideration here. If typing would have been proposed in its current shape to begin with, I think it’s clear that it would have never been accepted due to its many unexplored complexities, complexities that have been explored and gradually formalized in the years since. A consequence of this is that every individual typing feature has been gradually introduced to the language. For healthy reasons of not wanting to commit to any of those features in a way that would be hard to roll back, most new typing features starts out with zero changes to the language, and usually as a name in the typing_extensions module. At that stage of introduction, the experiment that is carried out is aimed at finding out whether the typing feature proves to be compatible with the already existing sets of typing constructs, as well as to gauge its popularity.

Features are introduced in a way that cautiously guards the language from changes that later would prove themselves as problematic or incompatible in some way. This, however, also means that they are introduced in a way that does not maximize the ergonomics, or user friendliness of these features. It’s simply considered more important to leave room for the type system to evolve in a way that it stays compatible with itself.

After this initial experiment a few typing features now have evolved into next stages, and have been deemed stable enough and useful enough to justify changes to the language and its builtins:

The common theme of these is that take an existing typing concept, first proved through the process described above, formalized and proven useful and popular as features importable from the typing module. Only later deemed as justified to make changes to the core language for.

PEP 695 latter has this in its Motivation section:

While generic types and type parameters have grown in popularity, the syntax for specifying type parameters still feels “bolted on” to Python. This is a source of confusion among Python developers.

There is consensus within the Python static typing community that it is time to provide a formal syntax that is similar to other modern programming languages that support generic types.

And to answer the specific question you are posing here, I think this passage serves well. I think the argumentation posed in the quoted passage here applies equally well as an argument for the proposal I have made.

With this history in mind, it makes perfect sense why all the special forms must be imported from the typing module. It is part of the process that has allowed Python typing to mature and to gradually be integrated into the language.

However, though the early stages of this process yields us typing features that are proven popular and are proven to work sensibly in the type system, it yields features that are “bolted on to the language”. It feels silly to me, because it would feel equally silly to have to import the final keyword in Java, or to import the const keyword in TypeScript.

The comparison with the datetime import is perhaps enlightening. This is something that I’m not aware of any language implementing as a language construct. Additionally, I can point at a few caveats that make it seem like a very good idea that the datetime module is not a builtin (ambiguity of utcnow() method for instance). The datetime type imported is a normal type. It works like any other type in the language, regardless of whether defined by a user or in the standard library.

On the contrary, Final is a special form (not a type), that changes the semantics of the program. They are two very different things.

The proposal I am making is that the type hints, just like other niche contexts of the language, like the formatting language, would be improved by having a concept similar to builtins. A set of core features that is readily available. Just like this is proving useful in the core language level, I think it would improve the user experience of writing typed Python code to have “typing builtins” available in typing contexts.

Part of why I think it’s an attractive proposal is because it would have no impact on untyped code, and only a positive runtime impact on code that does not introspect type hints.

36

Since apparently noone else has said this: Yes, this is a very silly program. Because it does nothing.

Short scripts do not benefit all that much from type hinting in general. What exactly is the benefit you gain in the above program by adding an annotation instead of a = 1 (or A = 1 if you want to signal by convention it’s a constant)?

Larger programs that do actually benefit from typing are

  • probably going to have other imports as well
  • probably going to import some library that then imports typing (so you don’t actually gain the loading time saving)
  • have a decent chance of using more advanced features of typing that require runtime support, like NamedTuple or Protocol.

But it has the negative impact on all of python that the scoping rules get even messier. For example, Enum could have injected auto as a special name within the class scope, or even gone ahead and not require assignments in the class body to create members. This could be perfectly valid semantics:

class Foo(Enum):
     bar, qux, corge

But it was decided against that, and while I haven’t seen the original discussions, I am sure that a decent factor was to reduce the surprises this syntax entails.

Currently, there is the simple rule that if you want to lookup a name it’s either defined within that module or imported somewhere at the top. Ofcourse, *-imports exists, but those aren’t well liked for at least partially the reason that it becomes harder to reason about where a name comes from.

Is it now a mistake to have a class that is named like something from typing even if you haven’t imported it? Currently, a: FInal = Final() is perfectly valid assuming you have a class Final from somewhere. With the new semantics you are proposing, this might still be unambiguous, but it’s more confusing for human parsers.

2 Likes
37

But it has the negative impact on all of python that the scoping rules get even messier.

No, scoping rules are not changed for “all of Python”.

Is it now a mistake to have a class that is named like something from typing even if you haven’t imported it?

No, semantics for that are unchanged, I already explained this here: Fall back to the typing.* namespace in lazy hint contexts - #33 by NeilGirdhar