Imported Final variable

[erictraut]

When a variable is marked Final in one module and imported by another module, should the imported symbol also be treated as Final? In other words, should the imported symbol inherit the Final type qualifier and the behaviors associated with Final?

Consider this example:

# mylib/submodule.py

from typing import Final
Pi: Final[float] = 3.14

# mylib/__init__.py

# This redundant import statement indicates that Pi
# should be re-exported from the top-level module.
# Should Pi be considered Final?
from .submodule import Pi as Pi

# test2.py

import mylib
mylib.Pi = 0 # Should this be allowed?

# test1.py

# Should Pi be considered Final?
from mylib import Pi
Pi = 0 # Should this be allowed?

from mylib import Pi as OtherPi
OtherPi = 0 # Should this be allowed?

I’d like to see if we can agree upon the intended behavior and amend the typing spec so it documents and mandates this behavior for type checkers. This question came up recently in this pyright issue.

I think there are several different behaviors we could settle upon:

1. The Final type qualifier is never inherited when a symbol is imported.
2. If a module imports a Final variable by name or through a wildcard import, the imported symbol inherits the Final type qualifier. Any attempt to assign a different value to this symbol should be flagged as an error by a type checker.
3. A Final type qualifier is not inherited upon import except in cases covered by the rules defined in this section of the typing spec. These rules apply only to stub files and modules within a py.typed package. They do not apply to non-py.typed libraries or local imports.

I can make reasonable arguments for all three of these options, but I slightly prefer 2 over the others. It seems to me what most developers would expect, and it’s less complex and easier to explain than 3.

Pyright and mypy currently implement 2, although mypy does not consistently detect and report reassignment of Final variables. This appears to simply be a bug or missing feature in mypy.

# target.py

ten: Final = 10

# test.py

from target import ten
from target import ten as ten_renamed

# Pyright and mypy: cannot reassign ten because it is Final
ten = 1

# Pyright: cannot reassign ten_renamed because it is Final
# Mypy: cannot reassign ten because it is final
ten_renamed = 1

# Pyright: cannot reassign ten because it is Final
# Mypy: no error
from target import ten

# Pyright: cannot reassign ten because it is Final
# Mypy: no error
for ten in range(10):
    pass

It looks like this difference can be explained by the fact that mypy doesn’t enforce Final in a number of situations where the typing spec indicates that it should.

from typing import Any, Final

z: Final[Any] = 1

from typing import List as z  # Mypy allows this

for z in range(10):  # Mypy allows this
    pass

match 2:
    case z:  # Mypy allows this
        pass

Which of the three options do you think we should adopt? Or is there another option that I haven’t considered?

[oscarbenjamin]

@erictraut

I can make reasonable arguments for all three of these options, but I slightly prefer 2 over the others. It seems to me what most developers would expect, and it’s less complex and easier to explain than 3.

I would definitely expect 1:

The Final type qualifier is never inherited when a symbol is imported.

Importing a name is just an assignment to that name. You can do:

a: Final = 3.12
b = a
b = 2

The rule for from foo import a as b should be no different from b = a.

[stroxler (Steven Troxler)]

I agree with @oscarbenjamin, I don’t really think Final should propagate across exports. Qualifiers are about what you can do with a specific reference, and from mod import name is creating a new, owned reference to the same value.

I think it might make sense for a linter (or an optional type check error) to complain on most overwrites of imported names, because the examples where an imported name get overwritten later are likely bugs where a user didn’t mean to overwrite the name. But whether to have an extra check that treats all imports as non-overwritable is a separate question from whether to propagate Final.

[carljm (Carl Meyer)]

My initial thought was the same as Oscar and Steven.

However, if you do want to re-export a final symbol as final, it would be slightly painful under (1):

from mod import final_symbol as _final_symbol

final_symbol: Final = _final_symbol

While imports are not really different from other assignments at runtime, they are different from other assignments in that you can’t apply an annotation to them directly. Which suggests that it could be reasonable to treat them differently in a case like this.

I would be OK with either (1) or (2). I would also prefer not to add more special cases for stubs.

[JukkaL (Jukka Lehtosalo)]

I think Final qualifier should be inherited on import. That’s how it behaves already in most cases in mypy (except for some cases where mypy doesn’t enforce it correctly). Mypy was the reference implementation for the original final PEP, and I believe that option 2 was the desired behavior, even if it wasn’t explicitly spelled out in PEP 591.

I generally prefer not to break backward compatibility unless there is strong evidence that the new behavior is significantly better for most users. Changing to option 1 would break backward compatibility, since mypy infers types differently for Final variables – the literal value is lost in cases like X: Final = 5 if X is imported unless the import preserves the Final qualifier.

Assignments are treated in many different ways by type checkers depending on context (e.g. type aliases, named tuple definitions, …) so I don’t think consistency by itself is a strong argument for choosing option 1.

[Daverball (David Salvisberg)]

While I agree that option 1 feels more consistent, it doesn’t seem like what’s most useful. While option 2 may occasionally cause some friction, I think that’s actually a positive, not a negative.

If someone decided to make a module-level attribute Final they probably did it for safety, since they want to preserve some invariant, so implicitly shifting the responsibility for preserving that invariant to each downstream user, just seems bad. I don’t think most people will think about whether or not an imported symbol was originally declared Final, unless they’re forced to.

If you want to create a non-Final reference you can still do so, by manual assignment. That should sufficiently signal intent, that you’re taking ownership and responsibility of that reference. An import doesn’t seem like a strong enough signal on its own.

[mikeshardmind (Michael H)]

Option 2 seems right to me. Final is a statement of developer intent requested to be enforced by the type system, this is often used to ensure specific invariants, but even if it wasn’t, the other options violate a developer’s explicitly stated intent.

[erictraut]

Thanks for the feedback. I’ve created this PR that adds a new section to the typing spec to clarify the intended behavior for imported Final variables. The wording is based on option 2 above. Feel free to add comments to the PR directly if you have questions, concerns, or suggested improvements.

[carljm (Carl Meyer)]

@mikeshardmind

Option 2 seems right to me. Final is a statement of developer intent requested to be enforced by the type system, this is often used to ensure specific invariants, but even if it wasn’t, the other options violate a developer’s explicitly stated intent.

I am OK with option 2, but I don’t understand this assertion. If I say PI: Final = 3.14 in lib.py, and then main.py has from lib import PI, the developer’s “explicitly stated intent” is that the symbol PI in the namespace of module lib is final and cannot be reassigned. I don’t think there is any obvious intent here that the symbol PI also be not-reassignable in the namespace of module main. I can certainly imagine scenarios where that is not the intent:

from lib import LARGE_CONSTANT

if TESTING:
    LARGE_CONSTANT = <smaller constant used in tests>

The proposal here is about trying to guess what is more likely the intent in a case where it’s not explicit, and we are creating an exception to the usual rule that qualifiers apply to the annotated symbol and don’t travel with the type.

I’m OK with that in this case, because I do think it’s quite likely more often the desired option, and the alternative if it’s not is pretty easy:

from lib import LARGE_CONSTANT

my_constant = <smaller constant used in tests> if TESTING else LARGE_CONSTANT

[mikeshardmind (Michael H)]

The easiest example of this is the one where a module has a private implementation but reexports certain things (including constants) via re-export, which you yourself gave.

I don’t think something ceases to be a constant reference just because it’s imported elsewhere, and forcing people to think of it this way probably isn’t useful even if it is more accurate to how the import system actually works. I think the extremely narrow view of Final, which applies only to a symbol in the exact namespace it was declared in doesn’t match developer expectations. It can become more important that it isn’t reassigned in the case of mutable containers used for caching, as the reassignment could break expectations that each module is referring to the same container. Mutable containers aren’t the only issue. another is shared (via import) threading locks, an example of this is visible with asyncio’s implementation and a shared threading lock.

[erictraut]

Thanks everyone for your input.

The typing council has signed off on the proposed clarification, and it has been merged into the typing spec.