Distinguishing initialized type from assignment type

Ideas
1

Sometimes we give a variable an initial value of a type that we don’t ever want to use in later assignment. This happens a lot for function kwargs that use a sentinel like None. I wonder if Python could gain a new type class that could take this into account.

A not-thought-about-for-too-long example

def list_logger(level: str = "INFO", contents: Initialized[None, list[str]] = None):
    if contents is None:
        contents = []
    print((level + ':'), '\n'.join(contents))

list_logger(contents=["just an idea"])
# INFO: just an idea
list_logger(contents=None)
# error: Argument "contents" to "list_logger" has incompatible type "None"; expected "List[str]"

This new type would treat contents as Union[None, list[str]] as the function is entered, but as list[str] at any call sight, as well as at the first assignment to contents and then forward within the function.
The Initialized type (hopefully a better name will come) only has two index arguments. The second could itself be a Union, but the first should probably be a single type.

2

I daresay that Python could do this, but whether it should is another question.

Generally speaking, if a parameter is given a default of None, or some other sentinel value, the caller should be able to explicitly pass that sentinel as a way of explicitly saying “give me the default”.

So I don’t think I would like to see typing support for “you cannot pass the default value as an explicit value”.

But you could try the Typing SIG mailing list and see what folks there think of the idea.

https://mail.python.org/archives/list/typing-sig@python.org/

3

I believe this is what PEP 661 – Sentinel Values | peps.python.org intends to solve.

4

This idea was actually inspired by that PEP which although I think was done well and look forward to using complicates typing further. The recommended way to type a sentinel is Literal[MySentinel] which must then be combined with the actual type accepted so str | Literal[MySentinel]. Even using the new briefer typing union this is a long way to spell “pass in a string” which complicates signatures as in the large majority of cases won’t be passing the sentinel.

I forgot to mention that part of this new type would be the ability for documentation or autocomplete of the function to just show the non-initialized type in a similar way to how these tools can choose to show Annotated types as just the first index argument. Since users of these functions will in the majority of cases never want to need to pass the sentinel, and the library writer might not intend for that sentinel to be part of the public API. Type annotations will not stop users from actually doing this, and for those that want to both write fully typed code and use undocumented features, there is always # type: ignore

5

Would this handle cases you want,

from typing import overload

@overload
def list_logger(level: str = "INFO", contents: None = None):
  ...

@overload
def list_logger(level: str = "INFO", contents: list[str]):
  ...

def list_logger(level: str = "INFO", contents: list[str] | None = None):
  ...

This allows contents to be None when not passed, but forces contents to be a list[str] if passed. I think this meets your requirements.

As for forward within function, type narrowing already does that.

def foo(x: int | None):
  if x is None:
    x = 0

  reveal_type(x) # mypy will show you int here because it understands that x can no longer be None.

Type checkers try to narrow type of variables when they can.

6

That’s an interesting solution, I hadn’t considered overload for this use case. It wouldn’t cover all cases that a new Initialized type would, but it would cover the biggest motivating factor.

Unfortunately I don’t think overload in its current form is a solution. Catenating your overload with the original example, mypy doesn’t stop the use of None

from typing import overload

@overload
def list_logger(level: str = "INFO", contents: None = None):
  ...

@overload
def list_logger(level: str = "INFO", contents: list[str] = None):
  # had to give contents a default. strangely mypy didn't care it was again None here
  ...

def list_logger(level: str = "INFO", contents: list[str] | None = None):
    if contents is None:
        contents = []
    print((level+":"), "\n".join(contents))

list_logger(contents=["just an idea"])
list_logger(contents=None)
list_logger(contents="ABC")

$ python -m mypy my.py
my.py:18: error: No overload variant of "list_logger" matches argument type "str"
my.py:18: note: Possible overload variants:
my.py:18: note:     def list_logger(level: str = ..., contents: None = ...) -> Any
my.py:18: note:     def list_logger(level: str = ..., contents: Optional[List[str]] = ...) -> Any

Playing with this idea further, I came up with this, which gave the desired type errors:

from typing import overload

@overload
def list_logger(level: str = "INFO", contents: list[str] = []):
  ...

@overload
def list_logger(level: str = "INFO", contents: tuple[str] = ("",)):
  ...

def list_logger(level = "INFO", contents = None):
    if contents is None:
        contents = []
    print((level+":"), "\n".join(contents))

list_logger(contents=["just an idea"])
list_logger(contents=None)
list_logger(contents="ABC")

$ python -m mypy my.py
my.py:17: error: No overload variant of "list_logger" matches argument type "None"
my.py:17: note: Possible overload variants:
my.py:17: note:     def list_logger(level: str = ..., contents: List[str] = ...) -> Any
my.py:17: note:     def list_logger(level: str = ..., contents: Tuple[str] = ...) -> Any
my.py:18: error: No overload variant of "list_logger" matches argument type "str"
my.py:18: note: Possible overload variants:
my.py:18: note:     def list_logger(level: str = ..., contents: List[str] = ...) -> Any
my.py:18: note:     def list_logger(level: str = ..., contents: Tuple[str] = ...) -> Any
Found 2 errors in 1 file (checked 1 source file)

Yes, this is ugly: I couldn’t use None as a default in the overloaded definitions because then mypy accepted it as a valid call type. I also couldn’t use just one overloaded definition because that raises a mypy error error: Single overload definition, multiple required.

This would actually be a nice solution not involving any new python types, if type checkers allowed a single overload definition, and if they ignored the actual type assigned to kwargs in overloads (it currently seems to not care about the actual type in the final function definition, neither adding it to the valid types, nor raising an error about it not matching annotated definitions).

7

If you don’t want to allow passing None explicitly you could do this,

from typing import overload

@overload
def list_logger(level: str = "INFO"):
  ...

@overload
def list_logger(level: str = "INFO", *, contents: list[str]):
  ...

def list_logger(level: str = "INFO", contents: list[str] | None = None):
  ...

Overloads do not need to have same number of arguments. You can change signature a lot with different overloads. I occasionally have overloads where one argument missing implies other arguments must also be missing/required. This allows you to express stuff like this function has two mutually exclusive arguments/more complex relationships.

1 Like