The text of PEP 604 lacks clarity around the desired end state:
foo: int | None
Optional[int] can be used forever moreI should point out, I pretty unhappy with it no longer being okay to spell the above as foo: int = None, but I’d prefer option 1 over option 2, so I was wondering what the official (steering council?) position is on this?
As a disclaimer, I’m not a typing expert, just a PEP editor. However, I’m not sure I’d say that the PEP “lacks clarity”, which implies a defect in the PEP; rather, it simply introduces a new typing construct without deprecating or removing existing constructs, leaving the decision on that to future PEPs (or other similar such community processes). This seems generally reasonable and consistent with what similar other PEPs have typically done, given it would require another whole discussion, and any such removal would have to be implemented at least 5-6+ years from the time of the PEP’s acceptance (or around 2-3 years from now) once all non-EoL versions of Python support the new construct in question, and presumably usage patterns and community feedback over that time would be the primary drivers of such a decision, data on which would only be available after several years have passed.
Also, FWIW, Union is what | directly replaces, whereas Optional is, or course, merely a shorthand syntax for a specific type of Union.
This PEP has has no direct relation to whether typecheckers should implicitly assume None in that case; that was instead due to a change to PEP 484. However, if you prefer that implicit behavior, you should be able to just enable it in your type checker, e.g. --implict-optional for mypy (or if your type checker doesn’t support it and it matters to you, consider switching to one that does).
I prefer to use Optional because I find the double None caused by the union syntax to be weird
def foo(
bar: Optional[int] = None, # Nice
baz: int | None = None, # double None looks weird...
): ...
But that’s just personal preference.
On a sidenote, I find it amusing that the “repetitious” syntax has fewer characters.
I would hope that Optional[int] remains valid, as there’s little or no benefit to deprecating it, and it’s unnecessary churn for projects that use it. So it’s simply two ways[1] of expressing the same thing. Yes, projects can have a style-guide policy that one or the other must be used, but they don’t have to, any more than they have to express a preference between any other equivalent constructs.
Personally, I prefer Optional[int] as it expresses intent more clearly to me.
Actually, there’s also Union[int, None], so two of three or more… ↩︎
I thought the same, until I saw a lot of incorrect use of Optional by people who are “cargo cult”-typing and never read or understood the docs:
# x is an optional coordinate (default: 0)
def f(x: Optional[float] = 0.):
...
Because of that, I have come to prefer the explicit union. It’s a statement about types, after all, not about semantics.
I’m not a steering council member, but I am a typing.py maintainer (and a typeshed maintainer), so I think I’m in a reasonably good position to summarise the thoughts of the typing community.
I don’t think there are any plans to officially deprecate Union or Optional in the near future. That would cause a great deal of disruption, that we don’t really have the appetite for. We discussed this in a typing-sig thread in December 2021 – re-reading the thread, it seems like we might consider deprecating them in, perhaps, 3.15. But there wasn’t really a strong consensus around even that point, and it would significantly complicate the implementation of a bunch of typing internals[1].
However, we would encourage using int | None rather than Optional[int] if it’s possible for you to do so, and we state as such in the docs for the typing module.
From a typing perspective, Optional[int] is a very bad name for the feature, because it implies to the user that it expresses something completely different to the thing that it actually means to the type checker. For the user, it implies that the argument is… optional, i.e., it’s okay to not pass a value to that parameter. But to the type checker, the meaning is identical to Union[int, None]. That has a very different meaning, as there’s all sorts of not-None defaults that you could give an argument that would make the argument “optional” without implying that it’s okay to pass None as the argument:
from typing import Optional, Union
# This function has a *required* argument that could be `int` or `None`.
# It's required in that you *have* to pass in a value to the `arg` parameter,
# or the function will fail
def a(arg: Optional[int]) -> None:
pass
# This function has a *optional* argument that could be `int` or `None`.
# You can call the function without passing in a value to the parameter,
# and it will work fine
def b(arg: Optional[int] = None) -> None:
pass
# This function has an *optional* argument that can only be `str`.
# You can call the function without passing in a value to the parameter,
# and it will work fine
def b(arg: str = "foo") -> None:
pass
If we were writing PEP 484 from scratch in 2023, I don’t think we would include the Optional symbol in the typing module. We would simply tell people to use Union[X, None] to express “it could be X, or it could be None”.
For example, types.UnionType does not support forward references, whereas typing.Union does – this causes complications in various places. ↩︎
lol, once again I’m reminded that typing is way too hard for the average user like me to get right 
And I’m sorry for that. But this is why we’d like to encourage people to use the newer syntax where possible – because “soft-deprecating” Optional means that there’s one less confusing, badly named concept for people to grapple with 
It is also problematic that there is no way to specify an actual optional argument with Callable:
from typing import Callable, Optional
def f(x: Optional[int] = None) -> int:
return 1
g: Callable[[Optional[int]], int] = f
v = g()
With mypy this is rejected:
$ mypy t.py
t.py:8:5: error: Too few arguments [call-arg]
v = g()
^~~
Found 1 error in 1 file (checked 1 source file)
However pyright accepts it.
Both type checkers can be made happy with a Protocol but that seems unnecessary given that optional arguments are such a common thing in Python:
from typing import Optional, Protocol
class OptionalInt(Protocol):
def __call__(self, x: Optional[int] = ...) -> int: ...
def f(x: Optional[int] = None) -> int:
return 1
g: OptionalInt = f
v = g()
But conversely, int | None also isn’t (to me, at least) an intuitive way to annotate a parameter that’s got to be an integer (although omitting the parameter and having a default get calculated at runtime is fine). Note that passing None explicitly to get the default is not (in my mind, at least) a published part of the API, it’s simply an artifact of the implementation (I may choose to switch to an opaque sentinel value at a later time).
It seems your complaint here isn’t with PEP 604, but with the decision in Should we change PEP 484 to disable implicit Optional when default = None? · Issue #275 · python/typing · GitHub and PEP 484: Do not require type checkers to treat a None default specially by JelleZijlstra · Pull Request #689 · python/peps · GitHub to remove “implicit optional”, the feature where arguments that default to None would implicitly be Optional. I believe that decision was made while Guido was still BDFL, so it wasn’t the SC’s call at the time.
This decision really has nothing to do with PEP 604. We can revisit it if users feel there is a strong argument for the “implicit optional” feature.
It’s a somewhat published part of the API, whether that’s intentional or not, in that it’s visible if anybody calls help(function_with_none_default) in the REPL, or otherwise uses inspect.signature to introspect the function.
I understand that. But to me, “there’s a way to annotate this, but it’s not entirely intuitive” still feels better than “there’s a way to annotate this, and it seems intuitive, but it actually means something completely different to what you’d intuitively think”.
I’m not sure if that was directed at me or the OP, but personally, I don’t think “implicit optional” is a solution either, as it’s special-casing None, which doesn’t interact well with explicit Sentinel objects (such as those proposed in PEP 661).
I disagree. The API is what I document it as, and if I don’t document None as a valid argument, it’s not supported. It doesn’t matter if tools display some of the internals, that doesn’t make them supported. But type checkers are supposed to help me validate that my code is used as intended, and so I want them to ensure that users either don’t supply the argument, or supply an integer value, as that matches the API I have designed. (Or they explicitly suppress the check to confirm they know what they are doing).
I’m fine with the answer “type checkers can’t support that API” (although I’ll note that won’t stop me from writing APIs like that[1]), but I’m not fine with being told the current situation addresses the issue - it doesn’t, it just pretends the issue doesn’t exist. And I don’t want to suggest that this is a fatal flaw with type annotations, or anything like that - they do an amazing job of capturing far more of the “programmer’s intent” than I honestly ever expected. But they still fall short in some cases, typically where Python is particularly dynamic (although that’s not really the issue in this case). In those cases, I think we all need to remember that typing is intended to be “gradual” - use it when it helps, but don’t make it a straitjacket that prevents using features of the language that work, and have been successfully used without type annotations for years.
Practicality vs purity is fine here, if int | None is good enough, then I’m not going to argue. But it’s still only “good enough”, and it doesn’t always capture my intent, whereas in some cases Optional[int] = None does, and I’d happily use it (note, I wouldn’t omit the = None and I understand that doing so is misleading). Again, practicality vs purity (in this case, saying what I mean even though it technically could be misused).
But going back to the original question, as a result of this discussion, I stand by what I said that both int | None and Optional[int] say the same thing, and which to use is (and should remain) a matter of developer choice. I will add the caveat that neither is perfect - int | None = None implies a commitment that wasn’t intended, whereas Optional[int] = None is misleading if the default value gets removed in a subsequent code change.
I know “strictly optional without a default” is really tricky to implement and use in Python, but I still think the more relaxed (“consenting adults”) form where the default sentinel is private but can be used on the understanding that you’re relying on an implementation detail is an entirely valid design. ↩︎
I’m not sure it was ever claimed by anybody that the current situation was perfect, and fully addresses the issue. I agree that annotating =None arguments with int | None can be unintuitive to users who are new to typing, and is perhaps not an ideal situation.
What I have claimed, and continue to claim, is that the current situation is better than it was, and that we’re trying to improve the situation by encouraging users to use PEP 604 syntax where possible.
I agree with this. I’m not sure who this comment is directed at.
It’s certainly better in the sense that having more options (with different nuances) is an improvement. But if you want users to move away from Optional[int] = None, then I personally don’t think your arguments are persuasive. Which is fine by me, I’m happy to not be persuaded. I’m also happy to explain why I think the way I do, if it helps you understand the disconnect, but I’m equally fine to just drop the discussion and stick with using Optional where I think it makes sense.
I find your argument against “implicit optional” to be quite persuasive—that it special-cases None as the one blessed sentinel of an optional parameter, and that it wouldn’t work with the proposed explicit Sentinel objects (which I also would like).
However, it seems to me that it applies in equal measure against the Optional annotation as well for the same reasons: because it inherently special-cases None as the blessed sentinel of an optional parameter, and it wouldn’t work with the proposed explicit Sentinel objects—at least without making a backward-incompatible change to it’s meaning, or inventing yet another new, typing-specific symbol for it, both things all involved seem to agree should be avoided.
It seems, though, that | with the proposed explicit sentinel (in place of None) could perhaps be close to the better solution you are looking for, as it both avoids these problems and also sends that clearer signal that it is a sentinel and is not intended to be passed by users as a normal value—all without having to invent another new typing-specific symbol or change the meaning of the existing one.
I am not in favor of a separate Sentinel class a la PEP 661. We already have None for that.
That’s a fair point. Although just to be clear, I’m not particularly in favour of PEP 661 style sentinels - if I wanted something more opaque than None, what I would typically use would be something more traditional like a singleton instance of object() - and that can’t be typed properly by any form of int | <something> annotation.
The reality is, I think, that it’s not possible to easily annotate the type of a function that’s intended to be called only as either foo() or as foo(int) - and by “easily” I mean, “without resorting to something like overloads, which I’m aware of but have no idea how to use”. And yet it’s trivial to define such a function in untyped Python, simply by using a default of None that isn’t intended to be ever supplied by the user, just like a non-integer argument is never intended to be passed. However, by promoting int | None as the “correct” way to annotate this, we normalise the idea that placeholders are part of the public interface of the function (which in turn leads to the interest in adding explicit sentinel types á la PEP 661).
All of which has drifted very far from the original point of this thread. But I’m afraid I’m still of the opinion that, given that there are two semantically identical ways of spelling “an argument that is either an integer which can be omitted, or None (meaning ‘use the default’)” - Optional[int] = None and int | None = None, then I prefer the former, for the following reasons:
| None = None repetition.None” semantics.Neither form allows using a sentinel other than None, and both still require the = None default. So I don’t see a distinction based on either of those aspects.
Conclusion: I’m not in favour of seeing Optional get deprecated, nor of | None being pushed as the “one true way” of annotating this type of argument. However, I’m not sure anyone is actually proposing to do either of those things[1] so I’m happy enough with the current status quo.
Although I’m getting the impression that “int | None = None is preferred” could easily drift into “is recommended” and from there to “is the correct way”, and then to being enforced by linters… ↩︎
“Can be omitted” is only meaningful right in the one location of a parameter list, though. Consider:
def f(stuff, end=None):
if end is None: end = len(stuff)
...
_usage_stats = collections.Counter()
def g(stuff, end=None):
_usage_stats[end] += 1
f(stuff, end)
What is the data type of _usage_stats? Clearly it’s the same data type as the end parameter itself, mapped to integers. But that isn’t “an optional integer”; it’s “an integer or None”.
So IMO the “can be omitted” semantics, which you like, are deceptive, but the “could be None rather than this type” is the truth.