Building on the ideas from PEP 505 and after many discussions over the years, I’m happy to present PEP 824: None-coalescing operators. Please be sure to also check out the discussion for the companion PEP 823.
Abstract
This PEP proposes adding two new operators.
The “None-coalescing” operator ??
The “None-coalescing assignment” operator ??=
The general idea is to provide a conditional operator, similar to or, which instead of truthiness, checks for None values.
The “None-coalescing” operator evaluates the left-hand side, checks whether it is not None, and if not, returns the result. If the value is None, the right-hand side is evaluated and returned.
The “None-coalescing assignment” operator will only assign the right-hand side to the left-hand side if the left-hand side evaluates to None.
They are roughly equivalent to:
# a ?? b
_t if ((_t := a) is not None) else b
# a ??= b
if a is None:
a = b
Just to clarify, the splitting of PEP 505 into these two PEPs is so that they can be considered independently? As in, we could accept one but not the other?
I think using ??= is a good idea. When I write code, I don’t use conditional expressions very often, for the reasons the PEP describes. In the code I work on, the difference between age = user.age ?? "unknown" and age = user.age or "unknown" feels small, since 0 is usually not a value I need to keep. That’s only my own experience, though. Using ?? does make the check more precise, because it filters out only None.
Of the pair of proposals, I’m more enthusiastically positive about this one. I’m glad to see the intentional limiting of both to handling exactly None.
Changing the existing None checks I have where 824 could be used is not something that would push me to upgrade minimum python versions any faster, but it is a real improvement and may help prevent people from using or when there is a potential falsey value they had not considered.
I would be very glad to see this accepted, and I am also glad to see it limited to exactly None. Extending to sentinels or other objects would replicate the pitfalls of __bool__.
Just a note on the precedence:
# x or y ?? 2
(x or y) ?? 2
# "Hello" if x ?? True else 0
"Hello" if (x ?? True) else 0
and
Furthermore, setting the precedence between or and conditional expressions matches other languages which have implemented the operator, like JS [8] and C# [10].
This is correct, but the JS and C# examples are interesting in that x || y ?? 2 is a type error in C# and a syntax error in JS. It may be worth exploring whether other languages permit intermixing with boolean operators, or if Python would be unique in this respect.
That said, I find the JS behavior surprising and annoying, so agree with the current design that parentheses should remain optional when precedence matches intent.
Thank you for putting this PEP together. I support this. I think PEP 824 is clearer than earlier versions like PEP 505. I think splitting this + PEP 823 was a good idea. Like others in this thread, I appreciate the intentional decision to only handle None and not anything false-y.
I think that, in practice, I would have more of a use for PEP 823: None-aware access operators ; but both PEPs describe features I miss every time I switch over from TypeScript.
What if both Coalesce operators had their own dunder methods to change the check/assignment for None to something else like a custom Sentinel object? Would that be redundant or overly confusing?
Something like:
class Foo:
MISSING = sentinel("MISSING")
def __init__(self, bar=MISSING) -> None:
self.bar = bar
def __cgetattribute__(self, name, fallback): # ??
# None-coalescing becomes MISSING-coalescing
return fallback ?? self.name
def __csetattr__(self, name, value): # ??=
# None-coalescing assignment becomes MISSING-coalescing assignment
if self.__cgetattribute__(self, name) is MISSING:
super().__setattr__(name, value)
obj = Foo()
obj.bar ?? "unknown" # Returns "unknown"
obj.bar ??= "bax" # obj.bar is now "bax"
obj.bar ??= "baq" # obj.bar is still "bax"
obj.bar ?? "unknown" # Returns "bax"
I’m +1 or maybe +0.5 on this (PEP 823 would make a bit more of a difference for me), as a nice pragmatic convenience. I think it makes sense to keep the scope narrow and behavior simple by targeting only None.
My only question is… how did you decide on this color for the bikeshed? When defining new syntax fore a feature there are always lots of possibilities for how to “spell” it, especially when introducing the new ? symbol. Is the rationale basically that other languages use these two spellings for the same concepts?
Massive +1. I’ve written libraries for dealing with APIs with hundreds of nested fields, most of them optional, and I’ve spent a lot of time making sure the user only has to deal with typed objects that are nice to work with in Python. That includes not making them deal with optionals or missing data unless absolutely necessary. These (PEP 823 & PEP 824) operators would make my libraries’ code much nicer by eliminating a ridiculous amount of repetitive None checks.
The question how to read the expression out loud was brought up during review. These are just some suggestions we did come up with. If there are other better options, I’m happy to include them. Just keep in mind that these are only suggestions. If the PEP is accepted, I’d think some option might emerge organically anyway which can but doesn’t need to be the suggested one in the PEP.
Sorry, I was referring to the actual syntax. i.e. how come a ?? b and not say a ? b, a ?| b, a coalesce b, or some other character combination. I didn’t spot anywhere in the PEP explicitly stating this other than the rejected idea about a keyword.
My bad. Almost all languages with coalescing operators use ?? and ??=. AFAIR all prior discussions, including PEP 505, used them as well.
Why ?? and not a single ?: The None-coalescing operator is a conditional operator. As such it’s more closely related to and / or than binary operators +, -, &, |, … Looking at other languages again, and is often spelled as &&, or as ||. Both use two chars to differentiate them from their binary counterparts.
+1 from me, I was very excited to see this idea get revitalized as a new PEP.
@a-reich While I usually like that Python generally prefers keywords over symbols (like and/or), I think in this case coalesce doesn’t provide much benefit mainly because anyone learning Python and unfamiliar with the concept might not immediately grasp what coalesce is supposed to mean in this context, and although the “formal” name for the operation is nullish-coalescing, I would assume most would read a ?? b aloud not as “a coalesce b” but “a if not null, else b”, which is basically already what we have now in the form of a if a is not None else b, so an already-established symbol I think makes more sense as the choice for that shortcut.
People suggesting other syntax options should also explain why this feature should use anything else but the notation that is already widely used in a dozen of other major languages. The value of reusing an existing convention is high, and the cost of breaking that expectation for newcomers who are already familiar with the convention is also high and should be justified.
This is not some experimental feature that is breaking a new ground in programming and therefore needing a brainstorm over syntax.
The ?? syntax has a very natural pairing with ?., and since they are two parts of what should usually be the same expression, it makes sense for both to have some level of consistency.[1]
By “natural pairing” I mean that a?.b ?? c should be a typical usage, where None is only really a special intermediate value and not an intentional result. Translated into current code, this would be a.b if a is not None and a.b is not None else c[2]. Both expressions are pretty good in this respect - the former uses punctuation operators and the latter using textual operators, and the precedence works as each naturally reads.[3]
It’s irrelevant since I’ve withdrawn my own efforts on this, but I wouldn’t have split the PEP up into two. The value of ?? is strongest when ?. exists, and with either half of the syntax missing, the value quickly becomes zero or negative. ↩︎
Though, of course, I don’t prefer either syntax over a traversal helper function or a data model that simply doesn’t allow Nones to reach my business logic. ↩︎
When defining new syntax fore a feature there are always lots of possibilities for how to “spell” it, especially when introducing the new ? symbol. Is the rationale basically that other languages use these two spellings for the same concepts?