Revisiting PEP 505 – None-aware operators

PEPs
224

??= would be very useful for evaluating default arguments. This is a plus from me.

def foo(a=None):
   a ??= default_a()
   ...
1 Like
225

Yeah yeah. As Guido already said, it’s useful. A bit cryptic IMHO and only a one line saver, but whatever. I will not start an holy war for that :rofl:

Side note: I was convinced many months ago about the usefulness of making ? a “postfix” operator that operates also on mydict[k]? and more.

But I never expressed my opinion because I felt that the proposal was controversial. I was against at the beginning too…

So I think that removing it from the current PEP it’s a definitely a good idea. If this PEP will be refused anyway, there’s no point of making it complicate.

Just my 0.0.1 cents.

226

IMO it’s best to start a new PEP.

??= is handy because of DRY.

x.y = x.y if x.y is not None else default()

7 Likes
227

I suppose that in 99% of cases it would be .get(0)or .get(-1) (i.e., again: some stuff vs. no stuff). Yet, you are right it is too niche to be comparable with dict.get(). You have convinced me it’s not worth including in the basic proposal.

228

I’m not necessary against the motivation for wanting these three new operators, but I am uncomfortable with the readability of the operators in practice, as described in PEP 505. I’m also not sure they all fit under the rubric of “coalescing operators”.

Here’s some of the ways I’m thinking about this. They may not be the choices a future PEP author goes with, but I want to provide them so you can think outside the box of the 10 year old PEP 505.

?? binary operator

This says, if the LHS evaluates to None, return the RHS. To me, this has a clear, stronger keyword alternative: otherwise:

x = a otherwise b

Sure it’s a little more verbose, but its quite readable and to my eye, doesn’t take the higher cognitive load for understanding its meaning.

??=

PEP 505 describes this as “augmented coalescing assignment” but I don’t think that’s really what it is. It’s not really augmenting in the way that += and friends are. This is really a “default assignment” operator. It says, if the LHS is already None, assign it the value of the RHS.

If otherwise is chosen instead of ??, then the mnemonic connection to ??= is broken, and we have other options for this operator:

  • Just use otherwise and a little DRY, such as a = a otherwise b. In the PEP 505 rejected section, the examples don’t change other than substituting otherwise for ??. This one seems the cleanest to me, saves us one new operator, and the minimal DRY isn’t so bad.
  • otherwise= as in a otherwise= b. Keeps the mnemonic association with the otherwise keyword[1]. I both like and don’t like the = at the end of the word. It looks odd because we don’t have any other word= analogs, but it is also clearly denoting assignment.
  • defaults to as in a defaults to b. Super clear intent I think, but there’s no direct connection to “assignment”. Given that this is a statement rather than an expression, maybe that’s not too bad. Otherwise[2]…
  • default= as in a default= b. Perhaps the better choice than otherwise= if you like the assignment mnemonic but want a word that’s closer to the actual meaning, and don’t mind the mnemonic differs from otherwise.

?.

This one is tougher if you’re married to modifying the dot operator. Let’s for the moment break that mental model and look at two alternatives:

x = a?.b?.c?.d
x = a?.b.c?.d

Is there really value in saying that b.c must evaluate to non-None in the second case? I’m not so sure. Which leads me to wonder if what you really care about is safe attribute access for the entire chain of attributes, rather than individual and independent safe attribute access.

This bothers me because I think a) it will be pretty easy to forget one of the ? in the middle of an attribute chain, b) it will be pretty difficult to pick out where that ? is missing, c) It will be easy to confuse ?. with .?[3], d) all the extra symbols lead to line-noise-ism.

So what if what you really care about is that the entire attribute expression is “safe” in that if any attribute in the chain is None, then the entire access chain is short-circuited and the value of the expression becomes None?

That gives us more alternatives to play with. The first thing that comes to mind, alluded to as a rejected idea in PEP 505, is a maybe keyword rather than a built-in.

x = maybe a.b.c.d

maybe applies to the entire following expression, and changes the semantics to perform “safe” attribute access through the entire chain.

Going a step further there are other keyword alternatives that might work better:

  • try as in x = try a.b.c.d. I think that would be syntactically unambiguous from try:.
  • safe as in x = safe a.b.c.d
  • guarded as in x = guarded a.b.c.d

Even better perhaps is to connect to earlier suggestions, and use a postfix operator.

x = a.b.c.d otherwise None

I.e. Appending otherwise to an attribute access expression means “do it safely, with None short-circuiting, and if short-circuited use the RHS of the otherwise”.

The one semantic ambiguity is when the attribute chain resolves, but the last item is actually None. In that case, x would still be None but you wouldn’t know whether that’s because of a missing attribute in the middle of the chain, or because the last attribute was actually None. I’m not sure whether or not that would matter in practice.

Summary

Again, I’m throwing these thoughts out there because while I acknowledge the problems that PEP 505 is trying to solve, I think we have better – or at least additional – syntactic choices we can make now that we couldn’t make back in 2015.

  1. I’d consider otherwise= as a separate new keyword ↩︎

  2. pun intended ↩︎

  3. and in fact the first time I wrote the above examples, I made exactly that mistake; it’s clearly not obvious which is correct ↩︎

18 Likes
229

I agree that a keyword would be preferable to ??. I read the PEP for the first time recently and it took me a good while to figure out which side of the operator was which and what it actually meant.

I would also propose

x = a else b

As a less verbose but (imo) equally readble option.

Of course, re-using a keyword may come with other readability disadvantages.

2 Likes
230

I’m not a fan of the question marks from a readability standpoint myself.

If needed this can currently be implemented as:

class default:
    def __init__(self, value):
        self.value = value
    
    def __ror__(self, obj):
        return self.value if obj is None else obj

a = None
a |= default('a')
print(a) # a
a |= default('b')
print(a) # a

which is arguably no less readable.

I like the idea of a maybe keyword.

May be ambiguous when in a ternary operator though:

x = a if b else c else d # is (b else c) evaluated or is (c else d)?
7 Likes
231

Certainly! It would naturally need to be considered in all the other places else is valid.

That said, I think it is similarly confusing with ?? or otherwise?

x = a if b else c otherwise d # is (a if b else c) evaluated or is (c otherwise d)?
x = a if b else c ?? d # is (a if b else c) evaluated or is (c ?? d)?

Iirc, the original PEP suggested making ?? more tightly bound than other operators, so this would be (c ?? d) or (c else d) or (c otherwise d).

1 Like
232

Those who are apprehensive about the syntax, I’m wondering, is this discussion the first you’ve encountered these operators? If so, I assure you that readability is only an initial concern and you’ll get used to them quickly.

When I encountered the null coalescing syntax the first time in C#, especially ?., I remember feeling as though it was very much unreadable compared to every other operator. I think this is because these operators are just very programmer-programm-y operators that only make sense specifically for the ergonomics of a programming language, unlike other operators that have some context behind them outside of programming.

I think the non-keyword syntax is best. Dealing with nulls is a common operation. Remember why operators are useful: they allow us to use our visual processing to extract meaning quicker. If the syntax were the more verbose otherwise, then you’d always be forced into reading it as an ‘otherwise’ out of the context of nulls and with all its context associated with that word outside of programming, and it becomes a cognitive chore once you become familiar with it and don’t want it visually in the way.

Reading ?? and ?. eventually no longer becomes an exercise in thinking in ‘maybes’ and ‘otherwises’. It simply becomes a ‘coalesce’ operation, and your mind knows what that means.

Personally, I think the ? character looks very snake like, and would fit into Python okay.

8 Likes
233

Usually I write an if statement instead of an expression for this:

if x.y is None:
    x.y = default()

which is clearer IMO and only requires writing x.y twice.

3 Likes
234

We should also consider that a question mark for null-coalescing actions (?? and ?.) is used in C#, Javascript, Swift, Dart, PHP, Kotlin, Groovy, Nim and PowerShell. Two of these (Kotlin and Groovy) use the Elvis operator ?: instead of ??.

Deviating from this widespread tradition has its own readability costs that must be weighted against the arguably very short-term discomfort of people who have never used any language from this list. It’s a bit like if Python never had any bitwise operators, and a new proposal to add them suggested using unwieldy keywords like x bitwise and y, x bitwise or y, x leftshift y etc.

12 Likes
235

The tradtion of Python is to eschew cryptic symbols (e.g. &&, ||, !) and use words (and, or, not).

5 Likes
236

I mostly agree.

Estimates of Python usage run anywhere from 4 to 20 million of users, and it’s reasonable to assume the readbility put into the OG language is a large part of the appeal. We should keep that.

I find maybe, otherwise et. al. problematic because,if a.b.c.d is not None, but is False, 0, or an empty container, I don’t know what x should be after:

x = a.b.c.d otherwise None

Since we already have nonlocal, I’d suggest:

x = non_none a.b.c.d otherwise None

1 Like
237

But practicality beats purity. Python already has symbols that are cryptic to newcomers yet become excellent once you gain a bit of familiarity or have experience with other languages. At this point, I’d argue that ?. and ?? are the most obvious and natural syntax for null coalescing, given how widespread they’ve become.

8 Likes
238

There’s nothing obvious about ?? meaning null coalescing. It’s generally used to indicate confusion, as in this xkcd strip

confusion
confusion180×266 21.1 KB

2 Likes
239

Obvious to whom?

Somethings you just have to learn, somethngs you can use past experience to understand. Like the rules on when you need to add a “:” for some statements in python.

6 Likes
240

Some things work better as symbols (@, :=, ^) and some work better as keywords (is, not, and). To find the best spelling, we should contemplate both options.

I don’t think the suggested maybe keyword is as clear as ?. based on two examples:

x = a?.b.c.d.e
x = maybe a.b.c.d.e

y = a?.b?.c or d?.e
y = (maybe a.b.c) or (maybe d.e)

In the first case, not every attribute in the chain is nullable – only the first one. maybe is too coarse grained to express this.

In the second case, scoping matters. You need two maybes. I can easily imagine people mistakenly thinking that maybe applies to the whole expression.

By contrast, otherwise doesn’t have these problems because it is infix between two expressions. But if a keyword is going to be used for this I wish it were one which new users could more easily distinguish from else and or. Various ideas which come to mind, but I’m not a big fan of any of them. Maybe one will be inspiring for someone: x ifnone y, x none? y, x yet y, x alas y, some x or y, option x y.
Of these, I would choose either x otherwise y or some x or y.

Given that I’m in favor of the “maybe dot” operator (?.}, I think there’s a strong case that ?? is a better rhyme with that.

3 Likes
241

It still violates DRY. But there’s a subtler item at play for me. When reading code, my brain (as I assume most experienced coders’ brains do) tries to recognize “idioms” so that I don’t have to use the tiny Python interpreter in my brain to understand what a statement does.

In TypeScript (where I learned this idiom), when I see

x.y ??= BLAH BLAH

this is immediately summarized as “the default for x.y is BLAH BLAH”, as no other interpretation is possible.

OTOH when I see

if x.y is None:
    x.y = BLAH BLAH

I carefully need to read both lines, let it sink in that the test is indeed ‘is None’ and not e.g. ‘is not None’ (also a common test), confirm that the variable tested and the variable assigned to are the same, and that no other code exists in the indented block, before I can summarize the whole thing as “the default for x.y is BLAH BLAH”.

By the time my brain has processed all that it may have lost the overall flow of the code I’m trying to understand.

So no, I don’t think at all that if x is None: x = BLAH is an adequate substitute for x ??= BLAH.

9 Likes
242

Indeed. The bytecode generated for a.b is much shorter than for a?.b, since the latter must insert a None check and a branch. We don’t want to have to pay that price for every dot in a?.b.c.d.

3 Likes
243

This gets to my question (and concerns) above:

If you do really care about safe access for only some of the attributes in an “attribute chain”, I’d like to understand why, and whether that is more important than being able to understand the entire expression, rather than having to pick apart individual bits of it.

It feels overwhelming to me to have to try to comprehend (let alone visually notice!) something like a?.b.c?.d.e. I’d have so many questions about that in a code review, like, "how do you know that b.c will always resolve or that you want an AttributeError if it doesn’t? Why are you conditionalizing c.d? Why are some attributes guaranteed but others aren’t? Did you forget a ? somewhere?

[edit: And ha ha! Once again, it was so easy to write .? rather than ?. - and that is a warning sign to me too.]

3 Likes