Revisiting PEP 505

Since I’ve seen this several times (sorry for picking on one person), I want to point us back to a relevant objection to thinking about these as catching errors:

1 Like

One thing I haven’t seen referenced is what the expected semantics for list index access would be with these operators if they handle more than None, since the syntax seems to apply equally well there. Would it coalesce over IndexError?

Built-in Functions — Python 3.13.1 documentation

According to the documentation, it’s more of equivalent for the following construction:

if hasattr(foo, "bar") and (value := getattr(foo, "bar", None)) is not None:
    bar = value
else:
    bar = None

Because getattr is not a safe operation, even with default, comparing with dict.get(…). By the way, having missing argument aware getattr is useful as well as null-coalescing access.

And my concern in fact was not about missing attrubute of b in the chain a.b.?c. I don’t mind if .?c don’t raise AttributeError after general discussion, but I feel like a.b have to raise it, because this call is not null-aware

Does that actually change anything though? foo?.ttypo still goes from an obvious attribute error to very sneaking logic error.

7 Likes

@ryanhiebert Well, we can describe it in hundred of manners, but the fact is that, without the ?, an exception will be raised.

Please note I’m not completely against silencing KeyError and IndexError. I’m -epsilon. About AttributeError, I’m -googol :smiley:

1 Like

I understand the value in returning None for missing attributes, which is often a dealbreaker when using dataclasses with complex schemas. However, the issue arises when the intent is to handle optional attributes rather than missing ones, which is the typical use case for dataclasses today. Does this lead us to the idea of adding omittable class fields?

2 Likes

Sorry about the small bikeshedding, I’d like to invoke “beautiful is better than ugly”/“readability matters" and suggest the alternate syntax a? or b instead of a ?? b.

It’s closer to the a or b idiom (and to a.unwrap_or(b) from Rust).

A code full of ?? looks like a code full of confusion :grin:

It may technically work if you treat the two-token combo ? or as one operator, but I personally don’t want it to happen because I’m still in favor of Guido’s idea of a postfix generalization, which will directly conflict with your idea.

It’ll also make the augmented assignment operator version awkward (a? or= b vs. a ??= b).

1 Like

I agree with this point.
If the ?. operator suppresses exceptions, we can’t easily extract variables from traversals.

If I have a["k"]?.f, I cannot extract a["k"] as a variable as it will lose the exception suppression.
I can use try-except, but that’ll be very verbose.
My automatic response is to reach for a["k"] ?? None in this case for the extracted variable, but that would require ?? to suppress exceptions as well.

In any case, it will make the logic of extracting a variable far more complicated than it is today.
In JS it works because undefined is a value. In Python we can’t represent it.

This really comes across as dismissive, flagrantly disregarding the plethora of examples and use cases for it that have been shown in this thread.
In Guido’s own post, e.g.

For what it’s worth, I really think that this thread is dramatically overestimating the benefits of symbols like ? to improve readability.

I implore the community to carefully consider whether we really want a cryptic language littered with symbols that have special meaning.

If this change comes, I for one know that I will see the pattern
foo?.bar? ?? default() quite a bit in code. And it wouldn’t be uncommon to see this clause repeated a a few times, so foo?.bar? ?? baz?.bar? ?? default().

So, for this, I really truly agree with the concerns Steve raises. We should think about what we want to be encouraging.
If this ends up encouraging the passing of None, and ? littered everywhere, then that is undesirable.
In particular, this can easily result in many redundant checks, as it is tempting to have every function that operates on the data to have its own check.
While this may very well already happen, making the check more ergonomic with a simple ? can make users reach for it even when it isn’t appropriate.

To give an example, I frequently see this issue with type annotations (and match). Imo, it is preferable to use something like @functools.singledispatch to nearly split logic for handling different types. But with the notation of a: int|str and match being so easy to reach for, this ends up being overshadowed.

Not sure what keyboard you use, but ? requires me to hold shift as well.

IMHO, this isn’t a compelling argument. Python has recently added lots of syntax bloat just to support static type checkers (square brackets for generic classes being the one I find the most egregious).

It very much is. See above. Also look at all of the recent syntax changes to support static typing. Python literally introduced a new statement and keyword that really only benefits static type checking.

In what way is it not a safe option with a default?

2 Likes

There’s no need for the last ? with the original proposal. You can write

foo?.bar ?? default()

and

foo?.bar ?? baz?.bar ?? default()

Is it less readable? Yes, but also list comprehensions are less readable than a good old for loop. I worked with a Java guy that disliked list comprehension a lot because he considered it an obscure feature. Nevertheless it’s quite handy.

I don’t think a sysop will run mypy or pyright to check the typing. I don’t think they will have mypy or pyright installed in many cases. Furthermore typing is optional in Python, and I’ve seen much code that doesn’t use it.

2 Likes

Aren’t sysops scripts relatively straightforward, and read more like a series of shell commands?

What I mean to say is that the use case of sysops has no bearing on the syntax support here. As an example, the operator @ is specifically designed for matrix multiply, which accounts for a tiny fraction of scripts, and certainly isn’t likely to be seen in the sysops domain.

Also, the use of this operator would be optional as well.

While I do think it is important to consider the different domains that use python, I’m not sure that sysops matters here.

I may have misunderstood Guido’s postfix proposal. I had thought that to swallow the key/attribute error, you needed to end it with a ?, that is, using it as a postfix operator.

If that’s not the case, I’m not sure what a.b.c? means in Guido’s example.

This usage described by @guido looks very much common in “modern” web development, or scripting against services that expose an API (e.g. software integrations/automations).

It has been natural for the applications I’ve worked on themselves to develop a domain model that is more complex. Objects tend to have optional fields to handle all sorts of cases or usages, and they tend to layer into aggregates. You end up with deeply nested structures with optionality at any possible level. Because it represents the complexity of the domain.

That may not be the case in scientific computing, training a machine learning model, or writing a micro service to get the result of an algorithm. But working with web applications, sure is.

Given that it is in the model, it is not about parsing/validating. Not all fields will be present. This is how many major APIs models work, and we’re not going to change that. So it’s beyond whether to use pydantic or not.

I know no package that provide None-awareness like proposed and static type validation at the same time. If b is not an attribute of A, then I should get a static type error for A()?.b. If such a package exists, I could use it.

I could “just use typescript”. Unfortunately, some thing do work better with Python (notably ML), so I am stuck with it, and have to suck it up. For now?

5 Likes

Presumably that’s because the ? syntax doesn’t exist yet?

More to the point (as it was a source of confusion before), you would get a type error for A.b? from a proper static type checker. (It’s how TypeScript works too.)

1 Like

Sorry for the lack of precision. What I meant is:

I know of no package that propose safe none-aware chaining like some have proposed, that would as well provide type checking on missing attribute. For instance, the earlier proposed traversal(x, "y", "z").

And I don’t see any type checker implementing an edge case like this if it’s not a first-class citizen in the language (unlike ?., which would stir the ecosystem in its direction).

Right, I wrote too fast. A().b?.c would then be the more accurate example.

3 Likes

No tools exist to do recursive static type checking like you are showing, correct?

This discussion and a lot Guido’s example makes me think of pydantic. Pydantic to my understanding is a library that you define models which represent external facing web services. You get the json response and it will parse and validate it.

So it seems there’s already 3rd party libraries that do handle these sorts of things. You can define a pydantic model who has an optional field, during parsing if the field isn’t present then the field will be populated with None.

I’m not sure if the existence of pydantic would be in favor or against this sort of syntax for the examples that Guido gives, which are parsing json response. Clearly there’s similar enough use cases in python as js for parsing json responses but simultaneously there’s 3rd libraries that can easily handle these things in addition to a lot more.

1 Like

I made the point that pydantic does not help.

The challenge is not safe parsing, it is the verbosity and safety of navigating layered aggregates with optional fields.

I am not a core developer nor am I a professional language designer. I am voicing that this syntax would very much help simplify code in modern web development. I’ll leave my contribution to the discussion to that.

7 Likes

There’s a point in the Guido’s proposal I don’t catch.

People that promotes Guido’s proposal says that, if there’s a missing attribute, it’s caught by the static type checker (eg your IDE). OK, let’s pretend that anyone uses an IDE and the code is completely typed, even third party code. If you type only valid attributes because of the static type checker, when an AttributeError is raised?

1 Like

Final statement from me on these as there’s 3 or 4 threads on discussion now.

If you parse and validate with a library such as pydantic then it removes the need the safe navigation operators. The entire input gets sanitized at once and missing fields are filled with None if that’s what the models define. This handles the json response problems Guido provides.

I have a feeling this was mentioned somewhere already and I’ve missed it.

1 Like

Also with pydantic you can define a field as Optional, so the None-aware operators are useful anyway.

1 Like