No. I’m saying I think adoption of 661 would address one of the objections to this proposal
That strikes me as a condescending. The community’s focus should be on helping users get the benefit out of Python, not preventing bad code.
Those of use of a certain age might remember Betrand Meyer’s: PRUDENT HIRING PRINCIPLE: Beware of C hackers.
It would be extended to the entire object chain.
if? a.b.c.d.e.f:
print(a.b.c.d.e.f)
Because I didn’t know it existed, pypi is uncurated, and trying to find what you want is stumbling around in the dark.
I would have used if I had known it existed this past Friday, when I wrote this monstrosity:
class PortOpen:
def __init__(self):
self.inputs = []
nft = nftables.Nftables()
nft.set_json_output(True)
rs = nft.cmd("list ruleset")
ds = json.loads(rs[1])
for item in ds['nftables']:
if (r := item.get('rule',None)) is not None:
if r.get('chain',None) == 'INPUT':
self.inputs.append(r)
...
def find(self,port:int,protocol:str)->bool:
"""See if existing accept rule exists"""
for r in self.inputs:
match = None
try:
for piece in r['expr']:
if (m := piece.get('match',None)) is not None:
if m['right'] == port:
payload = m['left']['payload']
if payload['protocol'] == protocol and payload['field'] == 'dport':
match = True
if match and self._is_accept(r):
return True
except KeyError as ke:
cc_logger.exception(f"key error {ke} for rule {r}")
return False
That looks like a good example to demonstrate how the code looks when written either with glom or with the non aware operators.
One thing I notice is that KeyError is being logged which would presumably not be possible in the same way if using ?[key] or whatever syntax since it silently becomes None without any trace of which key lookup failed.
If you are attempting to compile community opinions on this topic, I would like to register that “we aren’t likely to agree and therefore status quo wins” is an opinion in it’s own right and deserves consideration.
The community appears to be divided primarily into two camps, in favor and against. I’m not sure that much convincing is happening anymore – I think there are reasonable people making good points in both groups, but it feels like a lot of it is talking past one another.
I’m mildly against this change. I might have been in favor in the past, I can’t recall.
I write a lot of code which reads JSON data, and it’s rare that I would use this syntax if it were available. In cases where I really want it, glom and jmespath are more powerful options.
However, that’s independent from my opinion that such a controversial change shouldn’t move forward given the current stage of debate. I greatly appreciate your effort and initiative to push the discussion, as I think it’s best that someone finish and submit the PEP at this point – or that it be withdrawn.
Just noting: the floating point analogy for PEP 505 is “signalling NaN” (NaN-aware operations will pass them through, regular operations still throw an exception).
A “quiet NaN” equivalent would be having attribute lookup and indexing on None always return None. PEP 336 – Make None Callable | peps.python.org is the furthest any proposal along those lines ever got, and it was rightly rejected.
We are already living in the world where python doesn’t have None-aware operators and evidently that hasn’t stopped designers from making bad APIs but it has stopped the consumers from elegantly dealing with the same. So much so that this PEP was even written to deal with that problem. I think the benefit to consumers, who more often than not just want some data from some external source and do not need or want to write a full blown parser and validation library, far outweighs the possibility of enabling more bad APIs.
We’d add an else in that case:
i
f? a.b.c.d.e.f:
print(a.b.c.d.e.f)
else:
raise ValueError("ABC interface changed")
This thread also points out existing pypi libraries like glom but I think this (for most users who likely aren’t writing a full blown api wrapper) is simply not worth it.
Consider the walrus operator:
match = re.match(pattern, text)
if match:
print(f"Found match: {match.group()}")
if match := re.match(pattern, text):
print(f"Found match: {match.group()}")
The second one is certainly better but it’s not something I would add a dependency for. I do think most use cases for None-aware operators fall in this nice-to-have-but-not-worth-a-dependency category.
The point is that the original code logs the KeyError which will show which key was missing. The ? operators lose this information by discarding the exception.
I would still write the first of these because I like assignments to be at top-level as they needed to be before the Walrus operator was added. The motivation for the Walrus operator was not to save a line in cases like this where the assignment can just be on the line above.
Incidentally the Walrus operator PEP (572) says:
Another use of real code is to observe indirectly how much value programmers place on compactness. Guido van Rossum searched through a Dropbox code base and discovered some evidence that programmers value writing fewer lines over shorter lines.
Case in point: Guido found several examples where a programmer repeated a subexpression, slowing down the program, in order to save one line of code …
Another example illustrates that programmers sometimes do more work to save an extra level of indentation:
This is basically an observation that programmers will often use the facilities provided by the language to write fewer lines if possible even if it makes the code poorer in other ways like compromising on correctness, performance, readability etc.
Several of the examples in PEP 572 are precisely this.
Monarch is correct; the second one is certainly better.
Yes. That’s (one reason) why the walrus operator was added: to make the code not be poorer for being more compact.
The difference being here that the walrus actually accomplishes that goal, while this would not.
Lets compare
value = x["a"]["b"]
vs
value = x?["a"]?["b"]
In the case where you have the data you expect, these both extract a value
In the case where you do not have the data you expect, one of these tells you what data you are missing with an exception, the other turns it into None suppressing the context for why it is missing, at a cost of 2 extra characters.
In the “safe navigation” case, you need to always handle None after that, or forward that to users.
In the status quo, if you use exceptions, they are free in the expected path since 3.11
expanding that:
try:
value = x["a"]["b"]
except KeyError:
value = default
value = x?["a"]?["b"]
and we can’t anymore. Unless the default is None, because we no longer know if the value was missing or if the value was None. Which means this also only works for things that default to None or can’t contain None to begin with. But the people who want this aren’t validating that None isn’t a value or they wouldn’t need this.
The more terse code is worse here, much like recomputing the same value to save a line was. If the walrus was introduced to improve code on that observation, this should be rejected for that same observation.
So I think I can distill succinctly what python would need to have to satisfy both communities, which seems to just be better result / error handling.
A sizeable amount of people seem to agree on one of these two things, that either 1. Changing python to return None on KeyError is a big and undesirable change and 2. Adding a load of ? operators suppresses knowledge about what is missing at best and could become a code smell.
Therefore here is my final proposa:
Optional which can either be Result or Errorclass Result:
def __init__(self, value)
self.value =.value
class Error:
def __init__(self, value)
self.value =.value
class Optional(enum.Enum):
Result = Result
Error = Error
try?name = try? object.user.name
This will capture any exceptional and return an Optional, as if the code was like this:
try:
name = object.user.name
except Exception as e:
name = Result.Error(e)
if code = try? object.error.code:
logException(code.value)
match code:
case Result:
case Error:
This is a more succinct way of writing what we can already so with exception handling code which just has pass in the exception block. This can make the error handling more explicit like it is in swift, and nobody is suggesting we rip out exception handling in python even though it can be abused ?
API developers can adopt this new type to indicate a new error type which potentially could be extended to indicate what type of errors to expect rather than the unknown list of exceptions we have today
The error type would contain the details of the reason including the key index but if a person just tries to grab the value of the optional type without checking it’s an error or not. Then it will re-raise the error.
I think this allows us to keep the code succinct in way original author intended but in a way that if the user does something bad the language can force them to handle it. That’s something even the exception sytem today doesn’t do (You can call an API not realising it has the ability to fail)
This is using the rust pattern that is battle tested and you propose will map into python with the same benefits. I like this approach.
This may be a fruitful direction, given that result types are fairly well established in other languages.
However, I think that descending through multiple operators and only producing a single result type does not match the norm.[1]
If a.foo.bar is "baz", then “safe navigation” to a.foo.bar should be nested two times, i.e. Optional(Optional(Result("baz")))
That preserves the failures as distinct: Optional(Error(...)) vs Optional(Optional(Error(...))).
Failing to preserve that distinction is an issue in a lot of the proposed usages in this thread, and is related to why overly broad exception handlers are a code smell.
With that modification, I think it’s reasonable to consider a descending operator which stops at the first non-Optional value. e.g.,
maybe_val.get_value() # returns the bottom error or result
You can then pattern match on that and you’ve preserved the original chain of access which may have failed.
If ?. and ?[] produce Optionals, I could imagine this all working.
All of that said, I’m not going to act as a strong proponent for this design. I’m primarily noting that if x.y.y.y.y is an expression which fails, AttributeError: blah blah y doesn’t tell you which part failed.
I think the result type pattern is meant to wrap each layer of optionality, not just the “last one”. Unzipping the layers of that type once it’s in hand is okay though. (But I don’t think the type system can express it right now?)
I might be a bit out of touch – I haven’t used some of the languages I have in mind in many years. So please correct me if I’m mistaken. ↩︎
I’m largely following the pattern set out by swift regarding that but before I get too far into weeds
I think this discussion has three debates happening at same time.
I think if we introduced it in this order this discussion might get somewhere 
I would be happy to scope out 1 as a PEP if people agree.
I am emphasizing a point that I and others have made many times: in Python, we need to know exactly why an exception occurs.
Yes, we often see code like:
try:
...
except:
pass
…but we can fix that.
If ?. is used multiple times in a single line, it becomes a debugging nightmare. We cannot restrict users to only using ?. for optional data—they will use it simply because it’s trendy.
I see it becoming similar to the assert statement, which is almost always used in production—so much so that proposals to remove the -O and -OO optimization flags are frequently heard.
Or, removing indentation, so we could write code like this: def func(args): {print()}; def func2: {print()}, in a hieroglyphic style, etc.
At this point the thread is discussing things that are a long way from the safe navigation operator. Start a new thread to discuss exceptions-as-values or exception-catching-expressions.
Be aware though that there is already a rejected PEP for exception catching expressions (PEP 463). If you want to propose a nice syntax for except Exception then also be aware that most experienced Python programmers will have already spent some time explaining to others exactly why that should not be used in code. Anyone proposing convenient syntax to swallow exceptions like that should expect considerable resistance.
It is true that errors-as-values works nicely in Rust but there are also a bunch of other features in Rust that make it work the way that it does including its very different ? operator. Someone can make a library that defines Option/Result types (perhaps they already have) and people can try it out and see how well it works, but I think they will find that it is not so nice when the language is not fully designed around it. I think it is too late to retrofit that design into the core Python language but feel free to come up with a proposal and prove me wrong.
I disagree it’s getting away from original subject. In many other language all
of these concepts are part of their implementation of an optional system
Of which null operators are part.
People are saying they don’t want null operators because it makes it hard to debug errors. This is true in a language like typescript where optional only return the value or null
But other lanaguages show you can solve this by returning a result type which can be inspected to find out why it failed
So in order to introduce null operators we would effectively need to introduce result types to keep to community in agreement
However I do agree that a reformulated thread / PEP focusing on the one thing everyone agrees on and is also happens to be the prerequisite for the optional operators (null or otherwise)
I’m scoping out the result types proposal here
Let mw know what you think I missed or could explain better.