Okay so I have come up with a draft of a grammar (minus actions) that satisfies this intuition.
Basically for primaries without ‘?’, it’s the same as before, satisfying backwards AST compatibility (per the ast module).
But when ‘?’ is found, it gobbles up the “tail” – that is, all subsequent ‘.NAME’, ‘[…]’, ‘(…)’ and ‘?..’ operations. If a ‘?’ operation is found, it gobbles up the following tail (that’s just how the PEG parser works).
The grammar has ambiguities, which are handled by the PEG rule of trying alternatives in order until one passes.
The rest of the grammar only uses await_primary so this is the only part of the grammar we need to double-check (and write actions for).
In general, for a series of operations of equal precedence, A op0 B op1 C op2 D where op0, op1 and op2 have the same precedence we evaluate left to right, so A op0 B op1 C op2 D is the same as ((A op0 B) op1 C) op2 D.
Parentheses are used to change precedence within expressions, not the meaning of the individual operators. E.g. in (A + B) * (C + D), the meaning of + and * remain the same, but the order of evaluation changes.
This is true of both and and or as well as other operators.
Therefore (a.?b).c should have the same semantics as a.?b.c
To be honest, I find the proposed short circuiting semantics a bit weird. It is hard to parse, for both human and machine, and seems to offer no real advantage over the much simpler semantics of a.?b == (None if a is None else a.b).
Ah, but there is an important advantage. This has already been established in this thread and there is mostly agreement that the shortcut semantics are important. (In fact, the last 50 or so messages were triggered by the observation that in Marc Mueller’s prototype implementation, in e.g. (a?.b).c, the shortcut semantics would escape from the parentheses, which several folks here pointed out is a bridge too far.)
Here’s an example – made up, but closely matches a pattern found many times in TypeScript code bases, and it translates to Python (I translated that TypeScript codebase to Python and these were a major pessimization in readability).
Suppose I have a complex Configuration object that has sub-objects like “chatconfig: ChatConfiguration, embedding_config: EmbeddingConfiguration,” etc. The latter cannot be None and have fields like max_hits, min_score etc.
Now I have a function that takes an optional Configuration object.
Without shortcut semantics, we would have to write config?.embedding_config?.max_hits even though if config is not None we are ensured that config.embedding_config.max_hits exists. I have seen longer chains too, and it would be tedious to have to add all the extra question marks to each following dot.
To be blunt, examples like this one just convince me more that we shouldn’t have this, or at most should only have ?? and ?=, or have late-bound function defaults
It was slightly more persuasive when people were discussing cases where APIs they don’t control give deeply nested optional data[1], but you’re already showing that you’d use it in places where this isn’t the case.
There’s a simpler option that works today, and it has the added benefit of not having to update in-lined defaults in every line where you’re doing this if the defaults ever change: place defaults on the config class.
def nearest_neigbors(input: str, config: Configuration | None):
config = config or Configuration.defaults
# I am assuming `__bool__` doesn't possibly return false on a config object
max_hits = config.embedding_config.max_hits
return _find_nearest_neighbors(input, max_hits)
I don’t find this persuasive either, we have robust validation libraries that only require expressing the part of the data you care about, handle validation more robustly than any naive handling, and you already have to have some expectations about the data for this proposal to work. ↩︎
I think one of us is missing something, because reading this tells me that __bool__ should be called twice for (a and b) and c
The and operator calls __bool__
The first and doesn’t return the result of a.__bool__() - it just returns a
Once we finish evaluating what’s in the parentheses, we have an a b was never evaluated - that’s the short circuit so far.
Then we still have what’s outside the parentheses to deal with (equivalent to a and c)
Shouldn’t the 2nd and call __bool__ on the a that was returned by the first and?
That assumes a definitive default even exists. Because Guido’s code would suggest that 25 isn’t used elsewhere. It’s also plausible that the default value isn’t the same everywhere. And what if Configuration was part of a library? It’s not feasible to write out a whole default value object in that case.
To me it doesn’t suggest that 25 isn’t used elsewhere, it suggests the same kind of hard to maintain code I’ve seen people abuse this syntax for with other languages when better options are available.
If Configuration is part of someone else’s library, why are you using it for your library’s behavior? If the beahvior isn’t consistent across the library, why would you accept a library wide configuration object rather than just accepting the one value that function uses from it?
No matter the case here, something can be done better than the version that uses ?.
As we’ve found, the short-circuiting semantics of the existing logical operators are already a bit weird, e.g. in some cases parentheses don’t change the order of evaluation even though they look like they should.
We’ve already concluded that the Python docs are wrong about this – chains of s are actually treated as right-associative (regardless of how they’re parenthesised!).
Maybe is it better if the (interesting) discussion about boolean short-circuit discrepancies between Py version will be moved to another thread by a mod?
### OT - end ###
Returning to the PEP, I think the fact a?.b.c will short-circuit is a great addition. It’s elegant, it makes sense and it simplifies the code a lot.
EDIT: removing considerations that are out-of-date because I jump reading relevant posts…
But about (a?.b.c).d, I totally agree with people that says the parenthesis should stop the short circuit. It’s not the same as (a and b and c) and d. Here, the parenthesis can be safely removed, because there’s no way that the expression can evaluate to True if a is False.
On the other hand, to my eyes, (a?.b.c).d means “if a is None, a.b.c will be None, so trying to access the attribute d will raise”. It seems to me quite evident and, furthermore, desirable. On the contrary, there’s no way to stop the short circuit. Yes, you can write (a?.b.c ?? None).d. IMHO counterintuitive and – excuse me but it’s my POV – ugly.
What I can’t understand it’s why @cdce8p is so contrary about that. If you see the likes in the thread posts, it’s clear the people prefers that brackets will stop the short circuit. I really like the short circuit idea, but in this way we are stuck again.
It seems to me that my favorite language will have the ? operator only when I will be in the graveyard
By ‘suggest’ I meant as a means of intent. If I were to write a default value in a function like that, I’m intending to imply the fact that a one-off localised value is being used here and has no significance, at all, elsewhere.
I admit that my argument of “what if Configuration was part of a library?” doesn’t make sense in the context of Guido’s code as I had conflated taking an optional attribute off a configuration object (because that’s a common use case for ?.) with the configuration itself being optional.
What I found strange with your proposed improvement of the code though was that it assumes that creating an instance of the configuration was a possibility. The configuration type being part of someone else’s library happens all the time with frameworks and they’re always huge, so surely you can’t just instantiate one somewhere yourself like it’s nothing?
?. messes with an valuable quality of Python: That you can break down expressions into subexpressions and evaluate them one at a time. E.g. when you have code like
e = a.b(c).d
you can break it up and inspect intermediates like this:
t1 = a.b
print(t1)
t2 = t1(c)
print(t2)
e = t2.d
with no change in semantics other than the extra prints.
But
d = a?.b.c
can’t be decomposed in the same way. This will not work if a is None:
t1 = a?.b
print(t1)
d = t1.c
I see the usefulness of this proposed operator, but it would not be doing the readability and beginner-friendliness of the language any favors.
