PEP 661: Sentinel Values

[scoder (Stefan Behnel)]

@Jelle

That said, this is something we can revisit in the future, and if good use cases emerge we can enable subclassing in 3.16.

If subclassing is not something that is entirely out of the world, could we have a PySentinel_CheckExact() alias for PySentinel_Check() (or, rather, the other way round) already in Py3.15, so that users can make the distinction in their code already? Even if both do the same thing for the time being, I don’t see why we should let users pile up code on their side that turns out to be incorrect and needs to be changed if subclassing gets allowed at some point in the future.

[tmk (Thomas Kehrenberg)]

I had a minor typing questions while reading the new documentation and the PEP:

The PEP states

If the name passed to sentinel() does not match the name the object is assigned to, type checkers should emit an error.

but in the example in the documentation, we have this:

class Cls:
    PICKLABLE = sentinel("Cls.PICKLABLE")

assert pickle.loads(pickle.dumps(Cls.PICKLABLE)) is Cls.PICKLABLE

which seemingly breaks the above rule. However, it does seem nicer to write "Cls.PICKLABLE" here (and in fact, this seems to be needed for correct pickling), so maybe the typing spec can be formulated to allow this. (Or even require prefixing the class name if a sentinel is defined in a class?)

[tmk (Thomas Kehrenberg)]

Sorry, another thought I had:

Is it somehow possible to make sentinels work with match by, e.g., supporting something like the following?

MISSING = sentinel("MISSING")

def next_value(default: int | MISSING = MISSING):
  match default:
    case MISSING():  # note the parentheses here to avoid a catch-all match
      print("missing value")
    case val:  # this is now `int`
      ...

[HexDecimal (Kyle Benesch)]

"Cls.PICKLABLE" matches the name, the qualified name. This confusion is why I’ve insisted on the exact terminology.

[HexDecimal (Kyle Benesch)]

Match support is possible. I’ve had a working prototype of it earlier: PEP 661: Sentinel Values - #317 by HexDecimal (It works with isinstance which means it also works with match)

[Jelle (Jelle Zijlstra)]

I don’t feel too strongly about the class name issue, I think we can discuss this when we add sentinels to the typing spec (which I’m working on).

As for match support, this would need some runtime change to how the match statement works. (@HexDecimal’s implementation relies on making each sentinel object a new class, which is not the implementation we ended up with.) I think it’s an interesting idea but too late for Python 3.15.

[Jelle (Jelle Zijlstra)]

I posted PEP 661 (sentinels) in the typing spec to discuss incorporating sentinels into the typing spec, and a spec PR Add sentinels (PEP 661) to the spec by JelleZijlstra · Pull Request #2277 · python/typing · GitHub . Let’s move any further discussion of typing there.

[techdragon (Sam Bishop)]

I was out of the loop for a while and I just noticed this has been standardised… which is great to see but… I’ve got to be honest… I’m utterly horrified the decision was made to have sentinels evaluate ‘truthy’ in boolean comparisons.

I know its my opinion, and I know it changes nothing since it’s now accepted as written, but I have to say its the first time I’ve ever felt that a PEP got something flat out “wrong”… and I’m a bit sad thinking about how many codebases I work on in future will have a “ProperlyFunctioningSentinel” subclass which patches this “mistake” and has a doc string explaining that allowing sentinels to evaluate true reduces their effectiveness as a safety guard.

So to your points @StelarDream up here PEP 661: Sentinel Values - #340 by StelarDream and your reply @scoder PEP 661: Sentinel Values - #343 by scoder … Can we try and not ship this without support for subclassing since it wasn’t explictly restricted in the PEP? Because I’m going to be want to use code like this from the moment sentinels ship.

from typing import Never

class UniqueSentinel(sentinel):
    def __bool__(self) -> Never:
        raise NotImplementedError("This sentinel can not be safely coerced to a boolean value")

[taleinat]

Hi Sam,

It’s unfortunate to get your input on this so late in the process. This was discussed in detail over several years, and honestly I agonized over truthiness a lot.

It would be much, much more useful to hear your arguments on the matter, even after the fact, than just a statement that that is “flat out wrong”.

EDIT: Sam laid out his arguments on this matter in detail much earlier in this thread, I simply forgot

FWIW I’d consider adding the ability to set the truthiness, including to a value which would cause __bool__ to raise, much more reasonable than adding the ability to subclass.

[effigies (Chris Markiewicz)]

Their arguments were presented:

PEP 661: Sentinel Values

As someone who has actively used various PyPI available implementations of Sentinels value/objects for years now… I’ve got more thoughts as the conversation has gone on. Having a built in callable that didn’t need to be imported would be great. But other than that I’m kinda looking at this PEP as it stands and thinking its a step back over what I have now. If you combine the forced truthyness of the new draft and the lack of subclassing which has always been there… for about 50% of my use cas…

PEP 661: Sentinel Values

I think this has an extremely easy answer… Since Sentinels will be new to the standard library, give them a new standard library exception to raise… my personal suggestion would be raising something like ‘SentinelComparisonError’ which is either a subclass of ValueError, or a subclass of SentinelError which could potentially be a subclass of ValueError. Since existing code using sentinels will likely (in my experience) be using existing custom exceptions, and code adopting sentinels once they a…

PEP 661: Sentinel Values

This is the core of my argument for needing some ability to define a sentinel that errors on Boolean comparisons. There is a significant body of python code that functions by passing kwargs dicts, and while it’s easy enough for a developer to introspect the call chains and decide if None, object(), etc is best… the problem is that they aren’t in control of third party code, a point update could restructure some comparisons assuming nothing is different and introduce inadvertent changes to what…

The discussion has gone on for another 120 posts, so it’s easy to lose them.

[taleinat]

Wow, thanks! It’s been so long, and I’ve been away from this for a good while, I forgot

Those are certainly very well thought out and written arguments. I appreciate them a lot, and considered them deeply while working on this PEP.

The counter-arguments I mainly considered were:

1. A major focus was to be useful in the stdlib, and a careful review showed that all such sentinels in the stdlib were truthy.
2. It is extremely unusual for Python objects to raise an exception when evaluated in a boolean context. I was very worried about the many esoteric edge cases that would come up when this would be rolled out to many millions of users and broke things in myriad unexpected ways. So deciding whether to have __bool__ (possibly) raise an exception was an especially delicate decision.

In the end, in part to avoid endless “analysis paralysis”, I opted to keep this as simple as possible to begin with. In the future we can relatively easily add something like the ability to set truthiness, but that is the kind of thing we could practically never change in hindsight.

In the meantime, the existing 3rd-party sentinel libraries are fine! Having this in the stddlib doesn’t mean everyone has to switch to them. We can consider down the line if we want to make the stddlib sentinels more generally useful to address additional use cases.

[mwichmann (Mats Wichmann)]

Hmmm, Built-in Functions — Python 3.15.0a8 documentation says “Added in version 3.15.0a8” but my instance of that version still claims NameError: name ‘sentinel’ is not defined

[Jelle (Jelle Zijlstra)]

It will be in the first beta (to be released tomorrow). We might be reporting a slightly wrong version in the docs.

[hugovk (Hugo van Kemenade)]

@mwichmann

Hmmm, Built-in Functions — Python 3.15.0a8 documentation says “Added in version 3.15.0a8”

And note the magic final word:

Added in version 3.15.0a8 (unreleased).

[beauxq (Doug Hoskisson)]

@taleinat

A major focus was to be useful in the stdlib, and a careful review showed that all such sentinels in the stdlib were truthy.

Did you find any code that evaluated the __bool__ of these stdlib sentinels?
If they’re not evaluated, then it doesn’t matter that they’re truthy.

[jamestwebber (James Webber)]

I asked this earlier PEP 661: Sentinel Values - #301 by jamestwebber

And got the response:

@taleinat

I wasn’t looking for this, but I couldn’t see any use of these in boolean contexts during my review.

So the latest reasoning seems shaky to me.

[kapinga (Jon Harding)]

I think the most definitive reason to treat sentinels as truthy is that the SC explicitly said that was fine by them:

@barry

We think it’s fine for sentinels to evaluate as truthy in boolean contexts.

Rehashing that decision now seems to have little benefit to anyone.

[joshorr]

Going out on a bit of a limb here, but perhaps the discussion could be useful to the future version of python 3.16? Perhaps extra discussion could be informative to the future?

[tmk (Thomas Kehrenberg)]

Well, it would be bad to break backwards-compatibility. So if sentinels are truthy in 3.15, they should be truthy forever. (Though I’m not sure whether you’re maybe referring to some other discussion topic.)

[peterc (peter)]

The discussion is whether there should be a way to instantiate a ‘falsy’ Sentinal.

Personally I’ll be using

XXX = sentinal("XXX")
XXX.__bool__ = lambda: False

if that works (I haven’t been able to test the python 3.15 implementation), until any such time at which better support is added.