PEP 661: Sentinel Values

[MegaIng (Cornelius Krupp)]

No, that should be allowed, for e.g. wrapper functions.

[encukou (Petr Viktorin)]

The Reference Implementation in the PEP uses a registry that uses f'{module_name}-{name}' as keys. That would mean that all same-named sentinels in a module will be the same object. Additional Notes suggests that this is intended.

However, the implementation on GitHub is different – and the exact behaviour is, IMO, hard to explain. (Especially in cases where the module name can’t be guessed correctly. I’m still worried about relying on _getframe, or the frame stack in general, for correctness.)

I alluded to this above (see the full post), adding:

@encukou

This is API intended for library maintainers, who are creating their own APIs and want those to be pretty. IMO, it’s appropriate to lean towards more explicit (but verbose) API than usual.

Also, consider that Flask , a framework that tends to avoid boilerplate code, makes you specify the module name:

from flask import Flask
app = Flask(__name__)

… and I don’t think people find it overly complex to learn.

But I didn’t give a clear counterproposal. Here it is – a bit more typing for the user, but it removes the guesswork from the implementation (and so, makes the edge-case behaviour clear to the user):

• NotGiven = Sentinel('NotGiven', __name__) – the preferred form. The sentinel must be a module global (that is, available as import_module(__name__).NotGiven), which is checked at pickle/copy time.
• NotGiven = Sentinel('NotGiven') – OK, but raises an error when pickled. IOW, class/function-level sentinels aren’t pickleable.

IMO, stdlib code should be boring, and obviously correct, even if it means sacrificing features or ease of use. See dataclasses vs. attrs, or tomllib vs. tomlkit.

If you don’t agree, could you mention explicit module_name in the Rejected ideas section?

---

The sentinels package on PyPI (which would be broken by this PEP) had 421274 downloads last week. This suggests it’s actively used (AFAIK, unused packages get <100 weekly downloads).
I don’t think the package can be removed per the normal PEP-541 process. (That means that the sentinels maintainer would need to agree with giving up the package and breaking their users, or that this PEP would need to be approved by the PSF Packaging WG, not just the Steering Council. But I recommend choosing an unused name.)

[Monarch]

sentinellib is unclaimed on PyPI and follows an existing scheme with tomllib and pathlib.

[taleinat]

Hi Petr, thanks for the thoughtful response. I did read your previous post on this subject but failed to respond to it; allow me to correct that!

First, as you mentioned, it is indeed intended that same-named sentinels in the same module always be the same object. This is to ensure minimal surprises or edge cases with comparisons using is.

the implementation on GitHub is different

To clarify, that implementation is not entirely complete, and the PEP does not suggest using it necessarily.

I’m still worried about relying on _getframe, or the frame stack in general, for correctness.

I agree! I dislike using frame inspection for this and tried hard to avoid it. The precedent set by namedtuple and Enum is too strong to ignore, though, and they both use such frame inspection in their implementations. The pattern of allowing the module name to be provided explicitly, but intending that to handle edge-cases and normally using frame inspection, also intentionally mirrors them. They have both been in the stdlib for many years now, and the issues with their use of frame inspection seem to have been relatively very minor.

My takeaway from this is that using frame inspection for the implementation is currently okay. If in the future that were to change, the implementations of other, more prominent, stdlib modules would need to be changed in the same way. Arguably, using the same pattern introduces less overall complexity than using a different pattern would.

This is API intended for library maintainers, who are creating their own APIs and want those to be pretty. IMO, it’s appropriate to lean towards more explicit (but verbose) API than usual.

I think the same could be said regarding namedtuple and Enum?

But I didn’t give a clear counterproposal. Here it is

This is greatly appreciated!

NotGiven = Sentinel('NotGiven') – OK, but raises an error when pickled. IOW, class/function-level sentinels aren’t pickleable.

I think this would be a problem. As a user of a library or package, I would have no control over whether the library implementer passed in the module name or not. I wouldn’t want to be surprised by a sentinel value arbitrarily breaking my code in unexpected ways.

IMO, stdlib code should be boring, and obviously correct, even if it means sacrificing features or ease of use.

I must disagree: I think simplicity and ease of use for the many millions of Python users is more important than making the stdlib implementation simple and obviously correct. To put this a different way, it is more important IMO that the code using Python and its stdlib can be as “boring” and obviously correct as possible. I think your suggestion would make using such sentinels less straightforward, since they could not be trusted to always survive copying or pickling+unpickling.

---

I did consider making providing the module name required. However, as mentioned above, the precedent here is strong, and experience shows that using frame inspection is not a significant problem in practice.

---

If you don’t agree, could you mention explicit module_name in the Rejected ideas section?

Yes, I can add that.

[pitrou (Antoine Pitrou)]

I am also surprised by the complexity of the proposed API. I would expect Sentinel(name, module_name). I don’t know why bool_value is useful since the PEP states that sentinels should be detected through identity comparison. I’m also not sure what a custom repr achieves.

By the way, I also agree that proper picklability should ideally be guaranteed.

[taleinat]

Hi Antoine!

@pitrou

I am also surprised by the complexity of the proposed API. I would expect Sentinel(name, module_name). I don’t know why bool_value is useful since the PEP states that sentinels should be detected through identity comparison. I’m also not sure what a custom repr achieves.

Hmm, interesting to think if this could be made even simpler!

Regarding repr, various exiting implementations have different types of reprs for sentinel objects, so my thinking was to allow them to switch to use the new Sentinel without the repr changing. But that is a relatively minor consideration, and it may indeed be better to have Sentinel use a single style of repr. This would be simpler and also more consistent across code bases. We’d need to bikeshed the chosen repr style though .

Regarding boolean value, in discussions people brought up uses for being either True or False in different cases. And note that in “should be detected through identity comparison”, “should” is not “must”. But, again, it is possible to make this simpler and more consistent by always being True, or even change “should” to “must” and raise an exception in boolean contexts to avoid unintended bugs. However, doing so would likely reduce the number of places where we could use Sentinel for existing sentinel values in the stdlib, especially in the latter case.

I’ll let these thoughts sink in and run around in my head a bit.

[efimov-mikhail (Mikhail Efimov)]

@taleinat

I must disagree: I think simplicity and ease of use for the many millions of Python users is more important than making the stdlib implementation simple and obviously correct. To put this a different way, it is more important IMO that the code using Python and its stdlib can be as “boring” and obviously correct as possible.

I’m on your side. End users’ code should be as simple as possible, not stdlib itself.
IMO, NotGiven = Sentinel('NotGiven') is much better than NotGiven = Sentinel('NotGiven', __main__).

[XuehaiPan]

@encukou

The sentinels package on PyPI (which would be broken by this PEP) had 421274 downloads last week . This suggests it’s actively used (AFAIK, unused packages get <100 weekly downloads).

Maybe we can use typing and port it to typing_extensions.

[taleinat]

@encukou

The sentinels package on PyPI (which would be broken by this PEP) had 421274 downloads last week . This suggests it’s actively used (AFAIK, unused packages get <100 weekly downloads).
I don’t think the package can be removed per the normal PEP-541 process. (That means that the sentinels maintainer would need to agree with giving up the package and breaking their users, or that this PEP would need to be approved by the PSF Packaging WG, not just the Steering Council. But I recommend choosing an unused name.)

Yes, you’re right that it is still in use and would be broken if we added a “sentinels” module, and we should probably avoid that.

[taleinat]

@encukou

This suggests it’s actively used (AFAIK, unused packages get <100 weekly downloads).

Certainly. For compraison, the “sentinel” package had <13k downloads last week, and the “sent” package had only 343.

[sirosen (Stephen Rosen)]

As someone who would use this module if it were added, I’m in favor of always requiring __name__.

First, this is a nice and useful hint to users that the module name matters. If I pickle, move the definition to a new module, then unpickle, I won’t be surprised by any discrepancies.

Second, we can look at the number of times new users are confused by logging.geLlogger() as an indicator that an optional name will lead to questions about how or why it’s different to supply or not supply the name.

[Jelle (Jelle Zijlstra)]

Because this thread is so long and has lasted for several years, it’s a bit hard to follow all the discussion. However, I’m glad it’s finally moving towards a resolution now.

I’d like to reiterate my post from last year: PEP 661: Sentinel Values - #114 by Jelle. I think it’d be better to allow people to write sentinels in types as simply the sentinel itself:

MISSING = Sentinel("MISSING")

def foo(value: int | MISSING = MISSING) -> int:
    ...

Instead, the PEP currently requires that users write Literal[MISSING] in type annotations. Omitting Literal shortens the type annotation and removes the need for importing Literal. It shouldn’t be significantly more difficult to support for type checkers, as type checkers already would need to track definitions of sentinels in order to support the Literal[MISSING] syntax. It also removes an inconsistency: Literal["MISSING"] means the literal string "MISSING", not a forward reference to the MISSING sentinel.

As the PEP proposes an extension to the type system, we’re also discussing it within the Typing Council. However, what I’m writing here is only my personal opinion. I’d be interested to read other opinions on the typing aspects of the PEP.

[MegaIng (Cornelius Krupp)]

@Jelle

I think it’d be better to allow people to write sentinels in types as simply the sentinel itself:

This feels like something that could be added afterwards by a Typing Council only decision with no need for a larger PEP, and the SC does not need to make a decision on it (unless the absent or presence of this syntax sugar would change their decision on this PEP in general). Especially since Literal[MISSING, None] should be supported anyway to keep symmetry with None.

[taleinat]

Hi Jelle!

In case this thread isn’t long enough, there was additional typing-specific discussion related to this PEP on typing-sig back near when this was originally proposed, as well.

Typing is something I’m not an expert on, and was originally the thing I got stuck on for a long time. I’m very happy to have a subject expert chime in!

I’d like to reiterate my post from last year: PEP 661: Sentinel Values - #114 by Jelle. I think it’d be better to allow people to write sentinels in types as simply the sentinel itself:

Indeed, this was the other common suggestion, and I deliberated between this and Literal[MISSING].

It also removes an inconsistency: Literal["MISSING"] means the literal string "MISSING", not a forward reference to the MISSING sentinel.

That is an excellent point that was not brought up until now. Along with the precedent set by using None in type signature, I think this is convincing enough to switch my suggestion to use the sentinel itself in type signatures as you suggest.

I wonder if it would be okay to change the draft PEP now that it has been submitted for review…

[taleinat]

@Monarch

sentinellib is unclaimed on PyPI and follows an existing scheme with tomllib and pathlib.

Thank you for the excellent suggestion.

I think “sentinelslib” is perhaps more appropriate? As an added bonus it avoids the potentially-confusing double “L” in “sentinellib”.

I’ve gone ahead and registered both “sentinellib” and “sentinelslib” on PyPI.

[Jelle (Jelle Zijlstra)]

@XuehaiPan

Maybe we can use typing and port it to typing_extensions.

I wouldn’t want to put the Sentinel class in typing, because it is useful even if you don’t use type annotations at all, and the motivation for the PEP isn’t focused on typing. However, if the PEP is accepted it would be useful to have a backport available for Python 3.13 and earlier, and I’d be open to adding Sentinel to typing_extensions in that case.

@taleinat

I wonder if it would be okay to change the draft PEP now that it has been submitted for review…

I can’t speak for the Steering Council but I know it’s not great for them if they’ve already started deliberating and the PEP changes. However, they also obviously want the PEP to be as good as it can be. Personally, I’d recommend you post on the Steering Council issue that you’re considering this change, prepare a PR to the peps repo with the concrete change, and then wait for the Typing Council opinion to come in before merging it.

[Tinche (Tin Tvrtković)]

@Jelle

I think it’d be better to allow people to write sentinels in types as simply the sentinel itself:

This was my thought exactly when I was going thru the PEP. There’s obvious precedent in None, and (imo) the readability and usability gains absolutely justify a little type expression/value expression cross-pollination.

[steve.dower]

@taleinat

I’m still worried about relying on _getframe, or the frame stack in general, for correctness.

I agree! I dislike using frame inspection for this and tried hard to avoid it. The precedent set by namedtuple and Enum is too strong to ignore, though, and they both use such frame inspection in their implementations. The pattern of allowing the module name to be provided explicitly, but intending that to handle edge-cases and normally using frame inspection, also intentionally mirrors them. They have both been in the stdlib for many years now, and the issues with their use of frame inspection seem to have been relatively very minor.

They should both be using sys._getframemodulename now,^[1] at least for implementations that provide it (which I suppose to be easier that providing an entire introspectable and modifiable frame stack). I’d hope that an implementation here that is doing the same thing also uses that function by preference.

And of course, it ought to be overridable as a parameter (and replace an unspecified value with the function call asap, so you know how many frames away it is). But it sounds like you already do that.

---

1. I forget exactly how we ended up there, but my initial name was get_calling_module_name(), which I think better implies the purpose and just how light it is meant to be. ↩︎

[Monarch]

I prefer the singular form, but I don’t feel too strongly about it so I’m fine with either.

[taleinat]

@steve.dower

They should both be using sys._getframemodulename now

Thanks, I was not aware of that!

(BTW, I don’t see that mentioned in any “What’s New” (I’d expect it here), nor do I see an “Added in version” comment in the docs for the sys module, which is surprising.)