PEP 661: Sentinel Values

PEPs
183

PS Another option which, I seriously believe, is worth considering as well is a further simplification: to get rid of that global registry at all. Then it would be up to the programmer to avoid duplicate sentinel names, as it already is, e.g., for Enums and their members (and, in a sense, just for any other type) – applying the “consenting adults” principle.

(In that case, I would no longer see any problem with the optional reliance on the not-necessarily-100-percent-portable-or-reliable mechanism for determining the module name; which then, indeed, would be comparable to the cases of namedtuple/Enum).

9 Likes
184

Re: the open issue about module location, I do think it seems a bit odd to add a new one to the std lib for the sake of holding a single public name. (Is there any precedent for that?)
What about the types module? It’s been used for a few other “random” classes and class-creation things.

4 Likes
185

I am having second thoughts about this.

This proposal offers sentinels that are different in behaviour to None and other standard Python sentinels.

None is Singleton while this proposal offers instances of the same class.

It would be nice to keep things in line to make life easier:

type(None)              # NoneType
type(None)() is None    # True

sentinel · PyPI also obeys the above. The only part that they haven’t figured out is pickle.dumps(Sentinel.__class__).
But it is doable without changes to pickle by copyreg.pickle(SentinelMeta, ...).

While this proposal does not work like this:

Something = Sentinel('Something')
type(Something)    # Sentinel

I would suggest an alternative to introduce singleton.py module which implements singleton type sentinels, such as None and PyPI.sentinel. Then later on _singletonmodule.c could be introduced and common part (Singleton class) could be moved to C leaving its specialisation in singleton.py.

This way there would be a common standard and this would align with Singletonobject.c and unification of `singletons` and `singlenels`

This would be slightly more complex than what is proposed in this PEP, however I think the cost is worth it to keep things consistent. Especially when implementation is 100-200 lines long.

1 Like
186

Given the consensus seems to be “sentinels in the stdlib is a good idea” but different folks have different ideas about how to to this best, it seems to me a solution would be a sentinel module with multiple sentinel types.

Personally, I think sentinels which can only be tested using is/is not are best, while others want them validate predicates of if statements. So sometime like this:


class Sentinel:
    """"Sentinel marker that only supports 'is' testing"""

    def __eq__(self, other):
        raise ValueError("Use 'is' in lieu of ==")

    def __bool__(self):
        raise ValueError("Boolean not supported")


class DefaultTrue:
    """"Sentinel marker that can be used as a boolean"""

    def __eq__(self, other):
        return self is other

    def __bool__(self):
        return True


class DefaultFalse:
    """"Sentinel marker that can be used as a boolean"""

    def __eq__(self, other):
        return self is other

    def __bool__(self):
        return False

I’m not including the repr code because folks smarter than me would be implementing this, presumably.

187

Since users may want custom methods on their sentinal values, I would like to propose a @singleton decorator similar to @dataclasses.dataclass.

def singleton[T](cls: type[T] | str) -> T:
    if isinstance(cls, str):
        # Create a simple class with the given name
        # This can be used as `MY_SINGLETON = singleton("MY_SINGLETON")`
        cls = type(cls, (object,), {})
    elif not isinstance(cls, type):
        raise TypeError("singleton decorator expects a class or a string")

    # Some handling to convert the class to a singleton class,
    # i.e. `cls() is cls()`.
    ...

    # Disable subclassing
    def __init_subclass__(*args, **kwargs) -> NoReturn:
        return TypeError("singleton class does not support subclassing")

    cls.__init_subclass__ = __init_subclass__

    if '__repr__' not in cls.__dict__:
        def __repr__(self):
            return f'<Singleton: {cls.__name__!r}>'

        cls.__repr__ = __repr__

    # Create the singleton instance
    instance = cls._instance = cls()

    # Register pickle for singleton class and singleton instance
    ...

    # Register copy support
    ...

    # The decorator returns the singleton instance
    # instead of the wrapped class.
    # See the example usage below.
    return instance

An example snippet:

from functools import singleton


@singleton  # this decorator outputs the instance rather than the wrapped class
class MISSING:  # class name is the singleton variable name
    def __bool__(self):
        return False

    def __repr__(self):
        return 'My Custom Repr'

    ...


# The singleton value can be used like:
def reduce(function, iterable, initial=MISSING, /):
    it = iter(iterable)

    if initial is MISSING:
        try:
            value = next(it) 
        except StopIteration:
            raise TypeError("reduce() of empty iterable with no initial value") from None
    else:
        value = initial

    for element in it:
        value = function(value, element)

    return value
1 Like
188

I think the behavior in the PEP is enough.

Side note: I think that, if I have to import yet another module to use sentinels, I’ll continue to use object instead.

2 Likes
189

This is my proposed path regarding this:

Full implementation is ~200 lines long, but this sums it up.

# sentinel.py

class Singleton:
    """Base singleton class, which later can be replaced by `C` implementation and standardise things for `C` level sentinels as well"""

class SentinelMeta(type):
    def __new__(mcs, type_name, bases, ...):
        bases += (Singleton,)
        return type.__new__(mcs, type_name, bases, namespace, **kwds)

# API ----

class SentinelGetAttrMeta(type):
    def __getattr__(cls, name):
        ...
        return SentinelMeta(f'{name}Type', ...)()

class Sentinel(metaclass=SentinelGetAttrMeta):
    _registry = coll.defaultdict(dict)
    def __new__(cls, name, bases=None, /, **attrs):
        ...
        return SentinelMeta(f'{name}Type', bases or (), attrs)()

Usage:

# mymodule.py

from pickle import dumps, loads
from singleton import Sentinel

# With defaults
Null = Sentinel.Null
type(Null)           # mymodule.NullType
type(Null)() is Null # True (same as None)
bool(Null)           # True

# Serialises both sentinels and types
loads(dumps(Null)) is Null             # True
loads(dumps(type(Null))) is type(Null) # True

Null = Sentinel.Null
# TypeError mymodule.NullType in _sentinel_type_registry


# With custom methods and attributes
# Some shorthands can be provided for method definition
NULL = Sentinel('NULL', bases=(int,), __repr__='MyNULLSentinel', __bool__=False, __contains__=lambda obj: False)
repr(NULL)    # 'MyNULLSentinel'
bool(NULL)    # False
1 in NULL     # False
type(NULL).__mro__    # (mymodule.NullType, int, singleton.Singleton, object)

# registry per module
Sentinel._registry
defaultdict(dict,
            {'mymodule': {
              'NullType': mymodule.NullType,
              'NULLType': mymodule.NULLType
              }})

For anything more complex, one can just use Singleton directly:

class MySentinelType(Singleton):
    def __str__(self):
        return type(self).__name__ + str(bool(self))
    ...

MySentinel = MySentinelType()

This would be fully in line with existing sentinels and support full customisation.
Also, C stuff of Singletonobject.c and unification of `singletons` and `singlenels` would nicely fit into this.

If new implementation is provided I think it would only be worth it if it checked all the boxes and would be in line with existing standards. In other words, it would be a final stop. Otherwise if it doesn’t provide all features and its behaviour is divergent from existing standards people will just use object() and not bother. Or stop using it after needing to look for alternative solutions after some new attribute for sentinel is needed or find out that it can not do things like isinstance(obj, (int, type(NULL)). Implementation is fairly small, thus it is not difficult to get it all right.

190

@taleinat Thank you for this PEP, and the good news is that the 2025 SC has started discussing it. We haven’t gotten too far, but I wanted to give you my thoughts, mostly as a Python user and developer. My mind as an SC member is still open, so please do not take these suggestions as anything official or as a sign of which way the discussions will go.

I’ve certainly written my fair share of sentinels, mostly of the missing = object() variety, which generally does what I need it to do, in very localized use cases. I’d say primarily in “get” APIs where None is a possible legal value, e.g.:

missing = object()
if dict.get('key', missing) is missing:
    # The key really isn't in the dictionary.

I usually only need it right there and then, but if it’s convenient to use in multiple places in a module, I might define it at module scope.

Reading the PEP, I find the complaint about object()'s repr resonates with me. I can’t remember the last time I needed its repr, but I can see it wouldn’t hurt to have something more recognizable. Likewise, I can understand the motivation for being more type-friendly, even though again, I haven’t needed that personally.

I don’t find the motivation about copying or pickling too persuasive, mostly because I’ve just never needed it. For missing value singletons, I can’t imagine ever needing to pickle or copy them, but I guess it might be useful for a generic singleton object. I lean toward YAGNI; do you have some real world examples where this functionality would be useful?

Likewise, I’m not so sure about the registry. It feels to me like a YAGNI, and it complicates the semantics and implementation. Like someone said above, I’d eliminate that and leave it to the user to ensure whether they wanted unique sentinels on each instantiation or not. I’m also not comfortable with the sys._getframe() call, which would be a performance hit, though it could be avoided by passing a module_name in the PEP’s proposed API.

If I had to boil down sentinels to the most useful, simplest functionality, I’d just want something like object() with a configurable repr, and maybe a settable bool value.

I’d also consider adding a sentinel() callable as a built-in to really make it just as convenient as using object(). The big advantage is that you wouldn’t need to import anything to use it. Having to import a module first would be a mild deterrent to changing my own uses of the object() sentinel in my existing code. This would also mean that you wouldn’t have to worry about what module it lives in, but you’d have to implement it in C. For a simpler API, that might be easy. If you still want a more expansive functionality, or you need it for type hinting, then the built-in sentinel() could just be a front-end into some Python code, returning a Sentinel instance written in Python[1]. It would still be less important exactly where that Python code lives if you never need the underlying Sentinel class.

  1. along the lines of the breakpoint builtin ↩︎

15 Likes
191

FWIW, we have sys._getframemodulename() now to avoid this perf hit when all you want is the name of your caller (but I agree, it should be in the API).

4 Likes
192

I don’t have open source examples I can think of, but a few of the projects I work on use caching interfaces (redis, memcached, etc) to quickly reconstruct data and get significant performance wins.

If you pass an object through pymemcache, it gets pickled and unpickled on its way in and out of the cache. Ensuring that any unpickled sentinels still test correctly is a way of ensuring we don’t get nefarious bugs.

I hope that’s compelling as an example, even if I can’t share the source!

2 Likes
193

Thanks! I’d still like to hear about open source use cases if anybody has one.

I still think the pickling case won’t be too common, but I could be biased about that. Do you think there’s a way to keep things simple for the common case but enable optional pickling support (e.g. if you give it a module_name.name, but otherwise no)?

194

Doesn’t that option make the interface more complicated and therefore the whole thing “less simple”?

The picking is invisible to users who don’t use it. And it may not be obvious how a user of a library’s sentinel will use the sentinel, so the library can’t know whether to enable pickling?

4 Likes
195

As a user, I wouldn’t mind at all if what I’m asked to write is

MISSING = sentinellib.Sentinel(__name__, "MISSING")

There’s precedent in logging and typing for that kind of pattern and it doesn’t feel awkward to me.

With only one required positional arg you could make the module part optional, i.e.

Sentinel("MISSING")

And the implementation should simply error if you try to pickle a sentinel which doesn’t have a module name.

The downside is that a usage like _default = object() necessarily gets at least a little less ergonomic as a result. Is _default = Sentinel("_default") acceptable from that angle?

I’ll try to think of an open source case in which a sentinel gets stored somewhere via pickle. My go to open source example for this whole PEP is marshmallow.missing, which is definitionally

  • part of the library’s API
  • unique to the library
    and happens to be falsy. But I’m not sure if it’s ever stored, unless you were to cache a schema after instantiation, to try to micro-optimize? Seems far fetched.
196

I’d have thought singletons being passed through pickle via multiprocessing would pop up from time to time.

9 Likes
197

Just because you asked: I’d be keen to use sentinels in Sphinx, where a large part of the project’s design is that everything is pickled, written to disk, and then unpickled and re-read when writing output (e.g. HTML). We could probably get around a lack of copy/pickle support by clearing sentinel objects or doing something in __{get,set}state__, but it is something that would be nice to have.

A

10 Likes
198

I can also confirm that I have seen many uses for Sentinels in pylint. We use pickle in our multiprocessing based --jobs feature. I wouldn’t mind having to pass the module name or doing something else to make it work with pickle, but I would expect the stdlib to at least offer the possibility.

3 Likes
199

The registry is used to avoid having duplicate, separate instances created when serializing+deserializing, e.g. pickling and unpickling. The need to avoid this is to have comparisons with the is operator work as expected. I consider this to be a core requirement of this PEP: Working correctly after deserialization must be robust and work out of the box, without anything special needing to be done when using this new functionality.

3 Likes
200

I initially had this PEP suggest creating a dedicated type for each sentinel, which is what you’re asking for if I understand correctly. However, that has a significant “price tag” in terms of runtime during creation, additional memory footprint, and complexity. I eventually decided to drop creating dedicated types for those reasons.

What you ask for is doable, but I don’t consider it important enough to justify the “price”.

201

I appreciate the suggestions, but I consider it very important for this to be as simple and minimal an addition as possible. Creating several different classes, and/or a base class for singleton objects, is far far more than what I’m willing to consider in the scope of this PEP.

202

Thank you for spending your time and energy on this Barry! And for sharing your personal thoughts early in the review process. I appreciate it greatly.

Regarding copying and unpickling, this requirement comes up when using tools for caching (as mentioned by @sirosen), RPC (e.g. gRPC) and remote task execution (e.g. Celery, RQ), among others. When using generic tools such as these, one quickly runs into surprising and tricky edge cases with objects which do not survive copying/unpickling well. A great example is exception objects, and you’ll find a lot of complicated code for handling those in such libraries: Here’s an example from Celery.

There’s one main reason I think it’s worth making the effort to make this “just work” rather than leave it to the creator of the sentinels to decide: These sentinels are intended foremost to be used in function/method signatures, especially when those are part of APIs (hence the desire for clear reprs, for example.) However, those running into issues with copying and serialization will often be users of those APIs, who will have little or no way to address these issues other than complicated workarounds.

Regarding the registry, I don’t see how it complicates the semantics; it is an internal implementation detail. Unless you mean the semantics of “sentinels with the same name from the same module will always be the same object”, which is much simpler than any alternative I could get to going through the many possible permutations of implementing something like this.

Regarding adding sentinel() as a builtin, I’d thought of that but assumed adding a builtin just for this would be a no-go. If it is “on the table” then I’m certainly open to that. (Implementing in C if needed would be no issue.)

11 Likes