PEP 661: Sentinel Values

Isn’t the whole point of using a sentinel that nobody should be using it for anything outside of the specific use as a marker that it was created for?

I’m not sure I follow. The example function could be used like this:

Today = sentinel()

def foo(date=Today):
    assert_arg_type("date", date, datetime.date)
    if date is Today:
        date = datetime.date.today()
    ...

(Edit: Renamed Default to Today.)

It’s not itself using a specific sentinel, which is kind of the point of the example.

(I’m not sure how useful this concrete function is, it’s just an illustration what would be possible.)

Why not just do:

def foo(date=Today):
    if date is Today:
        date = datetime.date.today()
    assert_arg_type("date", date, datetime.date)

Please ignore the specifics of the example. Maybe the function would handle sentinels specially in some other way? Or maybe the default is not a date, but some other type, so that it couldn’t be passed to assert_arg_type? Or maybe whoever writes the assert function just prefers to have all type assertions at the top of the file? There are many possibilities to explore that wouldn’t be possible without a subclass.

I think the problem is that the use cases for this are very weak, and it’s hard to come up with any that would be compelling. For example, I would find the example you chose to be a bit dangerous because it seems subtly wrong: your function asserts that the argument is either a date or it’s some Sentinel subclass, but then only handles the specific case of Default. If someone passes another Sentinel subclass into that function, it will be used in places that are only expecting a datetime.date.

Any time I’ve used a sentinel I’ve wanted it to be a specific object, and even if I had a base Sentinel class it would have been a mistake to use it, since I never need to detect “all sentinels”.

I’m leaning toward objecting to the PEP as written. I’ve been thinking about this more, and it’s not fully clear to me why the rejected idea Using class objects was rejected.

I think for the infrequency of defining a sentinel, a base Sentinel class should be sufficient. A Sentinel base class can handle __repl__ and __str__ implementation. There would be no changes to any type checking implementations.

class MySentinel(Sentinel):
    pass

def foo(state: str | MySentinel):
    ...

Other than the extra line of code, why is the class-based Sentinel not right for the job?

That’s true. The lack of need for changes to type checkers is indeed nice, but I’m not sure how major a consideration that is (yet).

On the other hand, there are two drawbacks with using a class object for a sentinel which I find significant:

  1. It’s confusing! These are classes which are never instantiated and are not used as classes. For someone who hadn’t learned about these yet, seeing class MySentinel(Sentinel): pass for the first time would be unclear. On the other hand, MY_SENTINEL = sentinel('MY_SENTINEL') is rather straightforward (especially given the similarity to the interface of namedtuple).

  2. The use of a value in a type signature, rather than its type, is unusual. For objects signifying values, this currently only works for None AFAIK, while other values require the use of Literal[] (or their dedicated type, e.g. for Ellipsis?). For those working on/with Python type checking a lot, str | MySentinel may seem natural, but I fear it won’t for many users. And for something as relatively minor as sentinels, I’d rather there be a minimum of special behaviors to be learned.

Finally, in a discussion about type annotations for sentinels from 2019, Guido wrote:

That’s a very expensive sentinel though. Classes are notoriously heavy objects.

I’m not sure how significant of an issue that is, but it’s there. (The PEP currently suggests generating a dedicated type and object for every sentinel, which is even worse in this regard. Alternative implementation approaches can avoid this, though.)

While I’ll admit that the use cases I’m coming up with are a bit weak, I haven’t seen any argument why having a base class would be problematic. Not having it, on the other hand, could turn out to be regrettable in the future.

Pro:

  • Might be useful in the future for testing, logging, auditing use cases.
  • Can be used to type annotate functions that operate on sentinels.

Con:

  • ?

Edit: Please note that I am not advocating for Paul Bryan’s suggestion. My suggestion is basically to amend the code from the PEP as follows:

class Sentinel:
    pass

def sentinel(...):
    ...
    cls = type(class_name, (Sentinel,), class_namespace) 
    ...

I tend to agree. The only potential downsides I can think of don’t seem like big issues:

  1. People could mistakenly use the Sentinel class for creating sentinel objects (we could document against that and make the class’s new fail)
  2. People could use the base class in type signatures rather than something more specific (again, we’d document and make doing “The Right Thing” hopefully even simpler)

I think I’ll probably end up adding this pretty much as you suggest.

One goal not mentioned in the PEP is that after reload(), or otherwise re-executing code that defines a sentinel, the same instance remains, rather than having multiple “generations” co-exist in RAM.
This is similar to the pickling problem, but worth listing on its own.

Many of the rejected ideas fail this goal. For example every re-execution of a class statement bind the name to a new class object (unless a clever metaclass is used).


BTW, what if we settled for sentinels that’d robustly support == but not is? Probably a bad/unpopular idea, but worth explicitly saying it in the PEP — currently it kinda implicitly takes it for granted. Reasons I can think of:

  • if the other object redefines equality, it might wrongly match a sentinel.
  • people will try is, for most of them it’ll work in practice, and they’ll use it anyway without understanding the risks.
  • The is None idiom is so firmly established that diverging from it would feel weird, and people would just expect is to work.
  • performance.

Hi Beni!

Could you perhaps explain why a sentinel should remain unchanged after a reload() or other code re-execution? As you mentioned, this doesn’t hold true for other things like classes. I think having a new, different sentinel object could be considered the expected behavior, and the opposite surprising.

Another point maybe worth discussing in the PEP is “sentinels” vs “symbols”.
In languages that have symbols, they work like “duck enums”, in that 2 independent modules can construct the very same symbol.

It seems to me from reading the Rejected ideas there is concensus that’s an anti-goal. We want a way to get a “dedicated and distinct sentinel” without worrying someone else would produce it (other than by importing from the module where it was defined).

Plus Python has a long history of using regular strings (or interned strings when is matters) in most scenarios other languages would use symbols.

100% fair question! It’s possible that surviving reload() will not be worth the implementation trouble, or the surprise (better a predictable wart than magic?).

The short answer is "because that’s the good life I had with builtin singletons like None or Ellipsis :wink:

It only matters when the sentinel gets stored inside data structures that persist. Say I’m working on a game, and using reload/exec to retain game state. I may then get a mixture of several distinct instances in RAM, conceputally all the same — will my code know that?
(Compare that to working on a game and using pickle to save/restore state. Similar question…)

  • If I rely on identity (is), I want a single instance to avoid a mess.
  • If we settle for equality, the problem is certainly easier! As long as sentinels are named, 2 instances from same module & name will be ==.
  • _sentinel = object() idiom just can’t work, neither for reload/exec nor for pickles. Anonymous sentinels are a dead end for several reasons.

P.S. how about implementing sentinels as empty modules?! E.g. if I’m in module foo.bar and run sentinel('quux'), I get back something like <module 'foo.bar._sentinel_quux'>.
(In fact, a dead silly way to this right now is just create an empty .py file and import it!)

  • :+1: Modules already have a very “singleton” semantics in python, with a global namespace and create-once-then-reuse.
  • :question: Might have unreasonable overhead in some python implementations?

I think having a common base class should be a non-goal. Further, I think we should explicitly say the solution, if accepted, will not have a common base class among sentinels.

I think if we had a common base class it would encourage people to leverage that fact, and we don’t want to encourage that usage pattern. What could possibly be gained? A function that only operates on sentinels? A typed value that can be any sentinel?

And it’s not like we would go through and change every sentinel in the stdlib to be a “new sentinel”, so we’ll never achieve the goal of every single sentinel having the same base class, anyway.

7 Likes

Type hinting, even just for documentation purposes, is becoming a widespread practice. I propose that the PEP address type hinting explicitly.

If you agree with this proposition, then it would be good to get down to brass tacks on how to type hint a value with a sentinel. I believe settling this would also help settle some of the other questions, including whether there is a need for a base class.

I’m definitely in favour of a sentinel-as-a-type, so that it can be listed within a union hint (Union[...] or |). I suppose modifying Literal into accepting a sentinel is an option as well, though the aesthetics of it are less appealing to me.

In a nutshell:
def foo(x: str | MySentinel)”   >   “def foo(x: str | Literal[MySentinel])

What’s not clear to me—due to inexperience with static type checkers—is how well static type checkers can cope with what the PEP is proposing as a dynamically generated type.

Wouldn’t you want sentinels to indicate an optional parameter with no default value? Is it too late to add to the semantics of typing.Optional?

1 Like

One possible gain if there was a common base class is the typing module could add a new generic alias for sentinel kwargs, similar to Optional. Just as Optional[foo] is just sugar for Union[foo, None] a new type Maybe[foo] could just be sugar for Union[foo, Sentinel]. Some accuracy in type would be lost as the only way to make this convenient is to accept any Sentinel, but I think the readability of a function signature would go up.

1 Like

I think this would be a huge mistake, and the source of hard to find bugs.

I wish that we could type a function like this:

def foo(arg_one: int, arg_optional: Sentinel|int = Sentinel):
    if arg_optional is Sentinel:
        arg_optional = 99
    ...

as

def foo(arg_one: int, arg_optional: int = Sentinel):
    if arg_optional is Sentinel:
        arg_optional = 99
    ...

To my mind, that much more clearly expresses the intent of the function as having two integer arguments (where the user can omit the second because it has a default). After all, if we put the default directly in the signature, it would look like

def foo(arg_one: int, arg_optional: int = 99):
    ...

The sentinel is an implementation detail, and other than exceptional cases like wrapping the function, callers should never use it. I’d prefer it if the types reflected that. But I understand that would require a change to how typing works.

2 Likes

Very very early on, if you specified None as default you didn’t have to use Optional, but then this was disallowed. See the end of Union types in PEP 484.
It might be good to dig up the arguments around that decision.

1 Like

That’s an excellent idea @encukou! Has anyone got some references handy?