PEP 661: Sentinel Values

Well of course until type checkers don’t implement a particular feature it cannot be used to type code.
My reasoning is that that since this PEP doesn’t at the moment specify anything regarding type checking there is nothing for typing tools to implement. So before supporting Literal[MISSING] (or any similar syntax to type individual Sentinels) a new PEP would need to be created and approved.

2 Likes

Gotcha, thanks!

After additional feedback (here and elsewhere) I’m going to make another effort at finding a way to have type signatures better than just Sentinel.

Here’s my current thinking:

  1. It would be desirable for type checkers to be able to recognize that if a value isn’t a sentinel, they can deduce that is of a more specific type. For example, if a function argument’s type annotation is value: int | MISSING, and the function includes a if value is MISSING: statement, then in the else: we can be sure the type of value is int. This works now with None and likely also when using Literal.

  2. I’d like type checkers to have to do as little as possible to specifically support these sentinels. If they need to implement specific support, it should be reasonably simple and straightforward.

  3. The problem with using Literal (e.g. Literal[MISSING]) is that sentinel values are not literals (unlike string literals). It appears that type checkers would have to implement very different logic to support this than what is currently in Literal. I suspect that there would also be nuanced ways in which sentinel literal type annotations would work differently than those for strings, which worries me.

  4. Having a dedicated type for each sentinel is possible in various ways, as suggested in previous drafts of this PEPs and the implementation using a single-value Enum, for example. However, the type signatures would be non-obvious, and these wouldn’t achieve #1 above without special-casing by type checkers.

  5. Making sentinel values be their own types (e.g. using the method mentioned here by @njs and @BlckKnght) avoids needing a separate, dedicated class, which is fantastic! But using them for type signatures doesn’t work, because the magic involved apparently doesn’t work for static analysis. (Again, this could be solved by special-casing by type checkers.) And this also doesn’t support #1.

  6. All of which leads me to think about a new typing construct. It seems we’ll need special casing by type checkers anyways, so we may as well make it explicit. Also, I feel that re-using Literal for values which are not literals would be a mistake.

So, how about Sentinel[MISSING] or Sentinel['MISSING']? With Sentinel being the actual class, this makes sense: it’s a specialization of Sentinel. It parallels Literal, but avoids overloading it. It would be the most straightforward for type checkers to implement, since it would be a separate thing. And I’m hoping it would be obvious enough for readers of code to understand the meaning without needing to look it up.

Thoughts on this?

5 Likes

Sentinel['MISSING'] (assuming that’s supposed to mean that what’s annotated with it is Sentinel('MISSING')) seems like it would be problematic cross-module since you wouldn’t be able to tell it that you actually mean the sentinel created in another module (something that I think could be common, one example that comes to my mind is REST APIs that differ between None and not passing the field where you could want to have single MISSING sentinel representing not passing the field across the whole, potentially large, library). This could be solvable with a second argument when doing things cross-module but it seems like it wouldn’t be trivial to use.

I think Sentinel[MISSING] would make more sense from the 2 but I’m a bit unsure why that would be preferred over just doing MISSING directly as MISSING already is the specialization of the Sentinel class so it would obviously (in my mind, can’t speak for other people) mean that you’re referring to that specialization of sentinel class. Maybe there’s a reason type checker authors would prefer it though in which case I don’t mind having it written like that much. It would mean I have to import Sentinel in addition to MISSING in every place that I want to type hint something with MISSING though while I could avoid that if I could use MISSING directly.

It seems to me that regardless of the chosen method, type checkers will have to specialize for it so not having to special-case being a goal probably isn’t realistic. And type checkers already need to have a way to detect if something is an enum member so having to detect if something is a specific sentinel should work similarly (you have to detect call to Sentinel rather than a class definition subclassing from Enum so that part is somewhat different, there may be some other areas where that kind of detection is already performed though).

The (entirely?) new thing is the global registry of sentinels, the type checkers may now need to detect that doing x is Sentinel("MISSING") is the same as x is MISSING if MISSING is defined in current module (and in general track module_name). I know that ideally we would surely want that to work with type checkers as well but personally I’m not sure how often that’s going to occur instead of people doing x is MISSING. I imagine that tracking Sentinels instance across the codebases due to the global registry will make it quite a bit harder for type checkers so I personally don’t think that it’s essential for them to support it, even if ideally type checkers should be able to support new features in full.

I realized after typing it out that you made a nice numbered list of your thoughts and instead of addressing these directly I just went on about the Sentinel[MISSING] vs Sentinel['MISSING'] part :slight_smile:

So in short, I think that #1 is essential and I agree with everything else you said on that numbered list. For #2 I already said in a previous post that I don’t think it’s realistically possible to do without any special-casing but I support the idea of going with something that requires reasonably simple implementation on the type checker’s side (keeping in mind the kind of things they already need to implement to support other popular Python constructs such as the enums).

And thank you for reconsidering type checker support.

I assume Sentinel[MISSING] is supposed to be used like this:

MISSING = Sentinel("MISSING")

def func(value: int, *, flag: str | Sentinel[MISSING] = MISSING) -> None:
    if flag is MISSING:
        ...

But I don’t quite undersetnd what the string form means. Is it equivalent to Sentinel[MISSING]? This would not be a parallel to Literal, and why is it needed anyway (I can’t think of a use case for forward-declaring a sentinel)? Or is it supposed to mean using the literal string "MISSING" as a sentinel? That seems to make even less sense since the use case should be covered by Literal["MISSING"].