Introducing a Safe Navigation Operator in Python

Ideas
228

A semi-baked idea that has been kicking around in my brain the past few days is something I’ve been thinking about as “query expressions” (assorted potential enhancements omitted in the sketch below, like being able to capture the same code location information on the various query result instances as we capture when raising SyntaxError in the compiler, as well as offering common has_value, is_missing, and is_error properties for easier runtime introspection. Distinguishing between raised exceptions and returned exceptions is also a topic that would need further consideration):

class Value:
    """Any Python object that isn't missing or an error. Considered true."""
    def __init__(self, value):
        self._value = value

    @property
    def value(self):
        return self._value
   
    def __bool__(self):
        return True

class Missing:
    """A missing value. Considered false."""
    def __init__(self, sentinel=None):
        self._sentinel = sentinel

    @property
    def sentinel(self):
        return self._sentinel

    def __bool__(self):
        return False

class Error:
    """An error value. Considered false."""
    def __init__(self, exception):
        self._exception = exception

    @property
    def exception(self):
        return self._exception

    def __bool__(self):
        return False

def query(obj: Any, *, sentinel=None) -> Value|Missing|Error:
    """Classify a Python object as missing, an error, or some other value"""
    if obj is sentinel:
        # Do the fastest check first
        return Missing(sentinel)
    if isinstance(obj, (Value, Missing, Error)):
        # Avoid nesting the result wrapper objects
        return obj
    if isinstance(obj, BaseException):
        # Exceptions are always considered errors,
        # even when returned instead of being raised
        return Error(obj)
    # Wrap anything else as a regular value
    return Value(obj)

Given the above foundation, ? would then be defined as a unary prefix operator that was just a shorthand for operator.query, so ?expr would give the same result as query(expr) (aside from the latter lacking code location information).

On its own, that wouldn’t be interesting (aside from potentially being a way to capture code location information for arbitrary expressions), but where I think it has more potential is as a concept that underlies safe navigation in a way that handles the x is not None vs hasattr(x, "attr") discrepancy by saying it means both (and represents those potential results differently).

Firstly though, ?try could be an exception catching query expression, such that:

result = ?try some_call()

translated to:

    try:
        _inner_value = some_call()
    except Exception as _e:
        _inner_value = _e
    _query_result = ?_inner_value
    result = _query_result

and ?await some_call() (along with ?yield and ?yield from) could be defined as a way to avoid a common bug where coroutine and generator exception handlers inadvertently cover more than just the exceptions thrown in when the frame resumes execution:

    _awaitable = some_call() # Note: outside the scope of the try block!
    try:
        _inner_value = await _awaitable
    except Exception as _e:
        _inner_value = _e
    _query_result = ?_inner_value
    result = _query_result

(If you do want to cover both parts of the expression for some reason, then ?try await ..., ?try yield ..., and ?try yield from ... would all remain available)

Returning to the original topic of safe navigation (in this conceptual framework: “attribute query expressions” and “item query expressions”):

result = obj?.attr.subattr

would translate to:

  _lhs = obj
  if _lhs is None:
      _inner_value = _lhs
  else:
      try:
          _inner_value = _lhs.attr
      except AttributeError as _e:
          _query_result = _e
     else:
         # Resolve any trailing parts of the expression
         # This clause would be omitted when not needed
         _inner_value = _inner_value.subattr
  _query_result = ?_inner_value
  result = _query_result

Item lookup (such as result = obj?[some_calculated_item()].attr) would translate to:

  _lhs = obj
  if _lhs is None:
      _inner_value = _lhs
  else:
      _lookup_key = some_calculated_item() # Outside the try/catch!
      try:
          _inner_value = _lhs[_lookup_key]
      except LookupError as _e:
          _query_result = _e
     else:
         # Resolve any trailing parts of the expression
         # This clause would be omitted when not needed
         _inner_value = _inner_value.attr
  _query_result = ?_inner_value
  result = _query_result

With these definitions, obj?.attr would potentially replace a lot of hasattr and getattr usage with a type-safe alternative, and obj?[item] would provide a convenient way to attempt optional item lookups without having to handle KeyError or IndexError yourself (LookupError is their common parent exception).

Due to the way __bool__ is defined in the query result objects, ?? wouldn’t be needed - you would use or in combination with query expressions instead.

This approach would presumably be a bit slower than the simpler definition in PEP 505, but not that much slower:

  • try/except is essentially free these days when no exception is thrown
  • the actual implementation would presumably avoid querying objects when it already knows the result (such as for caught exceptions, or values that have just been checked against None)

Edit: I started a dedicated thread for further iteration on this idea: Safe navigation operators by way of expression result queries

3 Likes