Feature proposal: a `SequenceProxy` class

Proposal summary

I propose that Python should add a new SequenceProxy class to the standard library.

Feature specification

The new SequenceProxy class would aim to emulate the functionality of MappingProxyType, but would do so for arbitrary sequences rather than arbitrary mappings. It would implement the collections.abc.Sequence interface in full, and would additionally implement all parts of the MappingProxyType interface that are not mapping-specific. The result would be a read-only, dynamically updated, iterable proxy class for sequences.

Problem to be solved by this proposal

There is, obviously, no such thing as a truly “private” attribute in Python. This is a fundamental aspect of the language, and something I have no wish to change. However, the existence of the descriptor protocol and the builtin @property decorator implicitly concede that it is often useful to create a layer of indirection between an object and the ability to alter the value of an attribute of that object.

However, @property and other Python descriptors are somewhat problematic when it comes to mutable attributes. Say, for example, that I want to implement the common pattern of appending all new instances of a class to a list that is held as a class attribute. I want to signal to readers and users of my code that this list should not be modified from outside the class, so I give this list a name starting with a single underscore, and create a property for access to the list from outside the class:

class Foo:
    _all_foos = []

    def __init__(self):
        self._all_foos.append(self)

    @classmethod
    @property
    def all_foos(self):
        return self._all_foos

Anybody who really wants to can set/delete _all_foos from outside the class, but it is impossible to set/delete all_foos directly. It is, however, still very much possible to append to _all_foos, insert items into all_foos, pop/delete items from all_foos… etc. Arguably, this makes the @property decorator of very limited use for mutable attributes.

An easy solution to this problem exists for private mappings, which you can implement like so:

from types import MappingProxyType

class Bar:
    _all_bars = {}
    _all_bars_proxy = MappingProxyType(_all_bars)

    def __init__(self, name):
        self._all_bars[name] = self

    @classmethod
    @property
    def all_bars(self):
        return self._all_bars_proxy

However, no such out-of-the-box solution is provided for sequences.

Suggested solution to this problem

The Python stdlib should add a new SequenceProxy type – like MappingProxyType, but for sequences. It could look like this (sans type hints for the standard library, obviously):

import sys
import collections.abc
from typing import TypeVar, Sequence, Generic, Iterator, Any, Union, overload

T = TypeVar('T')
S = TypeVar('S', bound='SequenceProxy[Any]')

@collections.abc.Sequence.register
class SequenceProxy(Generic[T]):
    """Read-only proxy for a sequence.
    Similar in concept to `MappingProxyType`,
    but for sequences rather than mappings.
    """

    __slots__ = '_sequence',

    # Set __hash__ to None, since we don't know
    # if the underlying sequence is hashable or not.
    # It also maintains consistency with `MappingProxyType`.
    __hash__ = None

    def __init__(self, /, initsequence: Sequence[T]) -> None:
        if not isinstance(initsequence, collections.abc.Sequence):
            raise TypeError(
                f'SequenceProxy() argument must be a sequence, '
                f'not {type(initsequence).__name__}'
            )
        self._sequence = initsequence

    def __iter__(self, /) -> Iterator[T]:
        return iter(self._sequence)
        
    @overload
    def __getitem__(self: S, i: int, /) -> T: ...
    
    @overload
    def __getitem__(self: S, i: slice, /) -> S: ...

    def __getitem__(self: S, i: Union[int, slice], /) -> Union[T, S]:
        if isinstance(i, slice):
            return self.__class__(self._sequence[i])
        return self._sequence[i]

    def __len__(self, /) -> int:
        return len(self._sequence)

    def __contains__(self, item: Any, /) -> bool:
        return item in self._sequence

    def __repr__(self, /) -> str:
        return f'{self.__class__.__name__}({self._sequence!r})'

    def __str__(self, /) -> str:
        return str(self._sequence)

    def __reversed__(self, /) -> Iterator[T]:
        return reversed(self._sequence)

    # Implementation of the equality methods
    # takes its inspiration from `collections.UserList`
    
    @staticmethod
    def __cast(other: Any, /) -> Any:
        return other._sequence if isinstance(other, SequenceProxy) else other

    def __eq__(self, other: Any, /) -> bool:
        return self._sequence == self.__cast(other)

    def __ne__(self, other: Any, /) -> bool:
        return self._sequence != self.__cast(other)

    def __gt__(self, other: Any, /) -> bool:
        return self._sequence > self.__cast(other)

    def __ge__(self, other: Any, /) -> bool:
        return self._sequence >= self.__cast(other)

    def __lt__(self, other: Any, /) -> bool:
        return self._sequence < self.__cast(other)

    def __le__(self, other: Any, /) -> bool:
        return self._sequence <= self.__cast(other)

    def index(self, value: Any, start: int = 0, stop: int = sys.maxsize) -> int:
        """Return the index of a value in the underlying sequence."""
        return self._sequence.index(value, start, stop)

    def count(self, value: Any, /) -> int:
        """Return the number of times a value appears in the sequence."""
        return self._sequence.count(value)

    def copy(self, /) -> Sequence[T]:
        """Return a shallow copy of the underlying sequence."""
        # Raises `AttributeError` if the underlying sequence
        # does not have a `.copy()` method,
        # consistent with the behavior of `MappingProxyType`.
        return self._sequence.copy()    

Having this SequenceProxy class would allow me to write my original Foo class easily, like so:

class Foo:
    _all_foos = []
    _all_foos_proxy = SequenceProxy(_all_foos)

    def __init__(self):
        self._all_foos.append(self)

    @classmethod
    @property
    def all_foos(self):
        return self._all_foos_proxy

Prior art

This implementation is quite similar to the ListView class in the third-party immutable-views package on PyPI, among others.

Conclusion

I’d be very interested to hear people’s thoughts on whether something along these lines would be a worthy addition to the standard library. I find myself implementing something like this on a regular basis, so would appreciate having something that could be prepackaged with Python. It could possibly live in types, along with MappingProxyType, or in collections.

To solve your problem, you could also have the property return a tuple converted from the list. It wouldn’t be updated when the proxied list is updated however

MappingProxyType was added to the stdlib because it had use-cases in the
stdlib, and only much later was it made a public type.

Are there use-cases for SequenceProxyType in the stdlib?

Interestingly, MappingProxyType instances already work for some sequences such as strings:

>>> from types import MappingProxyType as MPT
>>> proxy1 = MPT("abcd")
>>> proxy1
mappingproxy('abcd')
>>> list(proxy1)
['a', 'b', 'c', 'd']
>>> proxy1[2]
'c'
>>> proxy1[-2:]
'cd'

But making proxies of lists and tuples is explicitly forbidden at cpython/descrobject.c at main · python/cpython · GitHub

That’s fascinating. Does anybody have any idea why lists and tuples are hardcoded in as being forbidden, and only those two sequence-types?

Because it is difficult to distinguish sequence from mapping in Python 3. Both have the same special methods (__getitem__, __len__, __iter__, __contains__, etc), but with different semantic. Lists and tuples are two common sequence types, so forbidding them explicitly prevents most of errors.

1 Like

I think the argument for having it in the stdlib isn’t that there’s a strong need for it as such, but that it:

  1. Is analogous to other useful things that are in the stdlib.
  2. Seems kind of trivial to pull in as a dependency in a lot of applications, so devs are more likely to just live without it (or re-re-re-implement it everywhere it’s needed).
  3. Some people find it disconcerting that there are stdlib libraries for things like email parsing, JSON, and handling WAV files, but some “basic” data types like this are lacking.

I’ve wished for a lot of similar “little” things along these lines: ordered sets, frozen dicts, frozen lists, et alia. I believe the former has two good implementations on PyPI, but I don’t know about the others.

1 Like