Slice Views for Python Sequences

Ideas
1

Hi, I have this idea that I would like to discuss and maybe find a sponsor for the PEP, if you agree that this is PEPable.

PEP: TBD
Title: Slice Views for Python Sequences
Author: Juliano Fischer Naves julianofischer@gmail.com
Status: Draft

Abstract
This PEP proposes a standard “slice view” facility for Python sequences, analogous in spirit to Go’s slices and NumPy views. A slice view presents a live window into an existing sequence: reads and writes reflect the underlying sequence, view-to-view slicing composes cheaply, and no data is copied unless explicitly requested. Existing slicing semantics remain unchanged; the feature is opt-in via a new standard type and protocol.

Motivation

  • Performance and memory: Today, seq[a:b:c] typically creates a new sequence, copying elements. For large data pipelines, this is costly. A view avoids copying while enabling idiomatic slicing.
  • Expressiveness: NumPy users are accustomed to view semantics; Python lacks a general-purpose equivalent beyond memoryview for bytes-like objects.
  • Interoperability: Libraries often implement their own view wrappers. A standard type and protocol enable zero-copy interop and predictable behavior.
  • Ergonomics: Go slices and similar constructs make sliding windows, subranges, and queue-like operations natural and efficient.

Rationale and non-goals

  • Non-goal: Change existing __getitem__ slice semantics for built-ins like list or str. That would be a large backward-incompatible change.
  • Goal: Provide a standard viewing mechanism that:
    • Works with any collections.abc.Sequence (and MutableSequence for writable views).
    • Composes slices in O(1).
    • Reflects mutations of the underlying sequence when possible and well-defined.
    • Allows types to opt into more efficient views via a small protocol.
  • Relation to existing facilities:
    • memoryview remains the preferred solution for the buffer protocol and contiguous bytes-like memory (e.g., bytes, bytearray, array, numpy via PEP 3118).
    • Slice views target generic sequences where the buffer protocol is not applicable (e.g., lists of Python objects, custom sequences).

Specification
New built-in: sliceview

  • Constructor: sliceview(base, start=None, stop=None, step=None) creates a view over base, applying an initial slice.
  • Helper: view(obj) -> sliceview returns an identity view over obj. Available in builtins.
  • Composability: sliceview implements the sequence protocol. Indexing and slicing on a sliceview return elements or another sliceview respectively, without copying.

Protocol for efficient producer types

  • New optional dunder: __sliceview__(self, slc: slice) -> sliceview | NotImplemented
    • If present, sliceview(base, slc) calls base.__sliceview__(slc) first. Types can return a specialized view (e.g., contiguous block, rope node) or NotImplemented to fall back to the generic wrapper.
    • For bytes-like objects supporting PEP 3118, sliceview may delegate to memoryview internally when that is more efficient and safe, but the Python-level type remains sliceview.

Data model and semantics

  • Identity and lifetime:
    • sv.base references the underlying sequence. Keeping the view alive keeps the base alive.
  • Read semantics:
    • sv[i] maps to base[start + i*step] (with negative step permitted). Index normalization uses the view’s logical length.
    • sv[:] returns a new sliceview pointing at the same base (O(1)).
  • Write semantics:
    • If the base is a MutableSequence, sv[i] = x forwards to the element mapping above.
    • Extended assignment sv[i:j:k] = iterable forwards element-wise. Size-changing slice assignment raises TypeError on a view unless the base’s own __setitem__ for slice with step 1 is used and the view’s step is 1 and contiguous (see “Resizing behavior”).
  • Resizing behavior:
    • Resizing operations on the base (insert/delete) are visible in the view and shift logical indices as they do for ordinary indexing. This is analogous to iterators observing live mutation but not crashing.
    • Direct resizing through the view is restricted to avoid ambiguous semantics:
      • Allowed: sv[a:b] = iterable only if step == 1 and the assignment length matches (b - a). Otherwise TypeError.
      • Methods that conceptually resize (e.g., append, insert, extend, clear) are not provided on sliceview. Users should call them on the base.
  • Length and membership:
    • len(sv) computes from the current base length and the view parameters. It reflects base mutations.
    • x in sv iterates over the view.
  • Hashing and equality:
    • sliceview is unhashable.
    • Equality compares elementwise to other sequences.
  • Slicing a view:
    • sv[p:q:r] returns a new sliceview that composes the slice parameters in O(1), with no data movement.
  • Iteration:
    • Iteration walks the base through the view mapping. Mutations to the base during iteration follow existing Python iteration semantics for lists (no guarantees beyond not crashing and visiting indices that remain valid).
  • Repr:
    • sliceview([1,2,3,4])[1:3] renders as sliceview(base=<list at 0x...>, slice=1:3:1).

Standard library integration

  • collections.abc: Add View ABC? Non-essential. This PEP initially proposes concrete sliceview only.
  • operator: Add operator.sliceview(obj, slc) convenience that mirrors operator.getitem.
  • itertools: No changes required.
  • bisect, heapq: Work on any sequence; they remain compatible but note performance implications.

Error handling and edge cases

  • Negative steps are supported and compose correctly.
  • Out-of-range indices follow slice normalization rules and never raise in slicing; element indexing still raises IndexError.
  • If the base drops elements, indices can become invalid for some positions; accessing them raises IndexError as with ordinary indexing.

Examples

  • Avoid copying a sublist:
    • sv = view(big_list)[offset:offset+window]
  • In-place windowed updates:
    • view(samples)[i:j] = denoise(view(samples)[i:j])
  • Composable slicing:
    • view(seq)[2:][::3][5:10] creates no intermediate copies.
  • Interop with memoryview:
    • view(b)[1000:2000] can use a memoryview-backed fast path under the hood.

Backward compatibility

  • No change to existing slicing. All new behavior is opt-in via sliceview/view().
  • Code relying on slice copies remains unaffected.

Performance considerations

  • Time: Creating a sliceview is O(1). Indexing and iteration overhead is comparable to indexing the base plus simple arithmetic. For tight loops, microbenchmarks should compare favorably to slice-copy approaches when the copy would dominate.
  • Memory: Views hold a reference to the base plus a few integers. Large intermediate slices avoid allocation and element reference bumps.

Security and safety

  • Views prolong the lifetime of their base. This may unintentionally retain large structures; tooling should document memory implications.
  • For types with non-stable element addresses (most Python objects), this is no worse than ordinary indexing; the PEP makes no new pointer-stability guarantees.

Rejected alternatives

  • Changing built-in slicing to return views: backward-incompatible for decades of code relying on copies.
  • Overloading slice objects with view semantics: clarity suffers and still wouldn’t enable opt-in behavior without breaking changes.
  • Only blessing third-party libraries: misses the unifying standardization benefit.

Open issues for discussion

  1. Write resizing via views:
    • Permit size-changing slice assignment when step == 1? Current draft forbids through the view to keep semantics predictable.
  2. ABC or Protocol:
    • Should we define a typing.Protocol like SupportsSliceView with __sliceview__ for static typing?
  3. bytes-like unification:
    • When the base supports the buffer protocol, should sliceview delegate to memoryview or always remain distinct but possibly sharing implementation?

Pure Python illustrative reference implementation
This is a minimal, correct but not optimized prototype that demonstrates the semantics.

from collections.abc import Sequence, MutableSequence

def view(base):
    return sliceview(base)

class sliceview(Sequence):
    __slots__ = ("base", "_start", "_stop", "_step")

    def __init__(self, base, start=None, stop=None, step=None):
        self.base = base
        if isinstance(start, slice) and stop is None and step is None:
            sl = start
        else:
            sl = slice(start, stop, step)
        # Normalize using current length
        b_len = len(base)
        start, stop, step = sl.indices(b_len)
        self._start, self._stop, self._step = start, stop, step

    def __repr__(self):
        s = f"{self._start}:{self._stop}:{self._step}"
        return f"sliceview(base={type(self.base).__name__}, slice={s})"

    def __len__(self):
        # Compute length like range does
        start, stop, step = self._start, self._stop, self._step
        if (step > 0 and start >= stop) or (step < 0 and start <= stop):
            return 0
        n = (abs(stop - start) + abs(step) - 1) // abs(step)
        return n

    def _map_index(self, i):
        # Support negative indices relative to the view
        if i < 0:
            i += len(self)
        if i < 0 or i >= len(self):
            raise IndexError("sliceview index out of range")
        return self._start + i * self._step

    def __getitem__(self, key):
        if isinstance(key, slice):
            # Compose slices: map to base indices and create new view
            start = key.start
            stop = key.stop
            step = key.step
            # Translate key to absolute on the view
            vlen = len(self)
            sstart, sstop, sstep = (slice(start, stop, step)).indices(vlen)
            # Compose with base mapping
            base_start = self._start + sstart * self._step
            base_step = self._step * sstep
            # Compute base_stop from count
            count = (abs(sstop - sstart) + abs(sstep) - 1) // abs(sstep) if vlen else 0
            if count == 0:
                # Empty view: choose a canonical empty slice on base
                return sliceview(self.base, 0, 0, 1)
            base_stop = base_start + count * base_step
            return sliceview(self.base, base_start, base_stop, base_step)
        else:
            return self.base[self._map_index(key)]

    # Optional: enable write-through for mutable sequences
    def __setitem__(self, key, value):
        base = self.base
        if not isinstance(base, MutableSequence):
            raise TypeError("underlying sequence is not mutable")
        if isinstance(key, slice):
            # Restrict to non-resizing, step==1 for clarity
            sstart, sstop, sstep = key.indices(len(self))
            if sstep != 1 or sstop - sstart != len(value):
                raise TypeError("resizing or stepped slice assignment via view is not supported")
            # Map each index and assign
            bi = self._start + sstart * self._step
            for x in value:
                base[bi] = x
                bi += self._step
        else:
            base[self._map_index(key)] = value

    def __contains__(self, x):
        for y in self:
            if y == x:
                return True
        return False

Typing

  • A typing.SupportsSliceView protocol could standardize __sliceview__. sliceview itself implements Sequence, and conditionally a MutableSequence-like subset for write-through, though it intentionally avoids resizing methods.

Reference documentation additions

  • builtins.view(obj): return a view proxy over any sequence-like object.
  • class sliceview: describe constructor, behaviors, and caveats.

Migration strategy

  • None required. Libraries can adopt sliceview to avoid internal copies and to expose efficient APIs. Tutorials should recommend view(seq)[i:j] when a non-copying window is needed.

Acknowledgements

  • Inspired by Go slices and Python’s memoryview, as well as longstanding community discussions around zero-copy sequence windows.
15 Likes
Helping people with "broken" sorting
2

Can’t you just use sliceview(obj)?

3 Likes
3

I’ve written my own view-like class before. The normal slicing is difficult to compose until you realize you can apply a slice to a range and that works it out for you. Don’t think this would be needed as a builtin but I think it would be useful.

1 Like
4

This is a nice writeup. I’m not sure this is a common enough need to warrant a new builtin, but I left some specific feedback.

Since the system mostly doesn’t rely on changes to the core, you should be able to implement this yourself (e.g., in a PyPI library) to prove its usefulness.

What does this add over the sliceview constructor?

Generally, the repr() should ideally be something that you can theoretically eval(). Why not mirror the constructor syntax, something like sliceview([1, 2, 3, 4], 1, 3)?

It should be in collections.abc; the typing variants of these ABCs are deprecated.

I’m not sure how useful this Protocol/ABC would be, since sliceviews are supported for all sequences: when would you write code that only works with objects that explicitly support __sliceview__?

Would any standard library classes implement this new dunder?

2 Likes
5

It could be.

6

I hope this makes progress! Generalizing the semantic power of memoryview() to more sequences would add real value.

I suggest losing view(). A 4-character shortcut for typing sliceview() isn’t needed, makes clear at once that there’s some conceptional connection with memoryview(), and matches the name of the new dunder method. view() on its own doesn’t bring anything particular to mind.

15 Likes
7

You’re right: as proposed it adds little over calling sliceview(obj, slc.start, slc.stop, slc.step) or sliceview(obj, slc). Convenience is marginal compared to operator.getitem, which is needed to dispatch the actual[] operator.

Agreed.

Thanks, I agree it belongs in collections.abc.

Right now, I can argue although not strictly needed, Protocol/ABC would be useful for specialized subtypes. For instance, a SortedList could have a SortedListView that exposes bisect methods. But I really need to think more about it and elaborate it better.

It could also be useful for types that are not strictly sequences but may “behave” like sequences. I collections.deque based RingBuffer (CircularBuffer) doesn’t support slices, but it could support a view.

This is my opinion. I would like to hear from you all.

Initially, the PEP should not require that any standard library implement this new method. The adoption should follow up change once the API proves out. If adoption proves beneficial, we should consider list, array.array/bytearray (maybe the most promising use cases?)

8

The reason I asked is that it wasn’t obvious to me what implementing __sliceview__ would add. If list were to add a __sliceview__ implementation, what would it do, and what advantage would it provide over using the default behavior without a special __sliceview__ method?

4 Likes
9

I was thinking same. I think the default implementation would just return a default sliceview, and classes that have specialized sliceview sublcasses would return that. So I think it wouldn’t be needed for most classes that implement the Sequence api but maybe for some that don’t. collections.deque would be an example that might use a custom view class to help with performance for iterating. Anything that doesn’t have O(1) random access.

Above based on previous comments:

1 Like
10

You’re right.
I guess this only makes sense if we treat the sliceview as a built-in
In this case __sliceview__to provide C-specific and optimized implementations (but here, I highlight I don’t have the needed knowledge about it, so that is possibly why I am talking some nonsense for you here).

Sorry, guys, I am still putting these ideas in place.

1 Like
11

__sliceview__ would still need to return a Python object. Assuming sliceview is optimized, there isn’t much that can be done. The only possible speedup (I can see) is to bypass the __getitem__ calls. But that’s questionable. Edit: Good for sequences without O(1) random access.

12

I think functionality is useful.
I would make use of such pretty much in all cases provided in examples.

However, in my experience, the most common use case is iteration.

E.g.:

a = list(range(100_000))
for el in itertools.islice(a, 50_000, 55_000):
    ...

So it is either the above or the need to create a new list via usual a[50_000:55_000]. And neither of them feel right.

This is analogous to __reversed__.

And __reversed__ would be a subset of this.

This is a bit more complex as performant C implementation __sliceview__ would need 2 objects. e.g.: list_sliceview and list_sliceview_iterator.

While __reversed__ only needs one.

Thus, if this was implemented for all objects that implement __reversed__, then __reversed__ implementations could pretty much be left only as shortcut returning iter(sliceview(obj, step=-1)), as:

a = reversed(obj)
# Equivalent
b = iter(sliceview(obj, step=-1))

In theory.

There would probably be some issues with backwards compatibility as reversed would be returning different object.

But slow deprecation could be an option.

2 Likes
13

For this example, it might be worthwhile considering extra method. e.g. advance(n: int)

E.g.:

def windowed_process(samples, window):
    for i in range(0, len(samples), window):
        sliceview(samples)[i:i+window] = denoise(sliceview(samples)[i:i+window])

For small windows, sliceview object creation could result in non-trivial performance overhead.
Instead:

def windowed_process(samples, window):
    view = sliceview(samples, 0, window)
    for _ in itl.repeat(None, len(samples) // window):
        view[:] = denoise(view)
        view.advance(window)

This is one of more useful ones from similar objects that I have seen. e.g.:

  1. pypy/rpython/rlib/listsort.py at main · pypy/pypy · GitHub
  2. glidesort/src/mut_slice.rs at master · orlp/glidesort · GitHub
1 Like
14

For this example, it might be worthwhile considering extra method. e.g. advance(n: int)

This would advance the view by 1 step or by 1 index?

15

Good question. Both are useful for different cases.

However, advancing by step is a subset of advancing by index.

Thus, maybe advancing by index would have more value.

It would require some extra operations for certain cases, but I don’t think it would be big inconvenience:

view = sliceview(arr, step=2)# step = 2
view2 = view[::2]            # sep = 2 * 2 = 4
# So say need to advance by step
# But it advances by index. Then:
view2.advance(2 * 2 * nsteps)

While if advancing by step was default, then advancing by index would not be possible. E.g.:

arr = [0, 1, 2, 3, 4, 5]
# Say I need: [0, 2], [1, 3], [2, 4], [3, 5]
view = sliceview(arr, 0, 3, step=2)
for _ in itertools.repeat(None, len(arr) - 2):
    yield list(view)
    view.advance(?)
    # advancing by index is needed:
    # view.advance(1)
16

Also, my mistake.

advance advances only start in timsort of PyPy, while glidesort has methods to advance both start and end.

Thus, maybe sliceview.advance(start=NotGiven, end=NotGiven) or something similar that can do both would cover this more properly.

1 Like
17

I think this is important but requires some attention. There is no requirement for collections.abc.Sequence or collections.abc.MutableSequence to support slicing because slicing doesn’t always make sense for the semantics of the concrete sequence subclass.

collections.deque can’t be sliced at all, the rationale being that slicing a linked list just can’t easily be efficiently implemented. The fallback strategy of iterating over a sequence while keeping track of the index and skipping items is fine, but mutable slicing by index

requires a sequence that one can efficiently index into, which most linked lists (and user-written sequences) probably aren’t (I suspect the time complexity is not just O(n) in the case of deque). I think that without further knowledge into the implementation of the concrete sequence type, the default fallback sliceview should be read-only, with the mutable version being opt-in even for otherwise mutable sequences.

18

Indeed, @rhettinger once said that the only reason deque supported indexing at all was so people could peek at “the first” and “the last” elements via the familiar d[0] and d[-1] spellings.

What we have now has O(n) time indexig in general, but very fast if the index is very near the start or the end of the deque. Worst case is d[len(d) // 2]. All specific to CPython, of course, and subject to change in the future.

Nevertheless, a sliceview view of a deque could implement arbitrary slicing on its own. I don’t think it would be worth the pain, though. I’ve only used 0 and -1, as Raymond intended.

1 Like
19

Move `collections.deque` to a growable ring buffer for fast random access · Issue #731 · faster-cpython/ideas · GitHub promises O(1) deque random access.

2 Likes
20

Amortized O(1), yes. deque as-is has no “slow” cases for pushing or popping, at either end, because nothing ever needs to be resized. It was designed to be … well, a deque :wink:.

EDIT: although, yes, so long as the size isn’t changing, indexing is indeed O(1) period (best, average, and worst case, always).

1 Like