# Backward enumerator

**URL:** <https://discuss.python.org/t/backward-enumerator/74480>\
**Category:** Ideas\
**Created:** [December 16, 2024, 9:53pm UTC](https://discuss.python.org/t/backward-enumerator/74480 "2024-12-16T21:53:04Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [December 16, 2024, 9:53pm UTC](https://discuss.python.org/t/backward-enumerator/74480/1 "2024-12-16T21:53:04Z")

</div>

Currently python provides `enumerate(Iterable)` for forward enumeration. The API returns a `generator` of tuples of index and item.

However, there lacks a builtin function to enumerate from the back of a `Sequence` (Iterable _ **plus** _ len and getitem). Currently the most popular solution on StackOverflow is `reversed(list(enumurate(...)))` (reference to this [post](https://stackoverflow.com/a/529427/13165694)). This consumes the entire generator and stores them as a list before it can be reversed.

I propose providing an additional builtin function `enumerate_back` **or** add an optional flag `enumerate(..., reversed=True)` to support reversed enumeration of `Sequence` objects.

> P.S. For the flag version, the input type should be narrowed from `Iterable[T]` to `Sequence[T]`.

Here is a demo python implementation of `enumerate_back`:

```python
from typing import TypeVar, Sequence, Generator, Any

T = TypeVar("T")

def enumerate_back(seq: Sequence[T]) -> Generator[tuple[int, T], Any, None]:

    if not hasattr(seq, " __len__") or not hasattr(seq, " __getitem__"):
        raise TypeError(f"{seq} does not support sequencing")

    indexes = range(len(seq) - 1, -1, -1)
    items = reversed(seq)
    yield from zip(indexes, items)

```

Given that ` __getitem__ ()` may incur reading from file or even a remote fetch, the proposed API will be more memory efficient than the solution from the aforementioned SO post which consumes everything before reversing.

* * *

> **My own use case (ROS2)**
>
> ```python
> import rclpy
> from rclpy.node import Node
> from collections import deque
> 
> class TaskScheduler(Node):
> def __init__ (self, maxlen: int = 10):
> self.task_queue = deque[tuple[float, callable]](maxlen=maxlen)
> 
> # Irrelevant code omitted ...
> 
> def schedule_task(self, ddl: float, task: callable):
> while len(self.task_queue) >= self.task_queue.maxlen:
> # No timeout (wait forever) - we need at least one task to be executed
> rclpy.spin_once(self)
> 
> # Insert the task into task queue in ascending order
> # Searching from back because the new task is likely to be the last
> for idx, (ts, _) in enumerate_back(self.task_queue):
> if ts < ddl:
> self.task_queue.insert(idx + 1, (ddl, task))
> break
> else:
> self.task_queue.appendleft((ddl, task))
> 
> ```

* * *

EDIT: inspired by an earlier [GitHub issue](https://github.com/python/cpython/issues/124178) shared by @Stefan2 (thanks!), I came up with another alternative approach of implementing this feature. IMO this will be the most ideal one among all proposed solutions:

```python
from typing import Iterable
import builtins

class enumerate(builtins.enumerate):

    # EDIT2: make `isinstance(..., collections.abc.Reversible)` happy.
    def __new__ (cls, obj, **kwargs):
        if hasattr(obj, " __len__") and hasattr(obj, " __reversed__"):
            return cls.reversible(obj, **kwargs)
        else:
            return builtins.enumerate(obj, **kwargs)

    class reversible(builtins.enumerate):
        def __init__ (self, iterable: Iterable, start: int=0):
            self.iterable = iterable
            self.start = start

        def __reversed__ (self):
            indexes = range(len(self.iterable) - 1, self.start - 1, -1)
            items = reversed(self.iterable)
            return zip(indexes, items)

```

Usage:

```python
# Works on all objects that supports len() and reversed()
reversed(enumerate(range(10)))
# Throws TypeError for objects that do not support these methods
# (e.g. a Generator)
reversed(enumerate((i for i in range(10)))) # TypeError

```

---

<div class="post-metadata">

**Author:** ![jamestwebber](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jamestwebber/32/12799_2.png) [@jamestwebber](https://discuss.python.org/u/jamestwebber)\
**Post date:** [December 16, 2024, 9:54pm UTC](https://discuss.python.org/t/backward-enumerator/74480/2 "2024-12-16T21:54:43Z")

</div>

Why not `((len(seq) - i, e) for i, e in enumerate(reversed(seq)))`?

edit: Only took me three tries 😂

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [December 16, 2024, 9:56pm UTC](https://discuss.python.org/t/backward-enumerator/74480/3 "2024-12-16T21:56:00Z")

</div>

> [@jamestwebber](#):
>
> Why not `(len(seq) - i, e for i,e in enumerate(reversed(seq)))`?

Because the actual index needs to be computed in each iteration 😂

---

<div class="post-metadata">

**Author:** ![jamestwebber](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jamestwebber/32/12799_2.png) [@jamestwebber](https://discuss.python.org/u/jamestwebber)\
**Post date:** [December 16, 2024, 9:56pm UTC](https://discuss.python.org/t/backward-enumerator/74480/4 "2024-12-16T21:56:27Z")

</div>

That’s true for enumerate too?

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [December 16, 2024, 9:57pm UTC](https://discuss.python.org/t/backward-enumerator/74480/5 "2024-12-16T21:57:28Z")

</div>

Well it should incur fewer numeric operation (for each iteration).

---

<div class="post-metadata">

**Author:** ![jamestwebber](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jamestwebber/32/12799_2.png) [@jamestwebber](https://discuss.python.org/u/jamestwebber)\
**Post date:** [December 16, 2024, 10:00pm UTC](https://discuss.python.org/t/backward-enumerator/74480/6 "2024-12-16T22:00:29Z")

</div>

Sure, a tiny bit. Does that micro-optimization justify adding this code, though? I’m skeptical.

Oh, another option to avoid this:

```python
zip(itertools.count(), reversed(seq))

```

Is that better?

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [December 16, 2024, 10:08pm UTC](https://discuss.python.org/t/backward-enumerator/74480/7 "2024-12-16T22:08:23Z")

</div>

I am running `timeit` tests for them. The point is to provide a standardized way of doing this so you do not have to hand write it every time and suspect its correctness when debugging.

The same claim that you made applies to `enumerate()` itself - everyone can implement their own in a few lines of python. But yet it exists and has been used widely.

---

<div class="post-metadata">

**Author:** ![Stefan2](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/stefan2/32/18492_2.png) [@Stefan2](https://discuss.python.org/u/Stefan2)\
**Post date:** [December 16, 2024, 10:15pm UTC](https://discuss.python.org/t/backward-enumerator/74480/8 "2024-12-16T22:15:39Z")

</div>

> [@jamestwebber](#):
>
> Why not `((len(seq) - i, e) for i, e in enumerate(reversed(seq)))`

Because all indices are 1 too high.

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [December 16, 2024, 10:16pm UTC](https://discuss.python.org/t/backward-enumerator/74480/9 "2024-12-16T22:16:06Z")

</div>

> [@jamestwebber](#):
>
> `zip(itertools.count(), reversed(seq))`

This is equivalent to my demo implementation after a tiny modification, but more efficient (maybe due to lack of `yield from` and lack of type checking?)

```python
zip(itertools.count(len(seq) - 1, -1), reversed(seq))

```

* * *

Test Code:

```python
def back_enumerate(seq):
    ... # See above

def comparison1(seq):
    return ((len(seq) - i, e) for i, e in enumerate(reversed(seq)))

def comparison2(seq):
    from itertools import count
    return zip(count(len(seq) - 1, -1), reversed(seq))

from timeit import timeit

seq = range(10000)

print("back_enumerate", timeit('list(back_enumerate(seq))', globals=globals(), number=1000))
print("comparison1 ", timeit('list(comparison1(seq))', globals=globals(), number=1000))
print("comparison2 ", timeit('list(comparison2(seq))', globals=globals(), number=1000))

```

Timeit results:

```python
~/playground $ python3 back_enumerate.py
back_enumerate 0.557957124983659
comparison1 0.8546551250037737
comparison2 0.44767408299958333 # Best

```

---

<div class="post-metadata">

**Author:** ![Stefan2](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/stefan2/32/18492_2.png) [@Stefan2](https://discuss.python.org/u/Stefan2)\
**Post date:** [December 16, 2024, 10:20pm UTC](https://discuss.python.org/t/backward-enumerator/74480/10 "2024-12-16T22:20:54Z")

</div>

> [@zhangyx](#):
>
> maybe due to lack of `yield from`

Yeah, that needlessly wraps the zip iterator in a generator, slowing it down. It’s an anti-pattern.

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [December 16, 2024, 10:29pm UTC](https://discuss.python.org/t/backward-enumerator/74480/11 "2024-12-16T22:29:27Z")

</div>

> [@Stefan2](#):
>
> Yeah, that needlessly wraps the zip iterator in a generator, slowing it down. It’s an anti-pattern.

I’ve modified the original implementation and added the SO solution for comparison. My modified original implementation now wins constantly.

> **Test code**
>
> ```python
> from typing import TypeVar, Sequence
> 
> T = TypeVar("T")
> 
> def back_enumerate(seq: Sequence[T]):
> 
> if not hasattr(seq, " __len__") or not hasattr(seq, " __getitem__"):
> raise TypeError(f"{seq} does not support sequencing")
> 
> indexes = range(len(seq) - 1, -1, -1)
> items = reversed(seq)
> return zip(indexes, items)
> 
> def no_type_check(seq: Sequence[T]):
> indexes = range(len(seq) - 1, -1, -1)
> items = reversed(seq)
> return zip(indexes, items)
> 
> def stack_overflow(seq: Sequence[T]):
> return reversed(list(enumerate(seq)))
> 
> def comparison1(seq: Sequence[T]):
> return ((len(seq) - i, e) for i, e in enumerate(reversed(seq)))
> 
> def comparison2(seq: Sequence[T]):
> from itertools import count
> return zip(count(len(seq) - 1, -1), reversed(seq))
> 
> from timeit import timeit
> 
> seq = range(10000)
> 
> def test(stmt: str):
> t = timeit(stmt, globals=globals(), number=10000)
> return f"{t:.8f}"
> 
> print("back_enumerate", test('list(back_enumerate(seq))'))
> print("no_type_check ", test('list(no_type_check(seq))'))
> print("stack_overflow", test('list(stack_overflow(seq))'))
> print("comparison1 ", test('list(comparison1(seq))'))
> print("comparison2 ", test('list(comparison2(seq))'))
> 
> ```

`timeit` Results:

```python
back_enumerate 4.11149596
no_type_check 4.09617529 # back_enumerate without type checking
stack_overflow 4.35428938
comparison1 8.54001383
comparison2 4.47707404

```

---

<div class="post-metadata">

**Author:** ![Stefan2](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/stefan2/32/18492_2.png) [@Stefan2](https://discuss.python.org/u/Stefan2)\
**Post date:** [December 16, 2024, 10:31pm UTC](https://discuss.python.org/t/backward-enumerator/74480/12 "2024-12-16T22:31:44Z")

</div>

Also note this:

> <https://github.com/python/cpython/issues/124178>
>
> \# Feature or enhancement
> 
> \### Proposal:
> 
> Currently, the \`reversed\` function does…n't support \`enumerate\` objects. This would be useful any time you want to traverse a list backwards whilst keeping indexes.
> 
> The implementation would ensure that \`enumerate\` objects support reversal only if the iterable argument also supports reversal and len. 
> 
> \`\`\`python
> lst = \["foo", "bar"\]
> for i, data in reversed(enumerate(lst)):
> print(i, data)
> \`\`\`
> 
> Currently, what I see being done is something like:
> \`\`\`python
> for i in range(len(lst)-1, -1, -1):
> data = lst\[i\]
> \`\`\`
> Which is less readable.
> 
> 
> 
> \### Has this already been discussed elsewhere?
> 
> This is a minor feature, which does not need previous discussion elsewhere
> 
> \### Links to previous discussion of this feature:
> 
> \_No response\_

---

<div class="post-metadata">

**Author:** ![Stefan2](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/stefan2/32/18492_2.png) [@Stefan2](https://discuss.python.org/u/Stefan2)\
**Post date:** [December 16, 2024, 10:35pm UTC](https://discuss.python.org/t/backward-enumerator/74480/13 "2024-12-16T22:35:06Z")

</div>

> [@zhangyx](#):
>
> Test code

Could you please include the needed imports?

---

<div class="post-metadata">

**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\
**Post date:** [December 16, 2024, 10:35pm UTC](https://discuss.python.org/t/backward-enumerator/74480/14 "2024-12-16T22:35:21Z")

</div>

> [@zhangyx](#):
>
> The point is to provide a standardized way of doing this so you do not have to hand write it every time and suspect its correctness when debugging.

I don’t think this is needed often enough to justify adding to the stdlib. It might be welcome in the `more_itertools` library, though, or maybe as a recipe in the itertools docs.

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [December 16, 2024, 10:38pm UTC](https://discuss.python.org/t/backward-enumerator/74480/15 "2024-12-16T22:38:07Z")

</div>

> [@Stefan2](#):
>
> Could you please include the needed imports?

Sry, fixed.

> [@Stefan2](#):
>
> [Support `enumerate` with `reversed` function · Issue #124178 · python/cpython · GitHub](https://github.com/python/cpython/issues/124178)

Interesting. How about exposing `enumerate(). __reversed__ ()` when the object being enumerated supports ` __len__ ` and ` __reversed__ `?

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [December 16, 2024, 11:08pm UTC](https://discuss.python.org/t/backward-enumerator/74480/16 "2024-12-16T23:08:06Z")

</div>

> [@zhangyx](#):
>
> Interesting. How about exposing `enumerate(). __reversed__ ()` when the object being enumerated supports ` __len__ ` and ` __reversed__ `?

Coming back with a demo implementation (I’ll update my OP and add this one):

```python
from typing import Iterable
import builtins

class enumerate(builtins.enumerate):
    def __init__ (self, iterable: Iterable, start: int=0):
        self.iterable = iterable
        self.start = start

    def __reversed__ (self):
        indexes = range(len(self.iterable) - 1, self.start - 1, -1)
        items = reversed(self.iterable) # Will throw TypeError if not supported
        return zip(indexes, items)

```

Usage:

```python
reversed(enumerate(range(100)))

```

> **Reproducible Test Code**
>
> ```python
> from typing import Iterable, Sequence
> import builtins
> 
> def back_enumerate(seq: Sequence):
> 
> if not hasattr(seq, " __len__") or not hasattr(seq, " __getitem__"):
> raise TypeError(f"{seq} does not support sequencing")
> 
> indexes = range(len(seq) - 1, -1, -1)
> items = reversed(seq)
> return zip(indexes, items)
> 
> def no_type_check(seq: Sequence):
> indexes = range(len(seq) - 1, -1, -1)
> items = reversed(seq)
> return zip(indexes, items)
> 
> def stack_overflow(seq: Sequence):
> return reversed(list(enumerate(seq)))
> 
> def comparison1(seq: Sequence):
> return ((len(seq) - i, e) for i, e in enumerate(reversed(seq)))
> 
> def comparison2(seq: Sequence):
> from itertools import count
> return zip(count(len(seq) - 1, -1), reversed(seq))
> 
> class enumerate(builtins.enumerate):
> def __init__ (self, iterable: Iterable, start: int=0):
> self.iterable = iterable
> self.start = start
> 
> def __reversed__ (self):
> indexes = range(len(self.iterable) - 1, self.start - 1, -1)
> items = reversed(self.iterable) # Will throw TypeError if not supported
> return zip(indexes, items)
> 
> from timeit import timeit
> 
> seq = range(10000)
> 
> def test(stmt: str):
> return timeit(stmt, globals=globals(), number=10000)
> 
> results = [
> ("back_enumerate", test('list(back_enumerate(seq))')),
> ("no_type_check ", test('list(no_type_check(seq))')),
> ("enumerate cls ", test('list(reversed(enumerate(seq)))')),
> ("stack_overflow", test('list(stack_overflow(seq))')),
> ("comparison1 ", test('list(comparison1(seq))')),
> ("comparison2 ", test('list(comparison2(seq))')),
> ]
> 
> baseline = max(tuple(zip(*results))[1])
> 
> for name, time_cost in results:
> pct = 100.0 * time_cost / baseline
> print(name, f"{time_cost:.8f}", f"({pct:.2f}% baseline)")
> 
> ```

`timeit` Test Results:

```python
back_enumerate 4.07600862 (47.78% baseline)
no_type_check 4.09555842 (48.01% baseline)
enumerate cls 4.14349108 (48.57% baseline)
stack_overflow 4.34685933 (50.95% baseline)
comparison1 8.53149825 (100.00% baseline)
comparison2 4.45040158 (52.16% baseline)

```

---

<div class="post-metadata">

**Author:** ![MegaIng](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/megaing/32/16162_2.png) [@MegaIng](https://discuss.python.org/u/MegaIng)\
**Post date:** [December 16, 2024, 11:09pm UTC](https://discuss.python.org/t/backward-enumerator/74480/17 "2024-12-16T23:09:12Z")

</div>

> [@zhangyx](#):
>
> > [@Stefan2](#):
> >
> > [Support `enumerate` with `reversed` function · Issue #124178 · python/cpython · GitHub](https://github.com/python/cpython/issues/124178)
> 
> Interesting. How about exposing `enumerate(). __reversed__ ()` when the object being enumerated supports ` __len__ ` and ` __reversed__ `?

That was literally the suggestion discussed in that github issue and rejected by the core devs in there. You would need to counter those arguments, and performance alone is not going to be enough.

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [December 16, 2024, 11:14pm UTC](https://discuss.python.org/t/backward-enumerator/74480/18 "2024-12-16T23:14:53Z")

</div>

> [@MegaIng](#):
>
> That was literally the suggestion discussed in that github issue and rejected by the core devs in there. You would need to counter those arguments, and performance alone is not going to be enough.

~~My latest proposed solution does not modify `reversed()` at all – instead it implements ` __reversed__ ` method in the `enumerate` class.~~

 

~~IMO this makes enough distinction for the proposal to be considered again.~~

---

<div class="post-metadata">

**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\
**Post date:** [December 16, 2024, 11:16pm UTC](https://discuss.python.org/t/backward-enumerator/74480/19 "2024-12-16T23:16:51Z")

</div>

> [@zhangyx](#):
>
> instead it implements ` __reversed__ ` method on `enumerate` class.

Did you read the linked issue all the way through? That approach was explicitly stated in the discussion, and the arguments against the feature were stated in full awareness of it.

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [December 16, 2024, 11:18pm UTC](https://discuss.python.org/t/backward-enumerator/74480/20 "2024-12-16T23:18:41Z")

</div>

My bad. I just noticed it. I jumped ahead of myself when reading it.

[Next page](https://discuss.python.org/t/backward-enumerator/74480.md?page=2)
