Support context-like behavior for loop iteration

Ideas
1

I found myself needing a contextual behavior inside the loop, for instance, a for loop going over a generator, which upon encountering an exception (anything but StopIteration), logs it and continues to the next item.
The problem is, that generators’ implementation doesn’t support any contextual behavior apart from __enter__ and __exit__ which only apply for the loop as a whole, and not for each iteration by itself.

I wonder how broad is this need, whether or not you folks think this is a problem worth addressing, and finally, if theres any simple solution i’m missing?
all the best, be gentle with me it’s my first ever post here :slight_smile:

2

This isn’t really to do with loops, but generators themselves: if a
generator raises an exception, it terminates, like any other function.
So there’s no “resume” for the for-loop to continue with; the
generator’s gone!

Possibly the simplest solution is that the generator doesn’t raise an
exception but yields it inline with whatever other values it in
producing:

 def gen():
     for ch in 'abc':
         try:
             assert ch != 'b'
         except AssertionError as e:
             yield e
         else:
             yield ch

This would yield:

 a
 AssertionError  # an AssertionError instance
 c

Using it:

 chars = []
 for ch in gen():
     if isinstance(ch, Exception):
         warning("got an exception: %s", ch)
     else:
         chars.append(ch)

or something like that.

Cheers,
Cameron Simpson cs@cskk.id.au

3

of course this solves this specific instance of the question, but not nearly covers the scope.
I think a more generic approach is to use a contextmanager inside the loop, wrapping the indented block, but what I was suggesting is to solve it in the generator, in a specifically dedicated functions resembling the enter and exit mechanisms but in finer granularity

1 Like
4

Can you write an example of what hypothetical syntax you’d like to have
work?

Cheers,
Cameron Simpson cs@cskk.id.au

5

Of course, in the for loop itself i expect no modifications, however, inside the generator class I would expect a function looking something like this:

class Generator():
    def next():
        ...

    def __enter_iteration__(self, item):
        # Here will be a hook for each running immediately after the "next" function invocation

    def __exit_iteration__(self, item):
        # Here will be a hook for each running at the end of the current iteration
6

Couldn’t you achieve the same thing with this?

def gen():
    for ch in "abc":
        #__enter_iteration__ logic
        try:
            yield ch
        finally:
            # __exit_iteration__ logic
1 Like
7

Isn’t this all logic that can be put in __next__ now, without any changes to the iterator protocol?

def __next__(self):
    # Stuff from __enter_iteration__
    ...
    rv = ...
    # Stuff from __exit_iteration__
    return rv

or

# No loop necessary; the try statement wraps the yield statement no matter
# where it occurs.
def generator_function():
    try:
        # __enter_iteration__
        ...
        yield rv
    finally:
        # __exit_iteration__

In the worst case scenario, you just don’t use a for loop (which “hides” the call to next that raises the exception), but use a while loop to iterate.

# Equivalent of
#
#    for v in your_iterable:
#        ...
itr = iter(your_iterable)  # Exception unlikely, but you can put this in a try statement
while True:
    ...
    try:
        v = next(itr)
    except StopIteration:
        break
    except ...:
        ...

This is all pretty broad; if there’s a use case you have in mind that doesn’t fit well into any of these three examples, I’d be interested to see it.

2 Likes
8

Yep that solves it :slight_smile: can’t think of a case not covered here.
Nice

1 Like
9

Could you expand a bit on how this solves the problem you describe in the OP?

Namely, blankly catching any exception but StopIteration, logging it and proceeding to the next item in the iterable.

I am interest in this.