Exhausted iterator evaluate as False

Ideas
21

Yes! And I do use that a lot for simple checks. The for … else is really just a way to add more logic to the empty case:

for val in iterable:
    log_status('populated')
    break
else:
    cleanup_connection()
    log_status('empty')
    ...

I guess I could also do that with a 2-arg next as well though:

if next(iterable, None) is None:
    cleanup_connections()
    log_status('empty')
    ...
else:
    log_status('populated')

Honestly, I’ve just gotten used to the for else syntax, this is one of like two places that I’ve found a use for it. The other being checking for containment in an iterator:

for val in iterator:
    if val == target:
        break
else:
    print('target not found')

So for me (a very personal flavor preference) I just kinda associate a for else with doing something with a generator/iterator when I see it in my code.

In my usecases, dealing with database stuff, the break allows me to avoid using a sentinel, since None is pretty frequently present in the output stream.

3 Likes
22

Well worth being aware of this pattern! For an odd reason you may not have realized: it’s (or at least was) the fastest way to get the next (if any) result from an arbitrary iterable. No Python-level function calls, exception handling, or comparison with a sentinel value. Lean and mean.

Doesn’t score high on “obvious at first glance”, though :wink:

3 Likes
23

Oh absolutely, which is why I think of it more as a flavor preference. If you run into a random for else when you don’t see it anywhere else it’s confusing. It’s only really useful when you use it in a specific context repeatedly.

I use it most often for really quickly checking database cursor generators for values so I know my hot loops that are checking hundreds or thousands of tables aren’t wasting any time.

1 Like
24

I knew this, but I just got around to doing some simple perf tests using %timeit in a notebook and surprisingly the difference is under 10ns it seems. So either option is going to be incredibly fast, and you should really just use whatever feels most comfortable or you think is the most clear.

1 Like
25

This would be the primary, or at least most iconic, use of for-else.

2 Likes
26

In my mind, it makes sense to extend this use to checking for empty iterators too. It’s just that the condition in the case of an empty iterator is a no-op so the body of the loop has no condition.

As Tim said though, it’s not immediately apparent, and it can take a second to parse what’s happening since there aren’t any explicit conditions. And you need to know that for consumes the StopIteration before entering the loop body.

1 Like
27

Hopefully that was just a minimal example and in reality you’d use if target not in iterator:?

The for-break(-else) pattern is something I occasionally use, too, for clarity/simplicity or for micro-optimization.

1 Like
28

Yeah, the situations where this pattern are useful aren’t really easily representable in short snippets since in is usually enough for simple comparisons.

I only really use this when the match condition isn’t easily expressed in a lambda or as an ==/is.

Plus the situation of quickly checking for an empty iterator.

Here’s an example with a bit more complexity:

for area in areas:
    target = ...
    for point in points:
        if source == point['SHAPE@'].projectAs(map_ref):
            target = point
            break
    else:
        # Warn that no ideal projected point was found
        print('No projected matching point found, using base target')

Using the for … else allows for re-assignment of something before the break.

29

I would never write that. That consumes part of the iterator. I want x in y to be a pure function; using it in a way that isn’t lays a trap for the maintenance developer.

in falling back on __iter__ is a legacy wart. I don’t think we can get rid of it, backwards compatibility and all that, but I wish we could.

30

Well, I disagree. And I’ve made good use of it a few times.

31

It’s not a choice between .. in iterator and the verbose for/break/else, though. There’s also:

if any(x == target for x in iterator):
1 Like
32

The any pattern is one of my favorites since it short circuits the iteration.

You can do weird stuff like:

if any(s == prefix for s in gen):
    return next(gen)

Great for discarding garbage data in a generator if you have a known value that signals the start of what you want.

1 Like
33

? And what’s the difference with if target in iterator?

1 Like
34

Again, terrible minimal examples here. Remember, the added verbosity always makes it easier to do more complex comparisons:

5 in gen

any(n == 5 for n in gen)

for n in gen:
    if n == 5:
        break
else:
    ...

All do the same thing, however now I want to check if a number divisible by 5 is in the gen:

# 5 in gen (can't do it)

any(n%5 == 0 for n in gen)

for n in gen:
    if n%5 == 0:
        break
else:
    ...

Okay, now I want to also keep a running tally of the sum of all numbers seen until one that is divisible by 5:

# 5 in gen (can't do it)

# any(n == 5 for n in gen) (can't do it)

total = 0
for n in gen:
    if n%5 == 0:
        break
    total += n
else:
    ...

As you add verbosity, you do gain some level of flexibility. That being said, you should always shoot for the least verbose option when possible. It’s much easier to add complexity than take it away later.

1 Like
35

Or

total = sum(itertools.takewhile(lambda n: n % 5, gen))
2 Likes
36

All three variants use the iterator protocol, no difference there. The difference is whether there’s a for keyword making it explicit. With in, a reader could easily make the false assumption that the right-hand side object implements a side-effect free __contains__.

2 Likes