A puzzle about the StopIteration exception

Python Help
1

This category is for help and general discussion. This is a general discussion topic.

I found the interface to the StopIteration exception a bit puzzling. I think I’ve solved the puzzle now. Perhaps some of you would like to look at it, and try to solve it yourself. I’d post the solution in about a week or so.

The puzzle is also available at Python's StopIteration exception puzzled me. Here's why. What's going on here? BTW, I've solved the puzzle. · GitHub.

Here’s the puzzle. What’s going on here? Extra credit for ideas about removing this puzzle from Python.

>>> def doit(*args):
...
...     exc = StopIteration(*args)
...     return f"{exc.value=}, {exc.args=}"

>>> doit()
'exc.value=None, exc.args=()'

>>> doit(None)
'exc.value=None, exc.args=(None,)'

>>> doit(1)
'exc.value=1, exc.args=(1,)'

>>> doit(1, 2)
'exc.value=1, exc.args=(1, 2)'

>>> doit(1, 2, 3)
'exc.value=1, exc.args=(1, 2, 3)'

>>> doit((1, 2, 3))
'exc.value=(1, 2, 3), exc.args=((1, 2, 3),)'

>>> doit((1, 2), 3)
'exc.value=(1, 2), exc.args=((1, 2), 3)'
2

I don’t think this is puzzling at all. The args attributes is defined on BaseException and stores the positional arguments passed to the constructor as a tuple. The value attribute on StopIteration gives the first argument, which is the first item of said tuple.

3

In your example *args is obscuring what is going on. What you are observing is:

StopIteration().value         # None
StopIteration(None).value     # None
StopIteration(1,2).value      # 1
StopIteration((1,2,3)).value  # (1, 2, 3)

The first argument passed to StopIteration becomes the value, if omitted the value is None, which is vaguely similar to bare return or implicit return.

Now why StopIteration can take more than one argument is not immediately clear. The reasons are historical. The *args, **kwargs signature is simply inherited from Exception. Version 3.3 added the value attribute to access the first argument to facilitate the use in exception based control flow for coroutines.

4

You’re constructing an exception. I’m not sure why, since the normal way to construct StopIteration with a value is to return a value from a generator; but like most exceptions, it has a specific collection of named arguments (in this case just one, value), and also collects all its args into args.

If you don’t understand these things, check the documentation; specifically, since it’s the args attribute that seems to cause confusion here (the value is simply given a default if one isn’t passed), try this:

Note that StopIteration is far from unique in this. Other exception types also capture some specific arguments.

Why?

5

Hi Jonathan

Your statement, that exc.value is args[0], is almost correct. However, it does not explain:

If your statement was correct, we’d get an IndexError. Here’s a challenge that might help you see how puzzling I find StopIteration.

The challenge is to write a function aaa() that on all reasonable inputs gives the same output as doit(). You’ll then see that the behaviour of doit() is a bit peculiar.

6

Hi

This is good. Of course, the reasons for any past event are historical. Whereas present events arise from our own free will. (Which might not be as free as we suppose.)

Here’s a related question. What behaviour would an experienced but not advance programmer expect from StopIteration(1, 2). Would such a programmer expect StopIteration(1, 2, 3) to give a difference outcome.

7

Hi Chris

Warden’s statement here is important.

I think a common way to construction a non-coroutine awaitable is to explicitly constuct a StopIteration exception. It may even be the normal way in this context. I encountered this problem exactly when I was constructing my own non-coroutine awaitables.

8 
def aaa(*args):
    value = args[0] if args else None
    return f"{value=}, {args=}"

I would’ve expected TypeError: StopIteration expects exactly 1 positional argument, without the historical context and my experience with other built-in exceptions.

9

Nice. You’ve done it in one line, and without resorting to obscure tricks. How about:

def aaa(value=None, *args):
    return value

The gives exactly the right result for value. But it’s impossible to modify this to get the args as well.

Why? Well these two give the same value, but have different arguments.

aaa() # Use default argument.
aaa(None) # Explicitly give the same value.

Now there’s something a bit odd. For value we can use the default argument method. But not for args.

10

Here’s something that really puzzled me.I felt I had to know the answer, to a high degree of certainty.

A StopIteration exception has a value attribute and an args attribute. In the context of constructing an awaitable is the tail part of the message send to the event loop? Or just the shared part, the head args[0]?

There are at least three ways to answer this questions.

  1. Pythonic intuition.
  2. Documentation, perhaps including examples.
  3. Running test code.

Now we have:

>>> x_12345 = StopIteration(1, 2, 3, 4, 5)
>>> 
>>> x_12345
StopIteration(1, 2, 3, 4, 5)

This suggests to my Pythonic intuition that the tail 2, 3, 4, ,5 is in some sense important. And I’ve not found documentation clearly saying otherwise. So I’m looking for test code, as the true way to resolve my doubts about what the interpreter actually does. (Disclaimer. I’ve not read the PEPs. These might answer this question.)

11

They’re important precisely because you constructed the exception with them. Nobody else will know what to do with them. You can do this with any exception.

>>> Exception("spam", "ham", "eggs")
Exception('spam', 'ham', 'eggs')
12

The key is realizing that value is computed attribute, and not raw input.

OSError ate my eggs.

>>> OSError("spam", "ham", "eggs")
OSError('spam', 'ham')
13

Well this certainly surprised me. Well done. It even passed my ‘not gone but hidden’ test.

>>> triple = OSError('spam', 'ham', 'eggs')
>>> triple
OSError('spam', 'ham')
>>> triple.args
('spam', 'ham')

I’m very impressed.

14

It’s even funnier if you start with a number.

>>> type(OSError(1, 2, 3)) is OSError
False
15

To clarify:

>>> type(OSError(1, 2, 3))
<class 'PermissionError'>

I believe this is allowed. It’s in __new__ that the class is determined, and as I recall it can return something in a different class. I don’t remember what happens to __init__ in this situation.

What I don’t see, and it may be off-topic for this discussion, why OSError does this. However at present this is not something I need to know.

16

It downcasts to more specific subtypes if the error number is passed in.

The key common element to the whole discussion is that (for convenience in reporting extra information) the default signature of exception constructors is to accept and save arbitrary args. However, subclasses are free to apply more semantic significance than that and when they do, the details will be covered in the docs for the specific exception.

17

I agree that the default signature is to accept and save unamed args. I think it is one of several key facts for this discussion. For clarity, keyword arguments are not part of the default.

>>> Exception(1, 2, 3)
Exception(1, 2, 3)
>>> Exception(1, 2, 3, a=4, b=5)
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    Exception(1, 2, 3, a=4, b=5)
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^
TypeError: Exception() takes no keyword arguments

Indeed. And StopIteration has a special attribute value, in addition to the usual args, I’m not sure the semantics of the arguments to StopIteration are adequately covered in the docs. Certainly, I did experiments to learn the semantics.

18

Before making the initial post I did some research on the signature of the exceptions in the builtins module. More exactly, I use attributes as revealed by dir, omitting those that begin with __ as a proxy for the signature. These investigation as now added to the previous github URL Python's StopIteration exception puzzled me. Here's why. What's going on here? BTW, I've solved the puzzle. · GitHub .

Here’s what I found. The following exceptions have a unique signature (or more exactly the proxy I described above). One of them is StopIteration. All of the signatures have args. But StopIteration is the only one that has value. Should it have both values? And if so, what are their respective functions?

---
1  ('add_note', 'args', 'derive', 'exceptions', 'message', 'split', 'subgroup', 'with_traceback')
   val[:4]=['ExceptionGroup']
---
1  ('add_note', 'args', 'name', 'obj', 'with_traceback')
   val[:4]=['AttributeError']
---
1  ('add_note', 'args', 'value', 'with_traceback')
   val[:4]=['StopIteration']

Here are the two most common signatures. Notice that StopIteration has the most common (34 exception classes) signature, except for the unique addition of a value attribute.

34  ('add_note', 'args', 'with_traceback')
   val[:4]=['Exception', 'ArithmeticError', 'AssertionError', 'BufferError']
---
18  ('add_note', 'args', 'characters_written', 'errno', 'filename', 'filename2', 'strerror', 'with_traceback')
   val[:4]=['OSError', 'BlockingIOError', 'ChildProcessError', 'ConnectionError']

Here are the remaining signatures.

---
4  ('add_note', 'args', 'end_lineno', 'end_offset', 'filename', 'lineno', 'msg', 'offset', 'print_file_and_line', 'text', 'with_traceback')
   val[:4]=['SyntaxError', 'IndentationError', '_IncompleteInputError', 'TabError']
---
3  ('add_note', 'args', 'encoding', 'end', 'object', 'reason', 'start', 'with_traceback')
   val[:4]=['UnicodeDecodeError', 'UnicodeEncodeError', 'UnicodeTranslateError']
---
2  ('add_note', 'args', 'msg', 'name', 'name_from', 'path', 'with_traceback')
   val[:4]=['ImportError', 'ModuleNotFoundError']
---
2  ('add_note', 'args', 'name', 'with_traceback')
   val[:4]=['NameError', 'UnboundLocalError']
19

But you didn’t think to read the documentation?

20

Hi Chris

I have read the documentation, several times. Particularly that relating to StopIteration. That documentation says:

The exception object has a single attribute value, which is given as an argument when constructing the exception, and defaults to None.

But my initial example shows that there is also an args attribute. And dir also shows the args attribute. In my opinion warden below has given the most helpful response so far to my puzzle.

If it’s wrong to supply more than one positional argument to StopIteration, then I’d prefer that the documentation said so. And also that the Python interpreter raise a TypeError.

In case you think the single argument is implied by the documentation, consider the following example, in which what might be called args is silently converted into a single argument, namely a tuple.

>>> d = dict()

>>> d[1]
Traceback (most recent call last):
    d[1]
KeyError: 1

>>> d[1, 2]
Traceback (most recent call last):
    d[1, 2]
KeyError: (1, 2)
>>>