I am remaking my original question as a suggestion here since it seems like there isn’t any immediately obvious reason against it.
itertools.tee “copies” an iterator. Part of the iterator protocol is to throw a StopIteration exception to indicate its end. StopIteration exceptions can contain information in their value field (such as the return value of generator functions), or a thrown exception might even be a subclass of StopIteration containing even more information. However when iterators created by itertools.tee end, they do not consider the exception thrown by the original iterable, instead they create new StopIteration instances with empty value. This essentially makes tee useless to anyone who relies on the information contained in the StopIteration of their iterables.
I propose that itertools.tee should store whatever exception object is thrown by the original iterable, and all child iterables should throw this instance when they end, thus presevering whatever information might have been present.
I will admit from the start that I don’t have an actual use-case for this (I didn’t even know about the value field of StopIteration until a few days ago), but it seems to me like a small oversight, in that tee isnt making as good a “copy” of iterables as it could be.
More than a decade ago, I proposed to make itertools iterators more aware of StopIteration – preserve its value if possibly. The main showstopper was the same – the lack of actual use-cases for this. Nothing changed for past years.
I’ve read that OpenBSD is very secure because they fix problems when they find them – they don’t wait for an exploit to occur.
We can fix this now and make life easier for those that eventually need it, or wait until they do, then make them wait for the next release, and possibly longer while the releases they actually use die off.
I believe the issue is the “s” here. You cannot throw the same exception twice from two different places. The stack traces would be concatenated or even interleaved, and any notes added in one route would appear in all other routes too.
You would also need to hold onto the initial exception, which causes unexpected memory leaks until all iterators are consumed.
IMO it’s unnecessary to retain the instance, just the value from it. Yes, that would mean losing the potential for a subclass of StopIteration to be thrown, but that seems a pretty narrow use-case and one that’s particularly tricky to handle (how do you duplicate arbitrary objects - for example, what if this hypothetical subclass had a reference to an open file?). But retaining the value and constructing a new StopIteration wouldn’t be too hard.
Yeah, I mentioned in my original question but forgot here, that if there are problems with throwing the same exception multiple times, then just retaining the value would be basically just as good. After all that value could be an object itself containing whatever information is needed.
Python 3.13 has actually just gained the ability to return the value of StopIteration from the close() method of a generator. It’s not exactly a common use case, but when you use generators as coroutines (in the original sense), accessing the return value of a generator is very useful
If the generator function is running, the return value does not exist yet, close() has nothing to return. If the generator function is finished, it should keep a reference to the return value for the following close(). It prolonges the lifetime of objects and can create unneded loops.
I like the idea, but note that returning a value via StopIteration is kind of a broken pattern:
it’s not supported by type checking
you cannot do it with async iterators
I’m not sure if this is really an intended pattern any more or just a language wart that it’s too late to remove but we don’t want to encourage…
Values in StopIteration are kind of an implementation detail of the way return values from coroutines are handled. It shouldn’t be an issue unless you’re trying to use itertools functions to stitch coroutines together. But fully supporting that would require more than just preserving StopIteration values – you would also need to handle send() and throw().
If the generator function is running, the return value does not exist yet, close() has nothing to return. If the generator function is finished, it should keep a reference to the return value for the following close(). It prolonges the lifetime of objects and can create unneded loops.
This sounds like a bad idea from the past coming back again. When the details of “yield from” were being hashed out, it was suggested that if you call next() on an exhausted generator it should raise a StopIteration containing the same return value again. That was decided against for these very reasons.
| 
|