What is `yield from`?

Python Help
1

Consider the following code:

def numberGenerator(number_range: int):
	for i in range(number_range+1):
		yield i


def loudNumberGenerator(number_range: int):
	normal_number_generator = numberGenerator(number_range=number_range)
	while True:
		print('I am going to yield something!')
		yield from normal_number_generator


zero_to_five = loudNumberGenerator(5)

for _ in range(6):
	print(next(zero_to_five))

Expected:

I am going to yield something!
0
I am going to yield something!
1
I am going to yield something!
2
I am going to yield something!
3
I am going to yield something!
4
I am going to yield something!
5

Got:

I am going to yield something!
0
1
2
3
4
5

Why did the string I am going to yield something! printed only once?
What is yield from in simple words?

The Python Tutorial does not talk about it, the PEP is too recondite for me :frowning: , and the Python Language Reference says:

The values produced by iterating that iterable are passed directly to the caller of the current generator’s methods. Any values passed in with send() and any exceptions passed in with throw() are passed to the underlying iterator if it has the appropriate methods.

which leads to what I expected.

2

Exactly what it sounds like: it yields the values from the other source, one at a time, until it runs out. Every time there is a request for the next element from this generator, if there is still something in the other source, it will use that. Otherwise, the logic proceeds.

The loop

while True:
    print('I am going to yield something!')
    yield from normal_number_generator

is effectively equivalent to:

while True:
    print('I am going to yield something!')
    for i in normal_number_generator:
        yield i

Because yield from makes all of the values from normal_number_generator get yielded (and thus printed in the main loop), but then the while True: loop does not resume, because there are no more next calls on that generator - so it’s “stuck” on the yield from line. If you try next(zero_to_ten) again after the code you showed, it will get in an infinite loop, because the while True: loop can continue, but yield from doesn’t have any more elements to offer, so it tries forever looking for the next actual value to yield.

Why would it? It does its own iteration for “that iterable”, which has to finish before your while True: loop can proceed.

1 Like
3

Thanks for your quick answer!

I thought yield from obj equals to yield next(obj) :man_facepalming:

4

Close! It’s that, but keep going until it stops, and also, if you get sent any values, pass those along too. It’s fairly complicated to describe in full detail, but the short answer is: It’s the generator’s equivalent of calling another function. That is, it allows you to refactor a generator without changing its behaviour.

2 Likes
5

One more thing, is it possible to use yield from with .send()? How? If yes, what are the use cases?

6

It doesn’t appear so:

>>> def by_yield():
...     x = yield 1
...     y = yield 2
...     z = yield 3
...     print(x, y, z)
... 
>>> y = by_yield()
>>> y.send(None)
1
>>> y.send('a')
2
>>> y.send('b')
3
>>> y.send('c')
a b c
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration
>>> def by_yield_from():
...     xyz = yield from (1, 2, 3)
...     print(xyz)
... 
>>> y = by_yield_from()
>>> y.send(None)
1
>>> y.send('a')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in by_yield_from
AttributeError: 'tuple_iterator' object has no attribute 'send'
7

That’s because you yield from a tuple. Use a generator instead, for example yield from by_yield().

2 Likes
8

I tried:

def by_yield():
	x = yield 1
	y = yield 2
	z = yield 3
	print(x, y, z)

	
def by_yield_from():
	xyz = yield from by_yield()
	print(f"{xyz=}")


g = by_yield_from()
next(g)
g.send('A')
g.send('B')
g.send('C')

Got:

A B C
xyz=None
Traceback (most recent call last):
  File "test.py", line 22, in <module>
    g.send('C')
StopIteration

Hmmm… Seems like xyz won’t be changed by g.send(). So I guess anything send()-ed to g will just go to the ‘inner’ generator yield from directly, and hence xyz will always be None?

9

Wait, I was wrong, add a return statement in by_yield(),

def by_yield():
	x = yield 1
	y = yield 2
	z = yield 3
	print(x, y, z)
	return 99

will get

A B C
xyz=99

According to the PEP:

Furthermore, when the iterator is another generator, the subgenerator is allowed to execute a return statement with a value, and that value becomes the value of the yield from expression.

1 Like
10

Well, yes. That’s the point of yield from - it’s a syntax for delegating to a subgenetrator. The send() goes to the current yield, not to the yield from itself.

Incidentally, it’s the presence of features like send() that make yield from so valuable. It’s more than just for thing in subgen: yield thing specifically BECAUSE it passes on any send/throw to the other. Thus it makes the perfect tool for this sort of delegation… or for some sort of, I dunno, async/await implementation done using generators :slight_smile: And in fact, the PEP that introduces the async and await keywords specifically references yield from as it was a way to create coroutines without explicit support for asynchronous functions.

2 Likes