Yield comprehension

Related to @storchaka’s comment: yield is already an expression, and it has a value if the calling code uses send (otherwise None). When yield from <expr> is used, the result of send gets sent to the inner generator. In a yield <expr> for ... , where does send go?

Thank you for your interests!

Is there any place you’d like the sent value to go? I don’t have one so far.

I don’t have a request, I’m asking what you would propose here.

Yes, just like your code did. So I assumed your get_some_value() etc stores the gotten value somewhere. If you meant to use the value somehow, then mine depends on how, I’d need to see it. I might repeat a return or assignment or print, or if you meant to include all that inside an even bigger expression (the horror! :-), then I might refactor it into a function and use returns.

Thanks for the explanation. I’ve understood you thought I wanted a statement (or perhaps a block) equivalent to the one formed by my expression, even though I had written in the sentence (which you quoted) containing my code that it was for “an expression” and was “for use in any other expression or a statement”. (A statement can’t be a part of an expression.) I guess I wasn’t careful enough to ask for “an equivalent code” rather than an equivalent “expression”.

If you were going to write the expression itself (rather than the statement consisting of it), then I didn’t think that the expression you would write had to depend on how it might be used. If you could do this, that would be great.

Alternatively, if you still prefer writing different things depending on how my expression is used, then how about writing a block equivalent to the one consisting of the following statement?

return get_some_value()\
    if some_check()\
    else ((yield <expression> for object_ in iterable)
          or get_value())\
    if another_check()\
    else get_other_value()\
    if ...\
    else ...\

(I hope the layout respects the structure better this time.)

Could you please choose one of these?

Thanks @jamestwebber for the clarification. I don’t see any way to use a sent value meaningfully, so I wouldn’t like the value to affect any result. I would be happy with the same behaviour as that with yield from (<expr> for ...) in interactions with all objects around.

Simplify the expression using multiple statements, and wrap the code in a function which you call in the return statement.

I would absolutely insist that an expression this complex be simplified if I were reviewing the code.

Thank you for your interests!

You are welcome to imagine that my return statement was the whole code of a wrapping function.

Great, but how may it simplify the code as a whole? The structure at least, of the expression looks simple to me. (I understand how my earlier layout was not fitting the structure very well.)

Could you please actually define your wrapping function so we can compare it with mine? (It would be great if you could also give in the code any comments which you think will be important.)

That’s the first case I mentioned, i.e., I’d write return before each of the get_...() calls.

Well, first of all, I can’t make any sense of your example - it seems to weirdly mix simple values and yield from expressions (rewritten as your proposed yieldfor expression) in ways that make no sense to me. That’s a common problem when people quote “made up” examples. If you had a real world example of a complicated expression with yield from expressions somewhere in the middle, maybe your example would be easier to understand.

I could argue that this by itself kills your proposal - if all it does is enable the writing of incomprehensible expressions, then it’s not a good idea (and remember, code is intended to be read by people other than you, so what matters is not whether you can understand it but whether other people can).

But just to make my point even clearer, here’s an example of how something vaguely like this could be rewritten, to refactor the code into multiple statements. I’ve added a few comments as well, to demonstrate that using multiple statements make it easier to describe the structure of the calculation.

def ret_value():
    # Check something - if it's true we're done.
    if some_check():
        return get_some_value()

    # Try another possibility
    if another_check():
        for object_ in iterable:
            yield <expression>
        if not something():
            return get_value()

    return get_other_value()

return ret_value()

If you can’t see that this is significantly more comprehensible than your expression, then I don’t think we have much left to discuss. We’ll just have to disagree, and you’ll have to accept that people in general don’t have the same view as you over what constitutes a “simple” or “readable” expression…

1 Like

Yours isn’t equivalent, at least the if another_check() should be an elif, your if not something(): doesn’t appear in their original (their get_value() is done unconditionally at that point, since it’s (None or get_value())), and your final return is outside the function.

I know. I couldn’t understand the original, that’s my point. Feel free to do an equivalent rewrite if you do understand the original.

And yes, the final return is the replacement for the original return (stupidly long expression). I replaced the expression with a call to a multi-statement function.

It would be rather strange to provide the same behavior, I think.

Consider a function outer_generator() using either yield from <inner-generator> or the proposed yield obj for obj in <inner-generator>, where <inner-generator> is some internal stateful function or class within the primary function. In the first statement, code using outer_generator can use send(value) and the value will be passed through the yield from statement into <inner-generator>, which might need that value for some purpose.

In the second case, what should send(value) do? To me, intuitively the value should be the result of the yield obj expression in what appears to be a comprehension of some kind. But that would collect the results in outer_generator, not pass them along to <inner-generator>.

I think it would be confusing if the value was passed to the inner generator, as that wouldn’t happen in the very-similar-looking code

for obj in inner-generator:
    yield obj

One possible answer to this question is “don’t do that”. In other words, a “yield comprehension” or whatever this should be called simply doesn’t support send. But it feels like this is just adding complexity and nuance to a set of syntax that can already do all the things we need.

Oops, I missed that the last return called your function.

I had already posted a rewrite, like I said later it just needs return before each get_...() call.

Thank you for your thoughts.

I meant I would be happy if outer_generator() using

yield obj for obj in <inner-generator>

behaved just as that using

yield from (obj for obj in <inner-generator>)

In the latter case, the sent value only possibly goes to the generator (obj for obj in <inner-generator>) and not to <inner-generator>. I’m sorry I wasn’t clear.

I’m not sure if @pochmann’s comments on your code should count as an evidence against your claim (He didn’t understand that if and elif made no difference in the code, or that the final line of your code had to be return (yield from ret_value()) [added in revision: See my next post on this.].), but I actually don’t like my or test now for making you think about the value of the yield expression (to be None), just to recognize it’s going to be forgotten. I can make the expression be more explicit about it than with or, or just comment in the code, but I’m still not sure how significantly better that could be than wrapping this part in a function (as I already suggested before, except that I should have given it a better name to suggest its connection to get_value

). So I’m afraid we have lost reasons to continue this.

I’m sorry @pochmann that I’m not very much interested in this example any more.

I’ve deleted two previous posts, which was misleading.

Since my code had to be a part of a generator function, we should have agreed explicitly that we were writing the same part of a same generator function, so @pf_moore would have written return (yield from ret_value()) on the final line of his code. It was my mistake to assume such an agreement.

(In the deleted posts, I tried to examine the equivalence of return ret_value() and return (yield from ret_value()) under the assumption that the former was a part of an ordinary (i.e., non-generator) function. However, codes cannot generally be compared between a function and a generator function even if all other parts were identical (so we have no other yield expression in particular).)

No, this isn’t true. The value get passed into the inner generator. edit: at least, I think so…I had tested an example earlier but now I’m unsure

edit: my mistake, I was thinking of yield from inner-generator where the send value does get passed on. So there would presumably be a difference between yield from inner-generator and yield x from inner-generator which seems confusing.

Right, with return it doesn’t matter. But elif is still corresponding to your original, and correct in other cases like assigning to a variable or printing (where if isn’t equivalent to yours).