Weird behaviour

I had issue with some production code and ended up clearing it, but I am still puzzled.

Would you expect this:

l = [ whatever ]
print([ f(x) for x in l])

and

g = ( same_whatever )
print([ f(x) for x in g])

To produce the same output ? Equal objects. “whatever” is NOT causing ANY side effect, pure functional.

YBM.

g = ( same_whatever ) assigns same_whatever to g. The parentheses are superfluous.

A 1-item tuple literal must include a trailing comma:

g = (same_whatever,)
2 Likes

Print out what l and g are, without the comprehension. I’m guessing that it’s a list containing one string in the first case, and just the string in the second.

>>> l = [ (lambda: i) for i in (1, 2, 3) ]
>>> [ f() for f in l ]
[3, 3, 3]
>>> g = ( (lambda: i) for i in (1, 2, 3) )
>>> [ f() for f in g ]
[1, 2, 3]
1 Like

So yes, you are performing side effects, specifically your are capturing variables.

1 Like

I’m not, definitely not “performing” side effects.

You should’ve posted this snippet in the first place since it’s fundamentally different from the code in the OP.

[ (lambda: i) for i in (1, 2, 3) ] is a list comprehension that creates a list of functions returning i, which isn’t evaluated until [ f() for f in l ], at which time i is already 3.

( (lambda: i) for i in (1, 2, 3) ) is a generator expression that doesn’t output lambda: i until [ f() for f in g ] iterates over g, so lambda: i gets created and evaluated while i gets incremented.

2 Likes

If I had asked you first to predict the results, would you have got them right ?

I should have said that both were comprehensions though.

Yes, I would have. This behavior may indeed be confusing to newcomers but would become a second nature once you understand the distinctions between a comprehension and a generator.

It looks to me as a generator defined by a comprehension.
the “distinction” you point out does not exist.
I’m not a newcomer.

A generator expression shares a similar syntax with a comprehension, but is not at all a comprehension. It defines a generator that does not evaluate the expression within until its __next__ method is called.

Please refer to the documentation.

1 Like

IMHO, no difference : for any kind of iterable (lazy — generator — or not — list): for is in both cases calling next on the object it is iterating through.

Sure, both are comprehensions :slight_smile:

No, the __next__ method I’m talking about is of the generator object that the generator expression creates, not the __next__ method of the object that is being iterated in the expression.

A generator expression is not a comprehension. Again, please read the documentation first.

l = [ (lambda: i) for i in (1, 2, 3) ] is a shorthand for:

l = []
for i in (1, 2, 3):
    l.append(lambda: i)

while g = ( (lambda: i) for i in (1, 2, 3) ) is a shorthand for:

def g():
    for i in (1, 2, 3):
        yield lambda: i

I have no more words if you still believe the two are the same.

Good point!
My point is that BOTH should be the same IMHO.

Why should they be the same when they represent two fundamentally different constructs and concepts?

1 Like

Because imho a list comprension and a generator comprehension are expressing semantically (if you have no side effects) the same data structure, just delaying evaluation in the former case. They should return the same data.

I may be wrong, but this is the way I’m used to in order to optimize production code.

1 Like

But whether or not evaluation is delayed is the crucial difference in the outcome because in the case of your list comprehension, the lambda functions are called only after i has finished iterating through (1, 2, 3), ending up as 3, while in the case of your generator expression, the lambda functions are called while i is iterating over (1, 2, 3).

2 Likes