Local assignments within comprehensions

I find myself occasionally using constructs like this:

foo = [
    (expression that uses `computed_value` multiple times)
    for entity in iterable
    for computed_value in (value_producer(entity),)
]

i.e. sometimes I need a value computed for each element of the iteration, to then use that computed value multiple times, and I want to avoid repeating the computation, so I stick in a single-element nested iteration just to inject that computed value as context inside the comprehension (this applies to all kinds of comprehensions and generators too), and sometimes I need more than one computed value like that.

I’m not happy with it — it’s kludgy, easy to miss, easy to overlook. It feels like this isn’t a very unlikely scenario, and perhaps could use some dedicated support from the language? Something like this perhaps:

foo = [
    (expression that uses `computed_value`, `another_computed_value`, etc as needed)
    for entity in iterable
    with computed_value = value_producer(entity), another_computed_value = some_other_producer(entity), ...
]

The assignment operator (:=) might help, depending on your use case. A simple contrived example:

# doubling each square by assigning j**2 to i and then adding i
>>> [(i := j ** 2) + i for j in range(5)]
[0, 2, 8, 18, 32]
1 Like

I’ve thought about it, but I find this actually even clumsier to read than my approach. It also depends on actually being able to use that assignment expression; here’s an example that will not fit the assignment pattern:

foo = [
    item: {
        'old_value': old_value,
        'new_value': new_value,
    }
    for item in iterable
    for old_value, new_value in update_something(item)
]

Do you have a concrete example? It’s hard to judge what’s clumsy in pseudo-code.

Of course, an even simpler (and clearer) workaround is to store your computed value in a variable on a separate line, and then use it in the comprehension.

Why not just use a normal loop? If a comprehension is too clumsy, that’s a good sign you’d be better breaking it up into multiple statements.

3 Likes

Can’t store it in a variable outside the comprehension if it changes with every item being iterated over

Oh, and if you don’t use entity itself in the expression, there’s always plain ol’ map:

[(complicated expression)... for computed_value in map(value_producer, iterable)]

Well maybe you can, depending on the use case, but I don’t know what the use case is so I’m just making guesses in the dark.

Moving this to the Help category as it appears to be a question about how to refactor code or structure it to be more readable.

1 Like

Quick note: it’s better to post a new reply rather than edit an old post, people are likely to miss the edit.

I don’t think I understand how this example relates to your original problem. What are you calculating and using more than once?

My original problem is wanting to provide additional context for each iteration in comprehensions and generators for various reasons. One reason would be needing to use the same computed value more than once (the original example), another reason would be this example, where a complex value needs to be broken down and restructured in the comprehension expression.

The point isn’t “help me figure out how to refactor this abstract need away”. If being able to refactor code in a different way was sufficient reason to dismiss a language feature, then probably half the language might be arguably unnecessary. You could argue that comprehensions themselves are unnecessary, because you could build the data structure in a regular loop just the same. I’m more interested in a more conceptual discussion about the idea of having control over the local context inside comprehensions like this.

1 Like

Sure. But justifying an change to the language requires some real-world examples, not just “I need this for various reasons and I will not elaborate.”

My current concept of this idea is “I have never really needed this, I’m not sure why I would, and I already can do it with the assignment operator.” So I was honestly trying to get some more context for a situation when this would be a meaningful improvement on what is already possible.

1 Like

If you don’t need entity in the expression, the usual pattern is to map iterable to value_producer as computed_value:

foo = [
    (expression that uses `computed_value` multiple times)
    for computed_value in map(value_producer, iterable)
]

Okay, since it’s apparently not obvious that I’m not talking about extremely trivial cases that obviously do not need anything as complex as adding new state in the middle of a comprehension, here’s a bit more contrived example that demonstrates both scenarios that I’ve already talked about — using the computed value multiple times in the assignment expression, and deconstructing its values into different parts of the assignment expression, while also using the original entity.

{
    foo_entity['id']: {
        'foo_detail': foo_entity['detail'],
        'bar_detail': bar_entity.detail,
        'foo_bar_mapping': {
            k: v
            for k, v in zip(foo_entity['mapped_keys'], bar_entity.mapped_values)
        }
    }
    for foo_entity in json.loads(serialised_entities)
    for bar_entity in [get_bar_entity(foo_entity['bar_id'])]
}

It seems that for some reason everybody assumes I have a concrete problem that I’m trying to solve. I’ve had hundreds of concrete problems of this kind that I’ve already solved and I’m not talking about any of them and I’m not asking for help with refactoring any specific case. I’m talking about whether it might be helpful to have a language feature to make scenarios like this easier to handle in a clearer way.

Yes, it might be possible to refactor any logic to avoid scenarios like this, but I find it helpful when the implementation matches the intention, as opposed to changing the logic to match the available mechanics. I believe this obscures the intention by bending it around what feels like a missing feature.

In a sense, in CPython, that’s already the case. Your “singleton iteration” is implemented as the simple assignment that you intend. You’ll also find a link to a prior discussion about new syntax for it there.

This is exactly what I was aiming for with this post. I tried to find prior discussions about similar ideas, but failed. Thanks, Stefan :+1: