If you have
l = [[1,2,3],[4]]
[*b for b in d]
it gives
File "<stdin>", line 1
[*b for b in d]
^
SyntaxError: iterable unpacking cannot be used in comprehension
Is there a specific reason for the inability?
3 Likes
Notice!! I’m not an expert actually I only started a few days ago, but I did some research and this is what I found
that is only available in calls and when using assignments
I linked the reference below, I hope it helps
1 Like
I don’t think that there is a specific reason, just the way the language has evolved. Comprehensions and iterable unpacking were added at different times.
See this thread here:
where supporting that is unpacking in comprehensions is discussed.
At this point, I expect all it will take is:
- someone to get a core developer to sponsor the idea;
- and then write a PEP;
- I expect that the Steering Council will probably accept it;
- then do the work.
If nobody wants to do the work, or write the PEP, it won’t happen.
I saw this, the accepted answer just says you can’t use iterable unpacking here
I came here because I thought it should be added to the language. But, is it really a worthwhile feature to have?
I can only think of using it to flatten 2d lists/tuples.
I.e. something like [[1,2,3],[10,87]] into [1,2,3,10,87]
Note that you can have multiple levels of iteration in a comprehension
a = [[1, 2, 3], [4]]
[c for b in a for c in b]
# [1, 2, 3, 4]
4 Likes
The following recursive function is designed to flatten a list of any depth. Admittedly, it is not the same as what you are trying to do, since it uses a test and a base case outside the list comprehension.
def flatten(a):
return [c for b in a for c in flatten(b)] if hasattr(a, '__iter__') else [a]
print(flatten([[1, [2, [3, 4, 5], 6], 7], [8, 9, [10, [[[[11]]]], 12]], 13]))
output:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]Unfortunately a general purpose flatten function is quite hard to get right.
Your version has at least five problems:
- Your test for iterable objects is not correct.
- Some iterable objects, like strings and bytes, may optionally need to be treated as if they were non-iterable.
- As it stands, strings lead to RecursionError.
- Recursive sequences also lead to RecursionError.
- Nor are infinite iterables.
More detail follows, but now, She Who Must Be Obeyed is calling.
Actually, the function was an attempt at flattening only lists, so a specific test for that type is needed.
Here’s a version 2:
def flatten(a):
return [c for b in a for c in flatten(b)] if isinstance(a, list) else [a]
There’s still a problem with this one, though. I don’t like that the following …
print(flatten(1))
… outputs …
[1]
EDIT:
Are we technically off-topic now, since the original post was specifically about iterable unpacking within a list comprehension?
The intent was to only flatten list objects. One of the reasons for this is that deciding what type the resulting objects should be when the original nested object is a mixture of iterable types would be problematic. There would also be the problem of how to handle dict objects. We would need to decide whether the values within embedded dict objects should be flattened. Also, in instances of any type that inherits from the list type should not be flattened. Maybe this would be a better test, assuming that a is the object to be flattened:
type(a) is type([])
Your revised flatten still fails on recursive lists, such as lists that contain themselves:
alist = [1, 2, 3]
alist.append(alist)
or lists which are mutually recursive (a contains b which contains a again).
alist = [1, 2, 3]
blist = [4, 5, 6, alist]
alist.append(blist)
Flattening lists is a deceptively tricky problem. You are not going to do it with a one-liner.
Yeah, it can be a challenge to deal with such things, including lists that have internal references to certain internals of themselves.
t = [[1, 2], [3, 4], [5, 6], [7, 8]]
def flatten(lists):
flat = []
for item in lists:
if type(item) is type([]):
flat += flatten(item)
else:
flat += [item]
return flat
t[2].append(t[1])
t[1].append(t[2])
print(t)
print(flatten(t))
Output:
[[1, 2], [3, 4, [5, 6, [...]]], [5, 6, [3, 4, [...]]], [7, 8]]
…
RecursionError: maximum recursion depth exceeded
Failure on flattening that list is probably a feature, not a bug. But a different failure mode might be better.
2 Likes
Those are good points, and the path ahead is left open as an exercise for the reader …
The wording of that error message could stand to be improved, as this restriction isn’t specific to comprehensions: you can’t write *b as a standalone expression in general, as the compiler doesn’t know what the type of the resulting sequence should be. Every time you use iterable unpacking, there has to be a surrounding construct that specifies what the iterable should be unpacked into.
Several other replies have covered ways of writing a flattening function that produces individual items from iterables in another iterable. If the goal is instead to unpack the comprehension into the list, then adding parentheses around the comprehension part will do the trick.
1 Like
Just my 2 cents,
I see this specific case a lot, one declarative way to do it which doesn’t seem to be mentioned yet. Like you I started with trying to use the splat operator which didn’t work.
l = sum(l, start=[]) # flatten a list of lists
Note that this is specialized to the case of a list of only flat lists like the specific case mentioned here, so it’s not as general as some of the other flattening methods mentioned here.
It looks like JS also doesn’t have this functionality for its spread operator, but rather a builtin method which is generalized to any depth.
l = l.flat(Infinity)