Adding itertools.doublestarmap (or a better name)

Ideas
1

Using one of the latest Ruff linting rules (from refurb), I recently cleaned a codebase where I had code looking like this (dump contains formatting code):

# Before
"".join(dump(*t) for t in some_iterable)

# After
from itertools import starmap
"".join(starmap(dump, some_iterable))

After finishing this cleanup, I grepped my codebase and realized I had many other occurrences of a similar construct:

[SomeClass(**kw) for kw in some_iterable]

The use case I have is basically an alternate constructor for a class where I collect dicts that will be used as keyword parameters (I can expand on this but I want to keep this post short). This made me think that I could have used an hypothetic itertools.doublestarmap (or a better name):

list(doublestarmap(SomeClass, some_iterable))

Is this a need that has been expressed before?

1 Like
2

To be honest, I’ve always felt that having this in itertools is the wrong level of abstraction, or rather, the wrong way to split the task. I’d rather have adapters equivalent to

def starred(func):
    return lambda x: func(*x)

def doublestarred(func):
    return lambda x: func(**x)

which would instead live in functools, and which would then allow

''.join(map(starred(dump), some_iterable))
''.join(starred(dump)(t) for t in some_iterable)
''.join(map(doublestarred(dump), some_iterable))
''.join(doublestarred(dump)(t) for t in some_iterable)
# etc.

Although it might not be possible to get as good performance that way…

But yeah, from a functional programming perspective, that sort of adapter seems a lot more practical than custom-purpose tweaked re-implementations of the map logic.

1 Like
3

cc: @rhettinger (as the maintainer of both itertools and functools).

A

4

starmap() is more fundamental than map(). map() can be expressed via starmap() and zip():

def map(func, *iterables):
    return starmap(func, zip(*iterables))

But map() is more convenient, so it is a builtin.

None of them pass arguments by keyword. There is no keyword argument analogue of zip(). This looks like more niche application. You already have several ways to write this using existing syntax, so I think there is very little chance of introducing new functions in the stdlib.

3 Likes
5

Can you explain what problem this solves? In your first example I would prefer the version without starmap, as it’s shorter and doesn’t require the reader to know what that function does. Is there some reason I’m missing why it’s better to avoid the generator expression?

5 Likes
6

Has been suggested at more-itertools.

2 Likes
7

? I don’t follow.

def starmap(func, iterables):
    return map(func, *zip(*iterables))

seems just as simple to me. Only the function names swapped and one of the *s moved. The choice is largely arbitrary, and the fact that there is a choice is really just a consequence of having functions that accept multiple parameters with “(un)packing” being a coherent concept. (That’s typical of most languages, including the Lisp family if you squint a bit, but not e.g. the Haskell family as I understand it.)

8

You are right. map() and starmap() are two sides of one thing. But there is no other side of doublestarmap().

9

The unpacking of the zip is eager, so the performance and memory properties of the reverse are much worse

2 Likes
10

I find the two forms fairly equivalent (one could also debate whether map is more readable than the equivalent comprehension).
I started rewriting some code using starmap instead of a comprehension for the small performance benefit, where it matters for me.
If doublestarmap had been part of the standard library, I would probably do the same, for the same reason.

11

If there is a performance difference, we should fix it by making the generator expression faster, not by adding a second way to do the same thing.

I suspect on Python 3.12 the fastest way may be to use a list comprehension, thanks to PEP 709.

2 Likes