Add built in flatmap function to functools

Ideas
1

Python already provides built in functions for functional programming paradigms like map, filter, and accumulate (using functools reduce()). However, an equally useful function flatmap is not directly offered. Of course, there are ways for a python developer to implement this on their own, as shown in here and here. But I think offering this function as a built in operator would be very helpful for new developers and help developers make more concise, elegant code.

Specifically, flatmap would take the following arguments:

flatmap([function], [list or iterable])

It would output the result of applying the function to each item in the list or iterable and then flattening the entire list.

Ex:
flatmap(lambda x: [x, x, x], [1, 2, 3]) = [1, 1, 1, 2, 2, 2, 3, 3, 3]

2 Likes
2

Is this just flatten(map(func, values))?

First, we would need to decide what the semantics of flatten are. There is a reason why Python still does not have a flatten function after 30+ years.

3

Fairly certain sum(map(func, values), []) would do the trick

In [1]: sum(map(lambda x: [x, x, x], [1, 2, 3]), [])
Out[1]: [1, 1, 1, 2, 2, 2, 3, 3, 3]

edit: I see this as solution in OP 2nd linkā€¦ops

5

I disagree. flatten requires learning and memorising a new word, one which newcomers likely havenā€™t encountered yet. List comprehension with nested loops can be surmised from basic examples of Pythonā€™s syntax.

3 Likes
7 
flatten = itertools.chain.from_iterable
8 Likes
8

Forgot about that one.

9

Soā€¦

flatmap = lambda func, iterable: itertools.chain.from_iterable(map(func, iterable))

Can we have this added to itertools Recipes for incubation to eventually include in itertools as a first-class function?

10
11

Good to know. Cannot say I like itā€¦makes me wish for a __sum__ so you donā€™t have to think about these things. I know itā€™s a pitfall for users of numpy too, would be nice if sum could just delegate down to numpy.sum but thatā€™s a different discussion

12

It is not clear on which type the __sum__ method would be looked up if the list is [a, b] where type(a) != type(b).

13

This is why we have operator.__iconcat__.

14

The problem is that a flatmap function is somewhat nieche. I have not had any need for a flat map function, and if I did I would just write my own. Unlike some communities we can and do write functions that are some what trivial, *Cough Cough* is-odd - npm *Cough cough*.

15

See https://github.com/python/cpython/blob/80b9b3a51757ebb1e3547afc349a229706eadfde/Python/bltinmodule.c#L2659

1 Like
16

So, ā€¦ because it is ā€œtrivialā€, and I can write it in one line, it need not be included??

10.12 Batteries Included :chains:

Python has a ā€œbatteries includedā€ philosophy. This is best seen through the sophisticated and robust capabilities of its larger packages. [ā€¦snip]

Letā€™s look at itertools for a second:

def chain(*iterables):
    return (element for it in iterables for element in it)

chain_from_iterable = lambda iterables: (element for it in iterables for element in it)

def compress(data, selectors):
    return (d for d, s in zip(data, selectors) if s)

def starmap(function, iterable):
    return (function(*args) for args in iterable)

Those are one-liners which were turned into precompile functions bundled in the itertools module. Why bother? We could easily write them ourselves.

The following are ā€œone-linersā€ from the itertools recipes section:

def take(n, iterable):
    return list(islice(iterable, n))

def prepend(value, iterator):
    return chain([value], iterator)

def tabulate(function, start=0):
    return map(function, count(start))

def tail(n, iterable):
    return iter(collections.deque(iterable, maxlen=n))

def nth(iterable, n, default=None):
    return next(islice(iterable, n, None), default)

def quantify(iterable, pred=bool):
    return sum(map(pred, iterable))

def ncycles(iterable, n):
    return chain.from_iterable(repeat(tuple(iterable), n))

def sumprod(vec1, vec2):
    return sum(starmap(operator.mul, zip(vec1, vec2, strict=True)))

def sum_of_squares(it):
    return sumprod(*tee(it))

def transpose(it):
    return zip(*it, strict=True)

def matmul(m1, m2):
    return batched(starmap(sumprod, product(m1, transpose(m2))), len(m2[0]))

def flatten(list_of_lists):
    return chain.from_iterable(list_of_lists)

def triplewise(iterable):
    return ((a, b, c) for (a, _), (b, c) in pairwise(pairwise(iterable)))

def unique_justseen(iterable, key=None):
    return map(next, map(operator.itemgetter(1), groupby(iterable, key)))

def first_true(iterable, default=False, pred=None):
    return next(filter(pred, iterable), default)

All of these are simple enough we can write them ourselves, yet theyā€™ve been added to the recipe area and into more-itertools. If ā€œI have not had any need for a flat map functionā€ is the discriminator here, how did tabulate get into the recipes? Iā€™ve never had any need for it. So I think ā€œnicheā€ and ā€œtrivialā€ are not entry barriers here.

1 Like
17

On the other hand (recently in the PSF blog):

Overall, there was agreement that the original motivations for a large, ā€œbatteries-includedā€ standard library no longer held up to scrutiny. ā€œIn the good old days,ā€ Ned Deily reminisced, ā€œWe said ā€˜batteries-includedā€™ because we didnā€™t have a good story for third-party installation.ā€ But in 2023, installing third-party packages from PyPI is much easier.

Iā€™m guessing Raymond found it educational (from the recipes introduction):

The primary purpose of the itertools recipes is educational.

18

A secondary purpose of the recipes is to serve as an incubator.

The documentation also has a Functional Programming HOWTO, indicating Python supports it (functional programming, that is). Including flatmap could be considered educational. At the very least, if it were included in itertools recipes, or in the Function Progamming HOWTO, typing flatmap into the documentation search would cause it to pop up!

1 Like