Don't forbid `map(nullary_func)`

Ideas
1

map(f, *iterables) calls f with elements from the iterables until one of them runs out of elements.

Except when there are zero iterables. That’s forbidden, for no apparent reason.

With the above general rule, map(f) would give an iterator infinitely calling f with zero arguments. Infinite because none of the zero iterables ever run out of elements.

So for example map(random) would be an infinite iterator of random numbers, map(time) would provide timestamps, and map(count) would provide counters. These are cases I’ve actually wanted every now and then. I can use starmap(random, repeat(())) instead, but all that extra code (also the import) is annoying.

Sadly that general rule is broken for the zero case, explicitly artificially banned by some “If map is called with fewer than two arguments, raise an error” code. That’s the only thing standing in the way. If I simply change it to “fewer than one”, then map behaves as desired. Doesn’t need extra code to support the behavior, the existing code naturally supports it already. And the only test that fails is one specifically testing the ban.

Why is it forbidden? I suspect it’s because that’s what Python 2 did, since map built a list there and it doesn’t make sense to try to build an infinite list. But that reason doesn’t apply to Python 3.

The bans that stand in the way:

In map_new:

    if (numargs < 2) {
        PyErr_SetString(PyExc_TypeError,
           "map() must have at least two arguments.");

In map_vectorcall:

    if (nargs < 2) {
        PyErr_SetString(PyExc_TypeError,
           "map() must have at least two arguments.");

If I change them to 1, then map works as desired and the only test that fails is one specifically testing that this causes the error:

  File "/workspaces/cpython/Lib/test/test_itertools.py", line 1372, in test_map
    self.assertRaises(TypeError, map, operator.neg)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: TypeError not raised by map

(Wanted to request this for a while, now a side note in an issue by @takuom prompted me to pursue it.)

3 Likes
2

iter(f, None) will do this - call f until it returns the given sentinel (if f might return None, use object()). I don’t think map(f) is a particularly obvious way to spell this[1], certainly no more obvious than iter(f, None), so I think the existing approach is fine.

  1. I’d be just as likely to say “there’s no arguments, so call f no times” ↩︎

9 Likes
3

map, just like zip, will limit itself to the shortest iterable, i.e. as soon as any one of them runs out of elements, map will stop. So if anything, it would feel more natural to me that passing no iterables results in an iterator that never yields anything. If you want to be pedantic about it, both interpretations are somewhat valid, since you have no iterables that can run out in your interpretation. But it seems just weird enough, that when semantics are this implementation dependent, it should just be disallowed, as it currently is, to avoid surprising edge cases, people didn’t consider, especially if that edge case is a potential infinite loop.

It’s far too easy to accidentally pass no iterables[1] and then just end up with an infinite loop. That’s far worse than an exception.

  1. e.g. when you use * to spread a collection of iterables, which may be empty under some circumstance ↩︎

6 Likes
4

No, iter(f, None) differs. It additionally compares with the sentinel value, and stops when an element equal to it occurs. That’s not the case in my three examples with sentinel None, but it can be in general. Plus I don’t want time spent on those comparisons.

5

Yes, like I said as well. Thus: No iterable, no limit.

That’s the opposite of what follows from what you said before.

6

Alright, then why should list(map(lambda: 1)) give you an infinite loop when list(zip()) gives you []?

1 Like
7

OK. It’s up to you whether you want to use iter(f, None).

But I’m -1 on map(f). Reasons:

  1. It’s unclear whether an infinite result or no results is the right behaviour, so it’s hard to teach and hard to document. Just because it’s clear to you which interpretation is right, we’ve now had two other people say that they can see it either way, so you can’t claim it’s “obvious”.
  2. Whether you like it or not, the behaviour you want can already be achieved with existing builtins.
  3. No other language I could find has this behaviour for map. I checked Haskell, Rust, Javascript, Go (which has no map function), and Perl, and I’m not aware of any other language that has this behaviour.
9 Likes
8

Because it’s useful. (Though not list(map(lambda: 1)) but rather map(lambda: 1), and really rather only with non-constant functions, otherwise I’d use itertools.repeat).

About zip() I came to the conclusion that it’s indeed better to break the rule, for three reasons:

  • zip is often used to “transpose a matrix”, like zip(*matrix), and transposing an empty matrix should better produce something empty rather than infinitely many empty tuples.
  • If zip() did produce empty tuples, that would be a single very narrow use case. Since tuples are immutable, and the empty tuple doesn’t even contain anything mutable, it has pretty much no value and you can’t even really do anything with it. This seems rather useless, unlike my three example cases, for which I’ve had use.
  • If you do want an infinite iterator yielding the empty tuple, repeat(()) (with itertools) is a better way. I’d prefer that even over map(tuple).
9
  1. I wonder: Are those two people also surprised that all([]) returns True?
  2. At the price of more code/work, which I’d like to avoid.
  3. I don’t know any other language that supports 0 <= i < n like Python does. Let’s ban that, too? No, Python doesn’t have to be like other languages.
1 Like
10

Do any of them even have a map function comparable to Python’s? Taking any number of iterables as arguments, and banning the zero case? I had a quick look and it seems half of them instead have map as a method of an iterable, and the other half only support one iterable as argument.

11

Semantically, the purpose of map is to “map” a function onto some inputs. If there is no input, it doesn’t meet the job description.

The semantics you describe, and your argument for them, make sense, but I don’t think it’s unambiguous that they’re the only correct semantics. Consider for example that min(()) similarly gives an exception, not any kind of representation of infinity.

Also works: iter(random, object()) (this is more foolproof than using None, although it could still break for a function that returns objects with a carefully designed __eq__).

I’m a third; and no, I’m not surprised by that, and don’t think it’s analogous.

7 Likes
12

No, not even object() is safe:

class C:
    def __eq__(*_):
        return True

def f():
    return C()

print(list(iter(f, object())))

That stops, prints []. (Attempt This Online!.)

And like I said, I don’t want time spent on those comparisons. And some comparisons might even cause errors, either the comparison itself or the truth value check of the comparison result. For example:

import numpy as np

def f():
    return np.arange(5)

print(list(iter(f, None)))

Outcome (Attempt This Online!):

Traceback (most recent call last):
  File "/ATO/code", line 6, in <module>
    print(list(iter(f, None)))
          ^^^^^^^^^^^^^^^^^^^
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Also, it’s simply misleading. Using iter(f, sentinel) makes the reader think that values from f might equal the sentinel value and that iteration might stop. Not good.

13

That’s different, as alsready evidenced by your “any kind of”. There is no universal “representation of infinity”. The min function really has no choice other than throw its hands in air.

And like I said, for map it’s the complete opposite: Not only could it do something useful, it already would if it weren’t for the ban. You can’t do that with min. If you remove its handling of that case, it would (attempt to) return the C null pointer.

14

Maybe you see it with sum([]) or math.prod([])? Should those throw exceptions? No, in all these cases (including map) the natural thing is to just not treat empty input specially. Let the straightforward implementation for the general case do its job and it works fine for the empty case as well.

15

Nor do I. I stopped, it’s you that carried on. I’ve explained why I disagree with this proposal. You don’t seem interested in considering anyone else’s viewpoint, just in trying to persuade people that you are right. I see no point in arguing further.

If you genuinely want this change to happen, you need at least one core developer to be in support of it. And in reality, you need more than just that one person, as I doubt any core developer would add a feature that has not received at least some level of support from the community. The way you’re managing this discussion at the moment suggests to me that you’re unlikely to get that support, and the proposal is dead in the water anyway.

11 Likes
16

No, it isn’t. If you give a function nothing to do, it should do nothing.

1 Like
17

More examples, including chain() and product(), which are maybe the most similar to map(f), as they also take an arbitrary number of iterables (just doesn’t unnaturally ban the empty case):

from itertools import *
from math import comb, perm

print(list(chain()))
print(list(chain.from_iterable([])))
print(list(product()))
print(list(combinations([], 0)))
print(list(permutations([])))
print(list(combinations_with_replacement([], 0)))
print(comb(0, 0))
print(perm(0))

Output (Attempt This Online!):

[]
[]
[()]
[()]
[()]
[()]
1
1

And I suspect none of them achieve this with some special handling for the empty case, likely they all just don’t look for that and do the general thing.

18

What are you talking about? If I weren’t considering your viewpoints, I wouldn’t be responding to them. I am.

19

Of course not. But I don’t think these cases are analogous, either. There isn’t a “zip identity” in the same way that 0 is an additive identity or 1 is a multiplicative identity.

1 Like
20

Wait, did you misunderstand what comparisons I was talking about? You seem to mean some comparisons we made here in discussing. I wasn’t talking about those. I was talking about the comparisons that iter(f, sentinel) does. The f() == sentinel comparisons. Their execution and the truth value checks of their results take time, and that’s the time I don’t want spent.