Passing anonymous functions as the last positional argument

Ideas
22

That, in my opinion, is a good thing. Given that we’ve been this long into Python’s history without the more general solution coming along, is it really right to deprive future Python of any sort of feature just because there MIGHT, maybe, be a more general one?

Only your proposal isn’t really an objection to the OP’s, it’s a completely different proposal and you’re trying to win support for it. It’s not going to improve the OP’s proposal, it’s either going to supplant it or be discarded. Thus it is independent.

@moderators Can this subthread be split out into its own thread please?

1 Like
23

Sure, I will post the proposal in a new thread in a bit, but I still believe that my proposal has sufficient overlapping goals with that of this thread that it is on topic for it to be also posted here as a possible alternative when they are directly at odds with each other.

24

This is perfectly possible with proposed syntax:

last_first_nm = re.sub( r'(\w+) (\w+)', string=first_last_nm) match:
    first_name, last_name = match.groups()
    return f'{last_name}, {first_name}'.upper()

If you pass first_last_nm as a kwarg, the repl automatically becomes the last positional argument. As mentioned in a previous post, we do not need to restrict it to positional-only. It can be the last unbounded positional arg.

1 Like
25

I thought your proposal requires the calllable as a positional-only argument, but in the code above you’re passing it as a positional-or-keyword argument. So the scope of your proposal has expanded, which is good, although it feels rather strange to see the third argument of re.sub passed as a keyword argument as I have never seen people use the keyword before.

In other words, you’re suggesting that all arguments after the callable be passed as keywords as a workaround, which is often not possible for the definitions of many existing library functions.

26

The problem with this approach is that it forces other parameters around it to follow some ambiguous indentation rule. Current python doesn’t even have a standard indentation convention for multiline arguments, not to mention enforcing it. The indentation rule never changed since the birth of python. What will be the implications if we introduce new indentation rules all of a sudden?

In your example, should we enforce the first and last parameter to align their indentation only if there is at least one multiline lambda passed as an argument? Or should we always enforce the indentation no matter if there is a multiline lambda?

If we choose the former, it creates inconsistencies that will lead to different indentation behaviors under different scenarios which will be a debugging nightmire, also, how should the parser even know if the first parameter needs to be aligned before it encounters the lambda? If we choose the latter, most of the existing codes will cease to work because of the newly added indentation rule. If we dig into more scenarios, complexity grows exponentially, and this almost becomes impossible if we still care any backward compatibility and don’t want to fork out a Python 4.

27

I also proposed this syntax, but I don’t think people will like it as it introduces a new : symbol

last_first_nm = re.sub( r'(\w+) (\w+)', :, first_last_nm) match:
    first_name, last_name = match.groups()
    return f'{last_name}, {first_name}'.upper()
1 Like
28

We can improve the indentation rule of my proposal by enforcing that the header of a multi-line lambda and/or def expression start on its own line, be followed by an indent, and end with a dedent, so this is not OK:

last_first_name = re.sub(r'(\w+) (\w+)', lambda match:
        first_name, last_name = match.groups()
        f'{last_name}, {first_name}'.upper()
    , first_last_name)

and this is not OK:

last_first_name = re.sub(r'(\w+) (\w+)',
    lambda match:
        first_name, last_name = match.groups()
        f'{last_name}, {first_name}'.upper()
        , first_last_name)

This is OK:

last_first_name = re.sub(r'(\w+) (\w+)',
    lambda match:
        first_name, last_name = match.groups()
        f'{last_name}, {first_name}'.upper()
    , first_last_name)

Please show me an example code if you think with the rules above the alternative proposal can still cause ambiguity.

29

Yeah that : definitely feels like a weird sentinel and reads like a slice notation, but the use of a sentinel is actually a decent idea. Maybe use @ instead as it is at least currently used for a somewhat relevant purpose of a decorator:

last_first_nm = re.sub( r'(\w+) (\w+)', @, first_last_nm) match:
    first_name, last_name = match.groups()
    return f'{last_name}, {first_name}'.upper()
1 Like
30

Interestingly, I initially considered @ as well, but the reason why I changed my mind to use : is because : marks the start of a new code block in Python. So, it is somewhat like a semantic cue to imply its correspondence with the : at the end of the line. Moreover, : frequently appears in Python and also appears in indexers like array = x[1, :, 0], so it doesn’t stand out too much to break the harmony. It’s just my own opinion though. I haven’t found any actual convincing reason to choose any particular symbol yet. Anyway, the gist is just to use a special sentinel to mark the desired position.

31

Well : inside brackets already means too many other distinctly different things to me (slice, key-value, type variable), all using : as a separator which it looks like, while @ looks like a marker aesthetically and already implies a function to follow.

But yeah, with a sentinel this proposal would then satisfy a vast majority of the use cases, with the rare exception being multiple anonymous functions as callable arguments to a call.

32

Then should we allow this?

last_first_name = re.sub(r'(\w+) (\w+)',
    lambda match:
        first_name, last_name = match.groups()
        f'{last_name}, {first_name}'.upper()
  , first_last_name)

And what if there are more parameters, do we do this?

last_first_name = re.sub(r'(\w+) (\w+)',
    lambda match:
        first_name, last_name = match.groups()
        f'{last_name}, {first_name}'.upper()
  , first_last_name,
    middle_name,
    nick_name
)

or this?

last_first_name = re.sub(r'(\w+) (\w+)',
    lambda match:
        first_name, last_name = match.groups()
        f'{last_name}, {first_name}'.upper()
    , first_last_name,
    middle_name,
    nick_name
)

or this?

last_first_name = re.sub(r'(\w+) (\w+)',
    lambda match:
        first_name, last_name = match.groups()
        f'{last_name}, {first_name}'.upper()
    , first_last_name,
      middle_name,
      nick_name
)

or this?

last_first_name = re.sub(r'(\w+) (\w+)',
    lambda match:
        first_name, last_name = match.groups()
        f'{last_name}, {first_name}'.upper()
    , first_last_name
    , middle_name
    , nick_name
)

or this?

last_first_name = re.sub(r'(\w+) (\w+)',
    lambda match:
        first_name, last_name = match.groups()
        f'{last_name}, {first_name}'.upper()
  , first_last_name
  , middle_name
  , nick_name
)

and what about this?

last_first_name = re.sub(r'(\w+) (\w+)',
    moms_name,
    dads_name,
    lambda match:
        first_name, last_name = match.groups()
        f'{last_name}, {first_name}'.upper()
    , lambda catch: # a coma before lambda? is this even allowed? 
          how_much_do_i_indent()
    , first_last_name
    , middle_name
    , nick_name
)

none of them look completely harmonic to me. It can easily break established conventions in organizations. It creates too many ways to align things, each of which somewhat makes sense, but it’s really hard to tell which one makes the most sense. it makes people hard to reach a consensus. It’s like there is always a wrong note, no matter what chord you are playing.

You should also consider how to construct a dict with multiline lambda under your rule:

{
    'a': lambda x: # illegal? since lambda doesn't start a new line?
             do()
             do_again()
}

{
    'a':
    lambda x:
        do()
        do_again()
    , 'b': # any better way?
    lambda x:
        do()
        do_again()
}

Anyway, I strongly suggest we discuss this in a different thread, as keeping this going will flood the original topic.

2 Likes
33

Hmm I was talking about disambiguation for the parser, but I now see your point being about reaching a reasonable consensus on a styling convention among humans. I do agree with you now.

I also didn’t think about the rather common use case (in other programming languages) of defining anonymous functions as values in a dict/hashmap, which is something your proposal skips. I think it’s a nice-to-have capability but am fine for the omission also.

1 Like
34

Very good point. With the sentinel of @ discussed above I was going to reply that we can do something like:

x = 13 if some_test() else fn(1,2, @) a, b:
    return a + b

and enforce that there can be at most one @ per statement.

However, I still don’t like the syntax where a, b are outside any brackets because if there are other tokens after a, b they would still be hard to read by a human:

x = fn(1,2, @) a, b if some_test() else 13:
    return a + b

So how about we instead enclose a, b in parentheses following a sentinel or the def keyword (and enforce that there can be at most one def per statement.)?

x = fn(1,2, def(a, b)) if some_test() else 13:
    return a + b

Moreover, if def(a,b) can be used in an arbitrary expression like the above then what happens when the def expression is in the header of a compound statement?

if fn(1,2, def(a, b)) if some_test() else 13:
    return a + b # does this belong to if or the anonymous function?

So there would have to be an additional rule that a def expression can only appear in a simple statement.

35

All of these proposals involving a sentinel and allowing anonymous functions in places other than the last argument are simply re-hashing discussions that have been covered many, many times before. If people want to have those discussions again, can they please be split out into a new thread? This thread should stay focused on the one genuinely new aspect of the OP’s proposal - limiting the supported cases to ones where there is no separation between the place where the multi-line function definition is used and where it’s placed.

I second the request by @Rosuav to split the subthread out. (But apparently I get a message saying I’m not allowed to @-link the moderators, for some reason, so I’ll have to hope they see the previous request and act on it…)

7 Likes
36

This lambda is passed as the fn parameter. Is it similar to a trailing lambda in Kotlin?

These blocks of code are functionally equivalent:

  1. Using the proposed syntax:
def foo(fn, /, x=1):
    fn(x, x*2, x*3)

foo(x=1) a, b, c:
    print(a, b, c)
  1. Using lambdas:
def foo(fn, /, x=1):
    fn(x, x * 2, x * 3)

foo(lambda a, b, c: print(a, b, c), x=1)
  1. Using functions:
def foo(fn, /, x=1):
    fn(x, x * 2, x * 3)

def fn(a, b, c):
    print(a, b, c)

foo(fn, x=1)

At first, I thought this entire block of code was another function definition:

foo(x=1) a, b, c:
    print(a, b, c)

It’s very confusing to use similar syntax for both defining and calling a function.

Note that the anonymous function is created at the point of the function call. If it becomes too large, creating it with each function call may not be desirable.

2 Likes
37

I don’t think the size of a function materially affects the time it takes to create one. But it’ll become unmanageable for the programmer long before that’s significant, anyhow.

1 Like
38

Worse is when the trailing function argument has zero parameters, and you pass local variables and/or keyword args to the function call:

foo(a, b, c=None):
    print('a and b are local variables, c is a kwarg passed to foo(), and this is a trailing function argument')

Is this what the author of this code intended, or did they just forget the def on their function definition?

This is where the proposed syntax starts to intersect with coding mistakes beginners might plausibly make. Perhaps all you need here is a SyntaxWarning to the effect of “did you forget a def?”. Or maybe the proposed syntax needs to be adjusted so that it looks a little bit less like a function definition.

Adding def before the parameter list might work:

foo(a, b, c=None) def:
    print('putting the def at the end is surely an uncommon mistake, right?')

# With parameters
foo(x=1) def a, b, c:
    print(a, b, c)

# Alternative with parentheses around the parameter list
foo(...) def ():
    ...

foo(...) def (a, b, c):
    ...
4 Likes
39

But the drawback of having a def like this is that it confuses function definition and invocation.

If I am a beginner, how can I tell apart which one is just a definition, and which one actually calls the function to do something with an anonymous block:

retry(n=3) def:
    do_somthing()

def retry(n=3):
    do_somthing()

It looks too confusing to me.

If we really want to use def, I agree with @blhsing 's proposal to put the def inside the parenthesis:

retry(def(), n=3):
    do_somthing()

implicitly passing def as the last positional argument:

retry(n=3):
    do_somthing()
3 Likes
40

Using def() would be the syntax for defining anonymous functions. If we allow the definition of anonymous functions using def(), similar to JavaScript’s function(), there is no need to pass it as the last positional parameter or impose other restrictions.

Using the hypothetical def():

def foo(fn):
    print(fn())
    
value = dict(
    result=foo(def():
        return 2)
)

Using the proposed syntax in this thread:

def foo(fn):
    print(fn())
    
value = dict(
    result=foo(def()):
        return 2
)

Both versions suffer from indentation uncertainty.

41

I agree, and that’s why we need some restrictions:

If you want to save the result to a dict, you can do this:

value = {}
value['result'] = foo(def()):
    return 2