Can `->` make a nicer lambda syntax?

Ideas
1

Reading some recent posts and thinking about PEP 677, I had a thought…

Could -> be used outside of defs to write a new alternative lambda syntax? Would doing so make people feel differently about 677?

I haven’t come up with something viable exactly but my basic thought is…

lambda x: x+1
\ x -> x+1

lambda: 0
\ -> 0

There are possibly problems with \ in particular as the leader and I’m not wild about adding a second syntax which doesn’t add expressive power. (In fact it just replaces tokens! It’s hardly a new syntax at all!)

However, the SC has said that they want typing syntax to stay well aligned with runtime syntax. Making use of the -> in this way seems to align with that goal.

2

\ seems like a non-starter for this kind of syntax. It’s already the line continuation character; currently it cannot appear outside of string literals except at the end of a line - the grammar doesn’t even allow trailing whitespace.

I think => makes more sense than -> here. It’s already proposed in PEP 671 with a similar meaning - the right-hand side represents a deferred calculation. If there’s a separate operator for this, I don’t see why the syntax would require a leading \ etc. at all - it would just need parentheses (like, a lot of the time to be correct, and even more often to be readable).

3 Likes
3

I think the regular lambda syntax is very nice and I don’t see how using arrows improve it. If you’re used to arrow syntax you’ll probably prefer it, and to me they are both reasonable but Python has made its choice and adding another way of doing it is a definite downgrade.

1 Like
4

PEP 677 also proposed this syntax:

(x, y) => x + y

I also think that’s better because it won’t be confused with return type annotations.

5 Likes
5

Thanks for the pointers at the proposals for =>! I had completely forgotten that was in 677 and didn’t know about 671.

Requiring parentheses seems reasonable, since it makes things unambiguous.

I confess that I’m not a big fan of the existing lambda syntax. In particular, the use of : as a delimiter. Colons otherwise indicate block scopes or slices or type annotations. That’s a lot of jobs for one little character.

Using =>, one sample improvement leaps to mind:

lambda: int
() => int
6

One can follow that same line of reasoning for = and > and say that this adds more jobs for these characters with plenty of jobs already.

7

The idea here is that the digraph => represents a single symbol - it would not otherwise be legal to have the = and > tokens adjacent. We’re only writing it that way so that code doesn’t have to use hard-to-type Unicode symbols in order to express the concept.

1 Like
8

Well luckily Python comes with a way to create anonymous functions without hard to type unicode characters :wink:

My point was just that you can look at any sequence of characters and decide that it supports your aestethic preference. Because that what => vs lambda is, aesthetic preference (unless there is something I don’t know of that arrow notation allows that is impossible with the current notation). This current discussion is unfruitful, Python made its choice long ago and introducing another way that introduces no upsides beyond satisfying some people’s preferences is at best confusing.

The only way I could see this discussion going somewhere beyond aesthetic preference is if you can demonstrate arrow notation doing something useful that is impossible to adapt the current notation to do. So it would, IMO, be best if the proponents of arrow notation focus on that.

1 Like
9

Language aesthetics do matter though.

Decorators are just sugar, but nobody I know considers them unimportant.
One could call context managers dressed up try-finally blocks, although that’s a bit reductive.
f-strings are “just” fancy format calls, aren’t they?
We didn’t strictly need __class_getitem__ on the builtin types, but it has made things nicer.

I respect the opinion that adding a new syntax for lambdas would be a mistake. I have some misgivings about it myself. But it’s a bit harsh to call discussion of it “unfruitful”, i.e. not worthwhile.

Arrow notation (->) is already being present in the language because of the way that it’s used in annotations. Importantly, that symbol combination does not appear in any runtime context. So the question I have is this: would giving it a runtime job (and the most logical one I see is a lambda notation) make the language more coherent?

I see a bit of a problem with => that does not apply to ->, regarding typos:

def foo(x):
    return (x) >= x + 1

Note that >- is not syntactically valid, and >= is not valid in a signature, where => is proposed in PEP 671.

If the community is discussing a range syntax when we have range(), I don’t see why we shouldn’t also consider -> or => when we already have lambda.

10

All of the examples you cited improved the ergonomics of the language in some way (e.g. you don’t have to define a metaclass for a class __getitem__, etc.); this is just a different way to spell “lambda”.

1 Like
11

Range objects are created quite frequently, but doing so currently requires a name lookup and function call, whereas with syntax they could become literals - compare these two functions:

def complex_with_literal():
    return 3 + 4j

def complex_without_literal():
    return complex(3, 4)

Both of them return the same value. If called in a tight loop, though, the second one would have WAY more overhead. Take a look at the disassembly:

>>> dis.dis(complex_with_literal)
  1           0 RESUME                   0

  2           2 RETURN_CONST             1 ((3+4j))
>>> dis.dis(complex_without_literal)
  1           0 RESUME                   0

  2           2 LOAD_GLOBAL              1 (NULL + complex)
             12 LOAD_CONST               1 (3)
             14 LOAD_CONST               2 (4)
             16 CALL                     2
             26 RETURN_VALUE

Your precise Python version will affect exactly how this looks, but the difference will be just as obvious. With range(1, 7) or similar, the difference will be equally stark: a range literal would allow this to be a constant.

The advantage is somewhat weaker if there are any variables involved, but there’s still a significant benefit to be had. Consider:

def make_tuple(n):
    return (n, n)

def make_diagonal(n):
    return complex(n, n)

(I’ve used a tuple here because a “complex literal” is really just the imaginary part, but a tuple would be fairly similar to the way a range literal would work.)

Again, the difference here is that the constructor has to be looked up as a name. The compiler can’t optimize this because you MIGHT (highly unlikely but possible) have reassigned that name or injected a global.

With alternate lambda syntax, this isn’t the case. The existing syntax is already handled entirely by the compiler. Unless the arrow function actually has different semantics to the existing lambda syntax (like how in JavaScript there’s different behaviour around this with arrow functions), there’s little reason to add this.

However, I would definitely recommend looking into some of the editor integrations that change how keywords like lambda look. You may find them of value, without making any changes to the language.

2 Likes
12

My reasons for calling this discussion unfruitful are that no core developer (that can be identified through Discourse) is participating in this discussion, signifying little chance of it making it into the language, and that I seemed to remember this being discussed plenty of times before, but I couldn’t find those discussions so I was wrong on that. Still, the first reason is enough to believe that this discussion will be unfruitful (the fruit being language changes), but that doesn’t mean we can’t discuss the benefits of both syntaxes. Also, English is not my native language so I might be using ‘unfruithful’ wrong. If that is the case, what word should I use to say the discussion is unlikely to ‘bear fruit’, but could still go on for the sake of discussion?

1 Like
13

No no, you’re using the word correctly :slight_smile: It’s a matter of opinion, of course, but your description of your opinion is entirely correct.

2 Likes
14

fruitless

2 Likes
15

I want to share that I also think you used the word correctly and there is no issue with using it as you did. There’s no problem or bad feeling from the tone of the word.

But! I misunderstood you. With your note about core dev participation, I understand better now. I thought you were saying that the conversation is not worth having at all, but your meaning seems to be more nuanced than that.

Also, my thanks to @Rosuav for the detailed explanation about the advantages of range literals. Watching the thread on that topic, I had not absorbed the importance of removing the name lookup.

I’d be somewhat nervous to introduce new semantics which could confuse beginners, but it might be an opportunity to address some of the scoping rules which often confuse people.

def funcs():
    for x in range(10):
        yield () => x

The fact that a lambda does not act as a closure here surprises people when they first learn it. The trouble with giving special early binding semantics to a hypothetical => lambda is that a def doesn’t get those semantics, so it would make the syntax uniquely powerful for building closures. Having a lambda syntax which is more capable than a def is unintuitive.

Similar problems arise with any special rules which make new syntax for the same idea subtly different. It probably makes the language more confusing, not less so. If => is not a synonym for lambda, I think it’s better not to have it at all.

1 Like
16

Be careful of terminology: It does act as a closure. It closes over the variable x and will continue to see changes to that variable. What you perhaps want is partial function application, NOT a closure.

Closure semantics are what most people want most of the time. You just use variables and they work correctly. And let me tell you, this is so common, so prevalent, and so useful, that you quickly forget that it’s even a feature… until you have to maintain some PHP code and closures aren’t automatic. It is insane.

But when you want early binding, yes, it would be nice to have actual syntax for that. I’m not sure what that syntax should be (the current idiom of lambda x=x: x is clunky and abuses the fact that you never intend to pass it that parameter), but it’s not something that I would expect to see from an arrow function. Ideally, I’d like to see these “bound parameters” to behave pretty much like constants (they can’t actually be constants in CPython as those are attached to the code object, not the function object), but what I don’t know is how to spell it in the code.

Also, a feature like early binding doesn’t really want to be locked to lambda-style functions; functions created with ‘def’ will be able to make good use of them too. So whatever syntax is used, it should be able to apply to both.

3 Likes
17

This is off-topic, but there really ought to be something like a local modifier, similar to global, that creates this private, local-scoped copy of a variable.

def funcs():
    for x in range(10):
        __local__ x
        yield lambda: x

Anecdotally, in this context I find one of the hardest things for beginners (and even unsuspecting experts) to debug is why

print(list(f() for f in funcs()))

works as expected, but

print(list(f() for f in list(funcs())))

does not. More so when the closed variable is buried more deeply in whatever is yielded.

1 Like
18

Good idea, but local is probably the wrong term. close over x might be better. (Or static, but static is so overused in various languages that it’s probably best avoided.)

Maybe someone should post this to ideas? I’d be interested in reading a discussion on that.

1 Like
19

The general form of this syntax proposal doesn’t make a lot of sense to me. It seems that you are talking about binding x to the lambda (i.e., creating a separation between the lambda’s concept of x and the loop’s concept of x, but the syntax (analogous to global and nonlocal keywords) looks like it’s talking about scoping x to the loop (i.e., creating a separation between the loop’s concept of x and the enclosing function or module’s concept of x). We did end up doing something similar with list comprehensions, too :slight_smile:

For binding, I also strongly dislike the lambda x=x: idiom. I generally prefer explicit partial application via functools.partial, but more recently I’ve been forced to think about the fact that x=x introduces the parameter as well as binding it. To write the equivalent of lambda x, y=y: x + y requires something like partial(lambda y, x: x + y)(y). Neither feels DRY; if y was previously an outer-scope variable and now I want to make the callable use its current value in later evaluation, I have to add two mentions of y - to parameterize and then fill in the parameter. Further, if I already have a callable accepting just x, I can’t easily modify it to introduce y as a parameter - I would have to forward arguments, possibly use something like functools.wraps, etc.

In other words, agreed there. It took me a while poking around at dunders and dis output to understand the limitation you’re talking about. Yes, that’s unfortunate and fixing it properly sounds like a “Python 4.0” scale change with rippling implications. Alternatively I suppose it could work to have the BUILD_FUNCTION opcode clone the underlying constant code object (in order to attach a fresh co_consts), but that seems inefficient and… very much missing the point.

20

I wasn’t talking about anything to do with the lambda expression specifically, but to say “uncouple this symbol from outer scope”. I meant it as a hint like global, where you say “I acknowledge this symbol I’m about to change comes from outer scope”.