The fact that this request for multi-statement lambdas (or real anonymous functions or whatever we want to call it) keeps coming and coming should be an indicator that many people want it/need it. Almost any of the most commonly used languages today (JavaScript, Kotlin, Java, C#, ruby…) has this feature, so most people that do some programming in any of those languages is missing the feature when working with Python.
I can not help adding that it’s just the same with the so many times requested “Safe navigation operator”.
I fully understand the “there’s no time, there are no resources to implement it so far” kind of answer, but I can not understand the “it’s not necessary/why would you want such an oddity” kind of answer.
3 Likes
It’s not an oddity. But every language has to make decisions, has to follow a design goal. If every idea that has any sort of merit gets implemented, the language ends up being a nightmare of different styles. Imagine if Python had floating-point literals in decimal, but integer literals in octal; or if “for” loops created a new nested scope, but no other statement did; or if “x = y” would take a copy of a list but make a reference to a dict. This sort of inconsistency is worse than just picking one option and sticking with it.
There are, indeed, languages in which you can embed a function in an expression, and then have statements inside that function. But that’s not Python. In Python, you don’t embed a statement inside an expression. What would the indentation rules be? Consider:
def spam():
pass # there's an INDENT
pass # there's a DEDENT
def ham():
pass # look, another INDENT
x = (
1
+
2
) # no DEDENT or INDENT here
pass # the DEDENT happens here
Whenever you start a “suite” (a block of statements usually introduced with a colon - check the grammar for full details), you have to know what your current indentation is, in order to be able to recognize a subsequent indent, and find the end of that suite. So, what’s the indentation level for a suite that’s embedded inside an expression? I’m sure you could come up with SOME sort of answer, but what are the consequences of that decision?
Multi-statement embedded functions usually require textual delimiters rather than indentation-based. Here’s how it works in Pike:
void spam() { //the open brace starts the function
ham(
1, //unindenting is fine inside an expression - although it would be outside, too
lambda() { //again, it's the open brace that starts this function
return 42;
}, //end of inner function
); //this code is ugly but the indentation doesn't cause actual confusion
//You could write it more tidily like this:
ham(1) { //Opening a brace after a function call automatically passes the lambda function as the last argument
return 42;
}; //But, regardless, the close brace marks the end.
} //end of outer function
There are a lot of advantages to Python’s choice to use indentation to define blocks. There are a lot of advantages to making indentation completely free-form inside parenthesized expressions. But with those two choices locked in, having statements inside parenthesized expressions is a problem.
5 Likes
It’s for sure a complex topic and I have almost no experience with grammars or compilers implementation. I guest there could be some restrictions to how multi-stament lambdas could work (like the restriction with assignment expressions where we can do “a := calculation()” but can not do “ob.a := calculation()”).
The beatuiful coconut language (a Python superset that translates to Python) supports multi-statement lambdas. I’ve had not time so far to fiddle with it, so I can not say what quirks/limitations it comes with.
The answer is right in the docs there, actually:
... and statement can be any non-compound statement—that is, any statement that doesn’t open a code block below it (so def x => assert x is fine but def x => if x: True is not).
Everything I said above about a “suite” still applies; what Coconut has done is relax the restriction about statements specifically. So as given in the example, you can now assert in a lambda function. What else can you do? Assignment and global/nonlocal seem like a useful pairing; a lambda function that increments a variable in a surrounding scope could be of value. I guess a raise statement might be of occasional value, but in the rare cases where you want that, it’s not THAT hard to flip to a def function. Tossing an import in prior to using the thing you imported (eg lambda x: (import pprint; pprint.pprint(x)) or something), maybe. So there are a few small benefits there, but nothing huge.
OTOH, it’s also not nearly as problematic as including a suite inside an expression. If there’s enough interest in it, I’m sure someone could come up with a reasonable proposal to implement that in Python. It’s still going to run into the fundamental problem of density, though. People already frown on code like:
if some_condition: print(x); print(y)
It’s fully legal, and both simple statements are guarded by the same if, but you’ll find a lot of people reject it and recommend breaking it over multiple lines. So what would be the use-case for a multi-simple-statement lambda function? It has to be included in some other expression (otherwise you’d use def), and it has to justify putting two statements into it. Maybe something like this??
handlers = {
"btn_spam": lambda e: print("Spam"),
"btn_ham": lambda e: print("Ham"),
# All unknown events, dump them to the console
...: multilambda e: (import pprint; pprint.pprint(e)),
}
I could imagine that finding a use somewhere, but aside from a highly contrived example, I can’t think of anywhere that wouldn’t be better done with def functions and a decorator (which is how I would generally do this sort of thing - def handler(f): handlers[f.__name__] = f and you get full power in your functions).
Trouble is, a multi-simple-statement lambda function still wouldn’t satisfy the people who want to put full suites into their lambda functions. I would be curious how much use Coconut’s statement lambda actually gets.
As someone that used to write long arrow-functions (lambdas in JavaScript parlor) everywhere in JavaScript, thanks to Python I’ve come to admit that for functions longer than 3 or 4 lines it’s much better to create a named nested function. Using that named function (assuming it has a meaningful name) in place of the inline-function is much more semantic. But for small functions however, I still find them pretty useful, and all the cases you mention are good examples of it.
2 Likes
Look, if you want to champion a proposal for a specific feature, not buried behind an emotional name like “real anonymous functions”, and laying out some actual examples where it’d be useful, please do. It would be refreshing to have a “variant on lambda” proposal that’s actually workable for a change. I’m not one to push for this since I can’t really see that many uses for it, but it’d be a solid topic for discussion.
4 Likes
Inline function can get additional type hints, which helps a lot in coding.
However, there’s no multi-line inline function solution, so i have to define a function before.
If i do not add type hint for
process_each_x, the return type will be lost .This is just what i want:
I’d like to weigh in.
Callbacks. The answer is callbacks. Callbacks are, and always was a pain in python. Defining a literal body of a method, just to use it in a single place as a callback is architecturally awkward. It creates an unnecessary mental load to map a method definition to execution of another method.
Though I also agree that most callbacks syntax, does not go well with python’s principles and current syntax.
I think @barneygale 's syntax is the most beautiful one with a naming suggestion from me.
def callback_requiring_method(cb: Callable[[Any], Any]):
...
with callback callback_requiring_method():
def cb(arg: Any) -> Any:
"""This is great as its both have nice typing and under its owner."""
If you dont like ‘with callback’ syntax, I would like to point out ‘raise from’ and ‘async for’, which does the absolutely same thing lexically.
And for returns, we dont need to find/follow the same pattern. Something as simple as this will both be pythonic and easy to follow.
def wrapper():
return def inner():
"""This is perfectly beautiful."""
I just think that a code, that is bound to another code should be live under that code. A callback should be defined under its own user. A wrappee that wants to be returned needs to be defined with the return.
1 Like
I would also say with the rise of declarative programming this is also becoming more apparent with frameworks hijacking generators or context managers
just to pull it off.
if python had this I would suggest following
function_call() with args:
Blah
How is this any different from using a decorator?
def callback_requiring_method(cb: Callable):
...
cb()
@callback_requiring_method
def cb(arg: Any) -> Any:
...
Besides the fact that one of the approaches currently exists 
1 Like
Perhaps if this pattern was in functools as a decorator to turn a function into a decorator that allows that ?
all functions are decorators.[1]
This is a particularly cursed way to print Hello world in Python:
def f(x): x()
@f
def f(): print("Hello world")
Technically only the functions that (can) take a single arg, and all classes where the init can take a single arg are also decorators. ↩︎
I was just meaning as a convention, this to me is not the most explicit and pythonic
def f(a, b, x): x(a + b)
@f(1, 2)
def callback(c); print(c)
I like how in other languages you can sign post without changing the order in which you read the code, so perhaps something like this
I was just meaning as a convention, this to me is not the most explicit and pythonic, sometimes it’s a decorator that returns a function and in this case its a function call pretending to be decorator
from functools import with_callback
@with_callback
def f(a, b, x): x(a + b)
@f.as_callback(2, 2)
def callback(c); print(c)
I don’t really know what this suggestion means. What would such a functools function even do?
I suppose I could see the benefit of having (in the standardlib) a function like
def callback(f: Callable, name:str):
def outer(*args, **kwargs):
def inner(func: Callable):
return f(*args, **(kwargs | {name: func}))
return inner
return outer
so that you could write
@callback(f, "x")(2, 2)
def my_callback(c):
print(c)
because this is complicated enough that I’m not certain I didn’t make a mistake, and getting the typing right would be a pain.
But there’s downsides to that too, because you’d have to design it with specifics that are suitable for literally everyone.
Like I’d hate your class-based solution with a property. You like it apparently. And in specific scenarios sometimes you’ll want to insert the callback as the first arg. Some people may need to insert it at the end of *args.
1 Like
It would exist just to create a explicit and clear convention for doing callbacks like this, idea is a callback marking decorator could start there similar to partial
It would work like partial does except instead of taking a function and applying args to it and the returning a function you can call.
It simply would call the original func and pass in the decorating function as the final argument.
Partial was such a simple function, functools didnt need to include it but in doing so its created a convention and now when you see it in a code base you know what it’s doing.
I think these kind of pattern could benefit from this, and then if adopted a nicer less ugly syntax could be figured out 
There’s a lot of different ways callbacks are implemented and used though. Sometimes they have to be passed positionally at some chosen index, sometimes they’re at a fixed keyword argument, sometimes they have their own keyword argument. Sometimes you can only use one, some can be stacked. Some have to be sync, some can or have to be async. I’m sure there’s a million other small details that every library chooses some subtly different set of behaviours on.
I don’t think there’s a single implementation that covers even just a majority of the use cases. It also really isn’t that complex to write something like the decorator you’re proposing in whatever library someone is making. So I don’t really see the benefit of adding something like this to the standard library.
4 Likes
functools.partial() is by no means a “simple function”. It’s a fairly complex class that just manages to hide its complexity behind a very elegant public API. This effort is worth it for something that covers basically every imaginable usecase for a partial function, and with better performance than a pure python solution to boot. I don’t think either of those are applicable for what you’re suggesting.
3 Likes
For this specific use case it would only work for functions where it’s the last argument
in other languages such as swift they didn’t even attempt to add these other cases.
I know partial attempted this but I think that’s fine, other languages also tried to do the same.
but with callbacks in particular this is only route as far as I’m concerned and solves the majority of cases where having multi lambda would help
I think multiline lambda hurts readability when it’s used for these other positional arguments and I think python is right to restrict it there
There’s another route that’s even simpler: just leave things as-is.
I can totally see something like this having merit when designing a new language from scratch. Having a simple way of doing a common thing that everyone will use is great. Swift’s implementation is great in that regard, it can’t do every specific little thing but it covers every use case well enough that the ecosystem evolved to suit it really well.
But we’re now in a totally different situation. We already have a hundred libraries that each want callbacks to be handled slightly differently. We can’t just make a small thing that works well enough because those libraries already exist and want things to be different. Python just evolved in a different direction from Swift, trying to emulate its features one to one here doesn’t really make sense.
At least in the code I’ve interacted with callbacks aren’t even as common in Python as they are in Swift. Probably because of some combination of the use cases for the languages being different and because of those early design choices. What are big problems with the way Python is currently structured that we need a change like this for? Just adding a feature because it can be used and other languages use it effectively doesn’t really give us an actual benefit. Would Python be a language where that feature would be used effectively? Can it not be solved reasonably well already?
1 Like