Even Python’s new PEG parser generates functions to make up for the lack of anonymous functional blocks in C.
As others have commented, it seems that the lack of multi-line functional blocks in Python has ben argued against because of disagreements about syntax, which should be something simple to solve now that the Python parser is PEG, and supports sub-languages, like in the most recent f-strings.
The syntax problem isn’t so much “is it parseable?”, but “how would it fit in with the rest of python’s syntax?”.
I don’t think I’ve ever seen a design which looked good, but wasn’t parseable with the old parser. But I’d love to see something!
Just catching up on the discussion here.
Yes you’re exactly right. Thank you for helping to clarify this point because I’ve been somewhat imprecise.
I think I agree with this.
Good point. Honestly this has bothered me in some other contexts (e.g. I wish that if statements could be evaluated as expressions, as they are in Rust). But that’s not worth arguing about here.
My (possibly incorrect) understanding is that the main problem comes from the ability to unambiguously delimit blocks (suites?). It seems like there are a few alternatives here:
do... end) although I suspect this faces a strong uphill battle due to the introduction of new keywords and the potential for backwards-incompatible changes. However, it’s possible that this could be unambiguous in the context of anonymous def expressions. And it’s not entirely unprecedented given the recent introduction of matchAlso to summarize the discussion so far, it seems like objections fall into two primary categories:
Personally I disagree with the former objection (although it’s a valid opinion) and I am not (yet) qualified to have a strong opinion on the latter.
Just for the sake of a concrete example:
foo = def (*args, **kwargs): do
print("anoynmous function")
if args[0]:
call_something()
return 42
end
foo(1, 2, 3)
this_func_takes_a_func_arg(
first_arg,
def (*args, **kwargs): do
print("anoynmous function")
if args[0]:
call_something()
return 42
end,
"whatever",
)
I don’t like lambdas. And multi line lambas sound worse. I doubt I would use them. But I can still weigh in on the syntax.
foo = def (*args, **kwargs): do
print("anoynmous function")
if args[0]:
call_something()
return 42
end
foo(1, 2, 3)
is ok. I don’t like the do... end... construct. That feels like not python to me. I would prefer to have the scope controlled by indentation, similar to other python control flows. But it’s not totally obvious to me how to accomplish this.
this_func_takes_a_func_arg(
first_arg,
def (*args, **kwargs): do
print("anoynmous function")
if args[0]:
call_something()
return 42
end,
"whatever",
)
Ugh, this is why multiline lambdas feel like a terrible idea. I assume it is given that multi-line lambdas would support the full expressiveness of regular python function definitions (e.g. position/keyword only arguments etc.)?
I would also prefer a notation that makes contact with regular lambda functions. Maybe something like def lambda(*args, **kwargs) would indicate a multi-line lambda is coming.
foo = def lambda(*args, **kwargs):
print("anoynmous function")
if args[0]:
call_something()
return 42
foo(1, 2, 3)
Here the indentation matters and indicates the end of the multi-line lambda. However I don’t think this is well-defined. Consider
(
a,
b,
) = (
3,
4,
)
which is equivalent to
a, b = 3, 4
How could a multi-line lambda be inserted in place of 3 or 4 in this case? How would indentation rules be enforced?
As to do/end, I think the do is unnecessary (you have the colon) and the end looks too much like an identifier to be a good idea. If I was going to design this, I would probably bite the bullet and use braces, require an explicit return, and then let the first line determine the indentation level of the base block:
def (arg1, arg2) {
if predicate(arg1):
return func(arg2)
make_side_effect_happen()
return default_value
}
Used in a function:
outer_function(
arg,
callback=def (arg1, arg2) {
if predicate(arg1):
return func(arg2)
make_side_effect_happen()
return default_value
},
)
Adding annotations gets you not too far from a typescript style anonymous function
def (arg1: Type1, arg2: Type2) -> ReturnType {
if predicate(arg1):
return func(arg2)
make_side_effect_happen()
return default_value
}
These braces would be explicitly anonymous function delimiters and not alternatives to significant whitespace.
It looks like anonymous functions are saving one name, and one or two lines of code… is that it? What am I missing?
If you want to be convincing, use real examples
lambda, and there you at least have the argument that it saves a line/return statement. Here it’s just useless noise who’s primary effect is that you no longer have the name of the scope in tracebacks. This pattern should honestly be illegal if any variation of this syntax is added, just like := is illegal as a statement.Note that both of these are completely independent of the actual syntax chosen. I have not see a good situation where defining a normal, named function a bit away is actually a problem and not doing that would improve readability so much that adding parsing complexity (both for humans and machines) would be worth it.
Also, as mentioned before: Forget about whether or not the PEG parser can handle the syntax. Think about whether humans will be able to easily understand the syntax.
Up until now, python has the very simple lexing rule that within parentheses, indentation and newlines are irrelevant. Any syntax proposal is going to change this assumption. Why is it worth breaking this? What is the killer usecase for this feature?
I think the main problem is that indentation is significant in multi-line Python code (and that’s non-negotiable IMO, it’s fundamental to the “look and feel” of Python). And a syntax that combines an indentation-significant multi-statement block inside an expression (that largely doesn’t care about indentation) is going to be very hard to pull off without looking ugly. And that’s been the factor that’s caused every previous proposal to fail, IMO.
Delimiting blocks is secondary, in the sense that any mechanism for delimiting a block that doesn’t involve significant indentation is going to be rejected as “not Pythonic”, so there’s no real point in worrying about it - if you don’t solve the problem of making significant indentation look acceptable, messing with the delimiters won’t matter. And if you do, messing with the delimiters isn’t needed.
I think the latter objection needs to be addressed as a priority, and the former isn’t worth worrying about until that’s been done. But FWIW, personally, I’m inclined to think that anonymous functions would be a minor convenience if added to the language. Useful on occasion, but not massively impactful. If they did become extensively used (in the sort of way they are used in Javascript) I’d consider that a significant issue, probably enough to swing me against the feature. However, it’s hard to judge whether that would happen - once syntax is dealt with, the tone of the discussion about how useful the new feature would be is likely to be crucial in getting a sense of people’s expectations here.
Yes, you’re right about that. I chose a bad example but just intended to show that it was useable as an expression. A better example would have involved return.
This objection falls into my first category. Personally, for whatever reason, I find myself writing a lot of code where functions are passed to functions. In cases where the passed function is only used once at the call site, it is cumbersome to separate the definition of that function from its usage. I’m not sure how to convince you unless you’ve encountered this pattern frequently in other languages.
Yeah that’s fair.
This is fair, but also largely dependent on the kind of code that you write, which is why it mostly boils down to a matter of (apparently strongly held!) opinions. As I’ve said elsewhere, I’ve written a fair bit of Rust and a little bit of Java and JS here and there which informs my opinion on this. It would be useful for me, would enable more functional-style code, and would lead to clarity and concision in a lot of the code I write.
Here’s someone else who expresses a similar opinion. I think they go too far but they do a decent job of motivating the use case.
Of course someone is always going to jump into this discussion and say “well you don’t really need that because X, Y, Z and there’s a better way etc etc etc.” That’s fair, but it’s also just an opinion. I obviously disagree and think that it can be a powerful construct for clean code. The reason we don’t see code like this in Python is because we don’t have the syntax to support it. The question remains as to whether it’s ultimately worth it.
Maybe it’s just me, but I don’t seen any use case in the post, other than “I really like JavaScript’s implementation, Python should have something similar, too”.
Assuming that were true, just to play devil’s advocate, what exactly is wrong with that line of reasoning? Where did match come from? Where did list comprehensions come from? There is a constant interchange of ideas between languages because developers use different tools and want to apply new solutions to new problems.
Then take some of that code, modify it to use some inline syntax, for example the one you showed above (it at least isn’t horrible to look at) and show before and after, of real, production level code.
There is a difference between “playing devil’s advocate” and “making a strawman argument”. Ofcourse it’s ok to be inspired by other languages, but it can’t be your only factor and it shouldn’t be the core of your motivation. match has decent-to-good usecases in conjunction with dataclasses, incoming json objects and similar stuff, and the altenrrive, old-style syntax is a lot more verbose. It isn’t just one line like this proposal would save.
The question should be “Why real anonymous functions? What Python has been missing for years.”
Show us what Python has been missing. Think of a use case that is not convenient to solve with the current language features, e.g., lambda, classes, modules, etc.
PEP 622 goes into great detail, with concrete examples, about how the match statement could improve existing Python code.
PEP 202 provides concrete examples of what a list comprehension should look like. I don’t know the extent to which the final proposal was discussed before the final form was accepted; things were different in 2000.
You said
I think they go too far but they do a decent job of motivating the use case.
Please quote the portion of that post where you see a use case.
This is a fragment of the code that TatSu generates as a parser for its own grammar language:
@tatsumasu('Grammar')
def _grammar_(self):
self._constant('TATSU')
self.name_last_node('title')
def block0():
with self._choice():
with self._option():
self._directive_()
self.add_last_node_to_name('directives')
with self._option():
self._keyword_()
self.add_last_node_to_name('keywords')
self._error(
'expecting one of: '
'<directive> <keyword>'
)
self._closure(block0)
self._rule_()
self.add_last_node_to_name('rules')
def block1():
with self._choice():
with self._option():
self._rule_()
self.add_last_node_to_name('rules')
with self._option():
self._keyword_()
self.add_last_node_to_name('keywords')
self._error(
'expecting one of: '
'<keyword> <rule>'
)
self._closure(block1)
self._check_eof()
The blockN() declarations are called so because they are one-time functional blocks passed to functions that do the hard work.
I thought about it, and I don’t see a way to improve that code with anonymous, inline functions, except for skipping the need to having to generate the blockN identifiers.
Then the code may not be “pretty” from the perspective of other languages (JavaScript), but what’s happening is very clear, as is the Python way.
If someone could provide syntax to make the above code fragment better, I’d jump in, and support the change. As it is, and as others have said, there’s not a clear use case for anonymous function syntax.
EDIT:
For context…
def _closure(self, block, sep=None, omitsep=False):
self._push_cst()
try:
self.cst = []
with self._optional():
block()
self.cst = [self.cst]
self._repeat(block, prefix=sep, dropprefix=omitsep)
self.cst = [self.cst]
return self._merge_cst()
except Exception:
self._pop_cst()
raise
The one use-case that I would have for this is UI creation. This is something that has ALWAYS been horribly clunky in Python, no matter which UI library is involved. Most often, it requires a lot of completely unnecessary intermediate variables, solely because Python’s syntactic structure requires it. That means naming things that don’t really need names.
With inline multi-statement functions, the structure of the code could directly mirror the structure of the window, instead of having to define a bunch of terms and then use them, forcing you to read up and down and all over the place to figure out what’s going on.
That said, though: this is NOT a strong use-case. Even with multi-statement inline functions, this kind of code would be awful and clunky. There are probably better ways than building a UI in code. It’s just that I’ve never found any that I’m happy with.
I don’t want to sound too dismissive here, but often when I hear people say this I tend to think it’s because they are trying to import idioms from other languages into Python when Python just has an alternative way of doing whatever they’re trying to do.
I’m most familiar with this in the case of trying to write JavaScript in Python. For instance, some JS programmers are used to a method-chaining style with a lot of filters and maps. But in Python a lot of those can be done using list/generator comprehensions instead. Similarly JS has a .forEach method that takes a function, and yeah, it’s more convenient in JavaScript to use an inline function, but in Python you don’t need that because the regular for loop is fine, and if you want more fancy behavior, you have the whole iterator protocol to play with. And one of things people like about JS “fat arrow” functions is that they solve some problems with an awkward this context, but Python doesn’t have those problems in the first place.
Even within JS, I feel that some of the reasons people use inline functions are basically workarounds for annoying behavior of JS (e.g., this and rather complex and confusing behavior of the basic for construct). In Python, where those workarounds are no longer necessary, there is even less reason to write code that way.
That style of functional programming isn’t inherently bad or anything, but Python just has alternative ways of achieving many similar things. I’ve seen people express points similar to yours in the past, but in each case my reaction is mostly the same: wanting to write code in a style similar to how you would write it in JavaScript (or Rust or whatever) is not in itself a compelling justification for adding features to Python. The question is whether adding the feature to Python improves our ability to write good code in Python. If there are already fine ways to do something in Python, we don’t need to add another way just because some other language does things that way. (And we definitely don’t need to take the JavaScript approach of constantly adding a bunch of revamped or alternative versions of old functionality, littering the language with the detritus of the past.)
What’s wrong with that line of reasoning is that it doesn’t make sense to add features just because another language has them. The features you mention were not added just because another language had them. They were added because another language provided some inspiration for a feature to add to Python that might be helpful when writing code in Python. It’s totally great to have interchange of ideas between different languages, but each language will choose the features that mesh well with its existing constructs. Just because a certain Indian restaurant makes a great chicken curry doesn’t mean the Mexican restaurant down the street should add chicken curry to their menu too. 
I have encountered it in other languages - what they have in common is brace-delimited or similar syntax rather than whitespace-significant syntax. I’ve often found the “idiomatic” way of doing things cumbersome in those languages (multiple consecutive occurrences of )}; in JavaScript code, or permutations thereof, are not my idea of “fun to read”) and I feel that it would be much worse in Python.
I have encountered many developers (and students) who I felt needlessly broke lines of code up into smaller statements and “baby-stepped” through the code. But sometimes names are useful, and sometimes breaking up lines is useful. Not all languages can be Lisp. If a function isn’t itself just making some trivial adaption to an existing function (such as one can easily do with lambda or functools.partial, in a way that the name of the wrapped function is still prominent), then it’s doing something according to its own logic; and that task probably deserves a name.
Let’s consider what it looks like when we pass a complex callback function, something where we want to write multiple statements in order to implement the logic. Currently we do something like:
def callback(framework_arg):
...
...
api_call(1, 2, callback, kw="example")
If we want to “inline” the callback with any kind of multiple-line, whitespace-significant syntax - then yes, we get to avoid naming the callback and then using that name (and to be fair, names are hard; “callback” doesn’t mean much in this example, and in real-world code a better name might not make itself obvious). But now we have to completely rethink the formatting of all the other arguments for the call. Does our new syntax let us start on the same line, like api_call(1, 2, magic_keyword etc., the way that a lambda does already? If so, how are lines inside the callback function indented relative to api_call? Or must their indentation be relative to the magic_keyword? How does that interact with existing guidelines and conventions for wrapping the other arguments for a multi-line (parenthesized) call? What about the argument afterward - is it forced onto a separate line? What about the comma after the callback - can it appear “inside” the function block, given some delimiter?
Yes, you can try to answer these questions; but so far, the consensus is that none of the possible answers are any good. Even if you find an aesthetic compromise that you can get everyone to be satisfied with, I don’t think you’ll find that it actually saves vertical space, anyway. And then it deprives you of a compiler-recognized way to explain what the callback actually does (i.e., its name).