Why not real anonymous functions?

Ideas
105

Yes, commentary, because people disagree that this is useful. Even if people find a good syntax there is still quite a bit of work to do in convince the python community and the core devs that it is actually a good idea. But to quote once again what was already said before “as long as we don’t have a syntax, this discussion is not worth having”.

I am also personally not sure if multiline lambdas are useful. IMO the examples I showed above are perfectly acceptable solutions that already work, and solve 2/3 of the examples brought up. IMO the pattern of instantly calling a function literal is a stupid idiom in other languages that makes code less readable. I am also open to a “scope” statement suggestion in python, eliminating the need for the weird idiom. (and I am even opening to that being able to have a return to bind to a variable in the outer scope).

106

I know mis-use is not usually a good reason for denying a feature, but after working with javascript code and trying to debug tracebacks that are 20-30 levels deep of nothing but anonymous functions, I hope this feature is never incorporated.

5 Likes
107

In fact, you don’t even need to define init; it’s already available as operator.call.

108

Well, yes. Python is also “the odd one out in most mainstream languages” for supporting “offside rule” syntax that determines syntactic structure by indentation rather than with braces or block-ending keywords.

These two facts are closely related.

The exception is Ruby, which is yet again quite unique in its design.

2 Likes
109

Indeed: if this were the criteria for feature inclusion/exclusion we’d have to immediately remove every metaprogramming facility provided by Python :sweat_smile:. We should trust people to use language features thoughtfully and know when certain constructs are appropriate and when they are not. Excluding anonymous function literals on this basis alone would be pedantic and paternalistic.

3 Likes
110

Why not write this as

def click_foo():
  ...
def hover_foo():
  ...
library.addWidget("foo", click_foo, hover_foo)

I’m unclear on why that’s worse? Personally, if I see a stack trace, I want to see a method name in it, not a mangled anonymous function identifier.

8 Likes
111

We should trust people to use language features thoughtfully and know when certain constructs are appropriate and when they are not.

Agreed. A similar argument was used against the walrus operator, and four years later since its introduction I haven’t seen it being abused too much in the code bases I’ve seen. I would assume anonymous functions would probably experience the same treatment. The convention to write readable code runs too deep with experienced Python developers at this point.

2 Likes
112

Sorry for bumping this.

I primarily use python, and love it the way it is, and have learnt to live with it’s design decisions. Nonetheless, I have missed this feature in a few occasions. One of which is bdd style testing akin to rspec / cucumber.js. Again, this is not a deal breaker, as I stated I have learnt to live with this (e.g. I use pytest-bdd, which is nowhere as nice, and many things aren’t well defined, but can be made to work).

If there were a way to do multiline lambdas in Python, I would be on it in Planck time. Now, as I understand, the main blocker seems to be the syntax. Once one gets past this hurdle, only then the idea would be worthy of serious consideration.

I recently had an #idea for one (ok, maybe two) such syntax. I believe it is:

  1. pythonic (IMHO)
  2. familiar to python users
  3. clearly (too clearly) delineates the anonymous block
  4. would work well in IDLE

NOTE: This is purely a demonstration of syntax. So I haven’t tried to demonstrate any usefulness of such pattern. If this syntax is palatable, then first, discussions could be had about finer points of the syntax, associated constraints, and finally, on the merits of multiline lambdas on whole.

Option 1:

  • IMO most familiar,
  • probable nightmare for parser? (but doctest, idle are used to this)
  • two many new tokens, changes to the grammar, for a feature that may see sparse use?
  • way too much ceremony for marking a block? hopefully IDEs and IDLEs can autofill
  • I like this one the most
def insanity() -> Callable[[int], Callable[[int, int], Foo]]:
    # doing a lot of stuff here
    return (
      >>> def (a: int) -> Callable[[int, int], Foo]:
      ...    # doing a lot of stuff here
      ...    return (
      ...    >>> def (x: int, y: int) -> Foo:
      ...    ...     return Foo(
      ...    ...         x,
      ...    ...         y,
      ...    ...         >>> def (z):
      ...    ...         ...     return x + y + z
      ...    ...         ...,
      ...    ...         x + y,
      ...    ...     )
      ...    ...
      ...    )
      ...
  )

Option 2:

  • less ceremony
  • probably less work for the parser (I don’t know)
def insanity() -> Callable[[int], Callable[[int, int], Foo]]:
    # doing a lot of stuff here
    return (
      | def (a: int) -> Callable[[int, int], Foo]:
      |   # doing a lot of stuff here
      |   return (
      |     | def (x: int, y: int):
      |     |    return Foo(
      |     |        x,
      |     |        y,
      |     |        | def (z):
      |     |        |    z = x ** y
      |     |        |    return x + y + z
      |     |        |,
      |     |        x + y,
      |     |    )
      |     |
      |   )
      |
  )

I have slight preference for def, but I have no problem with lambda if it is deemed more suitable.

PS: Probably something about the above syntax (or the entire thing) is absolutely horrible, and make no sense in terms of language design, if it is then I offer my sincerest apologies for wasting your time. Even though I have written some toy parsers, I am not a language design expert, I am sure more experienced people than me (as you all are) would be better judges.

113

Yes, that is a major blocker. Currently, Python statement syntax is heavily dictated by line endings and leading whitespace, while Python expressions are much more free-form (inside parentheses, you

can

add extra blank lines, or unindent, or anything you like). In order to have statements inside an expression (which is what you’re asking for - a lambda function has to be an expression, but you want it to be able to contain statements), you would need a way to go from free-form right back into indentation-matters. That’s an extremely tricky concept. The only way that I can imagine it ever working would be with some sort of clear delimiter, like triple-quoted strings have. This will most likely limit how deeply they can be nested, although I’m not sure that that’s a bad thing.

That isn’t how Python generally works. We don’t design a language with lots of boilerplate and then say “I hope programmers don’t have to type this”. That’s Java’s job. [1] The only case I can think of where Python requires a glyph identifying every line in a block is multi-line comments, and TBH if someone comes up with a good syntax for multi-line comments, I think it would be well used.

But still requires this same adornment on every line. It’s also extremely similar to the practice of putting all your operators vertically down a line when writing a lengthy expression, eg:

bits = (0
    | bit.Interesting
    | bit.External
    | bit.Curious
    | bit.Synchronized
    | bit.Quantum
)

which plays very nicely with source control since adding/removing bits doesn’t require other edits. If someone did this inside a multiline lambda function, it would be tough to figure out which part is doing what. (Maybe it’s technically unambiguous to the parser, but not to the programmer.)

So, yes, finding a good syntax is essential. And…

… yeah, unfortunately it is.

The nearest you’re going to get, as hinted at earlier, is triple-quoted strings. You could build something on top of that:

def insanity():
    return Def("""(a):
        count[a] += 1
        return sum(count)
    """)

and then you have a helper function that uses exec to actually build the function. That won’t give you true nested scopes, of course, but it’d be something. Maybe it’s enough for your purposes?

  1. Sorry not sorry, Java. You know it’s true. ↩︎

114

Thanks for your detailed reply. I don’t have much to say to be honest :sweat_smile:. It’s kind of unfortunate. And yes, I have tried triple quoted strings for this purpose. It is probably passable for one level of nesting. Though closure variables, and passing locals scope becomes hairy.

OT: The subinterpreters/channels in python PEP uses triple quoted strings for message-passing. I suppose, a block delineator could help there as well.

115

Maybe, but passing information between subinterpreters doesn’t really benefit from all the other aspects of nested functions - it doesn’t really make sense to have a closure cell that refers to something in a separate interpreter. So there’s less benefit there.

116

Feel free to edit in your ideas into this page: https://wiki.python.org/moin/MultiLineLambda

2 Likes
117

If I outline an idea, could someone very briefly point out the major things that are wrong with it please? :slight_smile:

The idea: add a compound statement (like with, match) that calls a function, optionally binds the result to a name, and allows keyword arguments to be specified using def in the body. I don’t know what to call the statement, so for the timebeing I’ve named it after my cat Oreo:

def connect(host, port, on_success, on_error):
    ...

# This calls connect() supplying all 4 arguments
oreo connect('localhost', 8000) as connection:
    def on_success():
        print('success!')

    def on_error(exc):
        print('boooo')

Functions defined in the body (e.g. on_success and on_error above) are supplied as keyword arguments to the function on the first line. Only function definitions are allowed in the body. The as connection bit is be optional.

I think this would be quite parsable, right? But perhaps the function call + keyword argument requirements are too limiting? My usual apologies for knowing so little of this despite somehow being a core dev. Thanks.

1 Like
118

I’ll trade a pic of your Oreo for a pic of Carbon.

IMG_20230913_065205499
IMG_20230913_0652054991836×3264 685 KB

Also:

def oreo(func, *args, **kwargs):
    def deco(cls):
        extras = {k: v for k, v in cls.__dict__.items() if not k.startswith("_")}
        return func(*args, **kwargs, **extras)
    return deco

@oreo(connect, 'localhost', 8000)
class connection:
    def on_success():
        print('success!')

    def on_error(exc):
        print('boooo')

(Carbon isn’t my cat, he has a different human; but I often serve as Carbon’s butler.)

6 Likes
119

This is an interesting idea. It seems like something pretty close could already be achieved with class definitions, using metaclasses and/or __init_subclass__ and/or class decorators.

120

I’m with Dan on this one. Every year or so I have to search on this topic to see if I can finally jump on the python bandwagon with the rest of the world, and it never happen for the bulk of my coding.

I work in security, and am taking tons of data, automagically decoding it(long before you could manually do it with Cyberchef), analyzing and annotating. When there is a new TTP, my coders write simple modules which run manipulations, test data, and determine what is applicable, often no more than 10 or 15 lines of code. They drop it into personal working directories to test it, and deploy with minimal modifications. Most of the modules can be dependent on the output of any module that runs before it, hence the need to anonymously run, and yes, we’ve built in debugging so we know when and where things go wrong…

Now, I’ve shifted to Javascript begrudginly, only because C has finally surpassed perl in regex efficiency, meaning JS has by proxy, but I would say, for me to jump ship permanently to python, one of 2 things would need to happen:

  1. Short cut regular expressions to be clean and easy like Perl. If it makes you feel better, you’re copying sed, which is where Perl got it from. I know everyone tries to make a language unique, but somethings are just great for a reason. Even ruby adopted this in some aspects. It’s so Apple suing Microsoft for stealing ideas from Xerox. This is a gripe I have with all languages, not just python.

  2. Most importantly - Solid modular programming for analysis requires complexities more than lamdas. You should have the ability to write anonymous functions and store them in an array and go.
    i. While I understand the whole “clean coding” aspect everyone harps on, I’ve debugged plenty of terrible python code that was like reading braile, and been told over and over that I write the easiest code in Perl to read and understand, which may not be much of a compliment, but if you audit and teach good habits with your staff, you end up with solid code. If you rely on a language to police you, you’ll still can end up with terrible code.
    ii. And debugging sort of makes sense. Ok, not at all. You still have the option to manipulate the namespace in your interpreter/compiler, and could clearly do something like require an identifier in tandem with an anonymous function that is used for debugging, or whatever. Make it an option if you like, but it’s not that difficult of a problem to overcome.

Whoever is developing for python, it’s up to you, you can keep fighting the tides. While you’re the current Monarch at the top of the stack, you have to ask yourself, if the next readable language comes up and handles this topic, are you going to stay the Monarch, or others you dig your heels in on…

Many languages have held this spot, prior. Why did they lose it?

121

Some of them lost it because they tried to be everything to everyone, and thus lost all focus and coherence.

4 Likes
122

The question was rhetorical. I mostly look the time to write that post because without solid anonymous functions, I am pretty much limited in my hiring choices. I’m sure I’m not the only one.

123

But why? How does Python’s sharp distinction between expressions and statements stop you from hiring/being hired?

Your post is nothing but hyperbole, unbacked by any facts. Or prove me wrong.

2 Likes
125

I just had this idea for a syntax, that kinda mirrors how tuples work (bare vs explicit). At least if you squint :stuck_out_tongue: The example is from the wiki article but with another syntax:

this_one_takes_a_func_arg(
    "foo",
    42,
    (def name_here(*args, **kwargs):
        """Does useful and interesting things, I promise"""
        call_a_func()
        do_some_stuff()
        print("print")
        return "foo"
    ),
    boop,
)

The idea being that the normal def is syntax sugar for (def name(): ...body...) similar to how 1, 2 is syntax sugar for (1, 2). Also note that the function still has to be named, even if that name is promptly thrown away. I like that because the reason I want embedded functions isn’t to make them anonymous (which is annoying for stack traces), but just to have the code co-located with some other code that it is related to.

I’m not currently 100% sure I’m serious, but I think this has a certain elegance to it.

1 Like