What is the purpose of Lambda expressions?

Python Help
1

This is a typical example for beginners:

x=lambda n:2*n
print(x(7))

Otherwise I would create a function:

def dbl(n):
  return 2*n

print(dbl(7))

Of course: I can write simply 2*7, but the idea is to save a complex formula in an object once, and reuse it several times.
In this case the Lambda expression saved me one line, and one of the characteristics of Python is that the code is short, clear and elegant; but are there more advantages to the Lambda expressions apart of it? In performance, in saving extra codes later, etc?
For example, does it save us code in lists comprehension (comparing to functions)?
Is it just an issue of self preferences? Some like functions and other Lambda expressions…
Thanks in advance.

1 Like
2

Welcome, @GeriReshef !

Good question. The basic distinction between creating a function with def and a lambda expression, besides syntactic sugar, is that lambda does so without assigning to a name; in fact, the other common name for a lambda is an anonymous function. If you’re going to create a lambda function and immediately assign to a name, it is much clearer, more idiomatic, easier to read, and more flexible and powerful to simply use def. Thus, I’d argue that the beginner example above is a rather poor example of using a lambda, both because it is widely considered a bad practice to use a lambda that way, but also fails to illustrate the primary reason why/use case where you’d use one, and the distinction between a lambda and a regular function def.

Essentially, you can think of lambdas as small throwaway functions you pass to something else immediately upon creation. Typically, you’ll see them used when you need to pass one function as an argument to another. For example, if you have some list spam of various strings (e.g. ["a1", "b2", "c3"]) and want to sort them as numbers rather than lexically, you could do sorted(spam, lambda x: int(x[1])), instead of having to define a whole new function just for this simple task. Or, suppose you have some timer that calls a function you pass it at a later time. If you wanted to do something simple, a lambda might be a good choice, e.g. timer.do_later(lambda: print("Time to wakeup!")). Does that make sense?

If you’re going to reuse a function more than once, use def—that’s what its for, for the reasons above. The whole intention of a lambdas (again, aka anonymous functions) are for throwaway, use-once functions that you don’t want to bother giving a name, because you’re not going to use them again.

To each their own, but lines are free, while time, clarity, explicitness and readability are not—it may look clear to you right now, but in the long term it is harder for others (or you, if you come back later) to read and understand your code, harder to annotate it, and harder to spot bugs. Per import this, the Zen of Python, “Explicit is better than implicit,” “Sparse is better than dense,” “Readability counts,” and most importantly, “There should be one—and preferably only one—obvious way to do it.”

Nope, and there are a number of disadvantages for non-trivial functions—less clear and obvious syntax, much less powerful, can only be one line, more complex to understand, and you don’t get as useful help, tracebacks, etc.

Using a list comprehension is generally an alternative to using a lambda function with map() or filter(), unless you’re making a list of simple functions, in which case lambda is useful to avoid having to define your functions beforehand (given you cannot, of course, use the def statement within a list comprehension, as the latter must contain only an expression).

Not really; each has their clearly defined use case, and it is strongly discouraged (and sometimes impossible) to use one where the other is more appropriate. As mentioned, lambdas are anonymous functions, and are used when you only need the function one place and don’t want to give it a name, whereas regular def functions are for other cases.

I don’t mean to be dunking on lambdas here—they are useful for the specific purposes for which they are intended, and I do use them a fair bit in scientific computing. But like anything, they have their place and are not really a replacement for a regular named function.

7 Likes
3

Thank you so much for the detailed answer.
This was my first question here, and your answer was the best welcome greeting I could expect.
:handshake:

3 Likes
4

Thanks so much for your kind words! For its part, knowing it was helpful was the best thank you I could expect. Cheers!

1 Like
5

Lambda functions are essentially identical to the equivalent def
function. They are a different syntax for the same result.

Lambda functions have the restriction that they can only include a
single expression, but other than that, these two functions are
essentially identical:

def double(x):
    return 2*x

double = lambda x: 2*x

If you were to give those two functions different names, and then
compare their code and internal structure, you would be hard pressed to
find any differences at all.

Perhaps the only difference (that I know of) is that the def function
has a nice name, ‘double’, that will show up in error messages, while
the lambda’s name is the generic:

'<lambda>'

It is for that reason that we discourage people from “saving a line” by
assigning lambdas directly to variables. It makes it hard to debug
errors in your code if all your functions have a generic, anonymous
name.

If you really care about saving one line, you can squeeze a def onto one
line:

def double(x): return 2*x

but again, we tend to discourage that sort of thing.

1 Like
6

Oh, a thought comes to mind, there is one other important distinction
between a lambda function and a def function, but it is not inherent to
the functions themselves, just a matter of the way we use them.

Every object in Python has a lifetime, the time between when it is
created and when it is destroyed and the memory re-used. Functions are
objects too, and so they have a lifetime.

A function is born when the def statement (or lambda expression) is
executed, and dies when the function object goes out of scope and the
interpreter’s garbage collection reclaims it.

When we create functions with def, we usually write them at the top of
the module, and never delete them, and so their lifetime is that of your
whole program.

When we create functions with lambda, we often (not always!) treat
them as transient objects that have a much shorter lifespan.

For example, compare using a def versus a lambda for a key function to
sort. If we start with this set of strings:

mystrings =	['fox', 'Gnu', ' bee', 'CAT', 'EEL', 'ant', '  dog']

and sort it using sorted(mystrings), we get this:

['  dog', ' bee', 'CAT', 'EEL', 'Gnu', 'ant', 'fox']

Not so helpful. We can do better with a key-function.

def keyfunc(string):
    # Case-insensitive sorting, ignoring leading
    # and trailing spaces.
    return string.strip().casefold()

sorted(mystrings, key=keyfunction)

will give us this:

['ant', ' bee', 'CAT', '  dog', 'EEL', 'fox', 'Gnu']

Now consider using a lambda:

sorted(mystrings, key=lambda string: string.strip().casefold())

That will give us the same result, but this time the lambda function is
transient: it is born at the moment we call sorted(), and dies when
the sorted function returns its result. Its memory is immediately
reclaimed, ready to be reused if needed.

Nine times out of ten this is a difference that makes no difference, but
for tiny functions that you use only once, using lambda can help with
memory usage.

3 Likes
7

Presumably if they are born and die quickly in a loop that would be inefficient?

8

In terms of memory efficiency, using a lambda function is essentially always going to be more efficient over the lifetime of your program than an equivalent function defined with def. However, given function objects rarely consume more than a small amount of memory, for all but extremely niche cases or very low-memory embedded devices, this isn’t something you should be thinking about.

And CPU-wise, per timeit instantiating the lambda above took 100 ns, a local lookup on a function objected defined with def took 75 ns, and creating the above lambda and then binding it to a name (as was done above) took 188 ns. So the difference is very small, and effectively in the noise (though that for name binding was nearly twice that of just declaring a lambda again emphasizing that if you’re going to bind a lambda to a name, either don’t or just use def).

1 Like
9

Hi C.A.M.,

I don’t understand your reasoning here:

“In terms of memory efficiency, using a lambda function is
essentially always going to be more efficient over the lifetime of your
program than an equivalent function defined with def.”

How do you work that out? The two functions are effectively identical
in size:

>>> a = lambda x: x+1
>>> def b(x): return x+1
... 
>>> sys.getsizeof(a) == sys.getsizeof(b)
True
>>> sys.getsizeof(a.__code__) == sys.getsizeof(b.__code__)
True

Their sets of methods and attributes are identical:

>>> set(dir(a)) == set(dir(b))
True

Their generated code is the same:

>>> a.__code__.co_code == b.__code__.co_code
True

If the lifetimes of the two objects are the same, how is one more memory
efficient than the other?

10

Also, I don’t know how to interpret C.A.M.'s timeit results:

“And CPU-wise, per timeit instantiating the lambda above took 100 ns, a
local lookup on a function objected defined with def took 75 ns, and
creating the above lambda and then binding it to a name (as was done
above) took 188 ns.”

If I compare apples-to-apples, I get effectively identical results for
building a function object and assigning to a name, whether you use def
(which combines the two as one operation) or lambda plus assignment:

>>> from timeit import Timer
>>> t1 = Timer("def dbl(x): return 2*x")
>>> t2 = Timer("dbl = lambda x: 2*x")
>>> min(t1.repeat(repeat=7))
0.06949841813184321
>>> min(t2.repeat(repeat=7))
0.06824702001176775

In my tests, the version with the lambda is consistently marginally
faster, but I wouldn’t credit that as a real difference, the amount is
very small and well within the variation from run to run.

So from an efficency viewpoint, I don’t see any difference between
using def and assigning a lambda to a name. They both take about the
same time and space.

As I mentioned earlier, I think the real reason we recommend against
assigning lambdas to a name instead of def is simply that (1) it is less
familiar, and therefore more surprising, to read; and (2) being
anonymous, error messages generated from lambdas are not as nice.

1 Like
11

Hey Steven, sorry for the confusion—I should have been more explicit. The statements in my reply were predicated on your stated presumption about their lifetimes not being the same, which was the crux of your detailed explanation regarding their memory efficiency above, and to which I assumed @MartinPacker was responding to:

I interpreted @MartinPacker 's question,

to be asking if a lambda function being repeatedly created and destroyed in a loop is less efficient (which I addressed in terms of both memory and CPU) than defining one named function outside the loop and just referencing it repeatedly. In this case, as you explained, the function object created by the lambda only occupies memory during the execution of the callable which it is passed to, while the named function object lasts the life of the program.

Therefore, memory-wise, the former is strictly more efficient over the program lifetime. My timeit results compared CPU time needed to create a lambda versus to look up the name of a function in the same scope, and found them to be nearly equivalent (which is a best case versus non-local/global scope), as well as illustrating the relative inefficiency of assigning a lambda to a name, and then using it versus simply passing it directly (rather than versus using def, which was identical as in your results and as previously discussed).

There is another potential interpretation of this question: defining a named function within the loop and then passing it (as before), versus simply passing a lambda directly. This in fact has essentially similar implications to the above, with the addition of def needing to pay the CPU cost of name binding in addition to function creation every time (which at least in my results above over doubled CPU execution time), and the named function object lasting the lifetime of the loop’s enclosing scope, rather than the entire program, but still more than lifetime of the execution of the callable to which it was passed as with the lambda. In either case, these both support the conclusions of your previous message discussing memory efficiency, which was my intent.

Your interpretation above seems to imply creating both functions inside the loop (in that case, it is not clear which one is being referenced), but only if comparing a named function using def to a lambda that is bound to a name, which we both previously explained is something that doesn’t make sense to do, as it produces equivalent results to a def but is less readable, while not actually having potential CPU and memory efficiency benefit of passing a lambda directly. As you mention, a lambda bound to a name has the same lifetime of the explicitly named function at the same scope (they are both just objects bound to local names, after all), and thus there is of course no other meaningful difference (as you illustrate).

In the end, my real point though was that in almost all cases, this is an over-optimization, at least for a newer user, spending any mental CPU time worrying about this will waste more than it will ever save, never mind time lost due to the readability, familiarity and clarity impacts, which is something I think we both strongly agree on.

1 Like
12

Oh, for some reason I never received Martin’s question about functions

in a loop, so it looked to me (and my mail client) like you were

responding to the original post.

I’ve noticed over the last few weeks that I have been receiving obvious

replies to posts but not the original post that prompted the reply. This

time it wasn’t obvious :frowning:

2 Likes
13

Ah, no worries. Sorry to hear that!

14

Ah, I see what is happening. Discuss has now started breaking email
threading. If you reply to a post, even by email, it only sometimes is
correctly threaded as a reply to that post.

In this case, I had Martin’s message, but your reply to Martin was not
threaded as a reply to his message. And my reply to your post is threaded
as a reply to the original message, as is your reply to my reply.

2 Likes
15

Ah, I see. What I suspect may be happening is that replies to the last message on the thread (such as this one) don’t get threaded as replies, and rather are displayed as replies to the original thread, given that they do not show as replies in the web UI once posted, and this was true of the two messages that you mentioned.

16

@CAM-Gerlach answered the question I originally had in mind.

But people who know me - and this might gradually include people on this board to which I’m newish - know I like to widen questions.

(I’m actually a worldwide mainframe Performance expert - so no Performance question is really off limits.)

I do, though, think lambda expressions take some getting used to. (Despite my Masters thesis - in 1985 - being about Functional Programming.) And the “some getting used to” might be bad enough for most people to swear off them. In most of my use cases any speed / memory / CPU difference will be immaterial.

17

Perhaps I’m mistaken, but I had the impression that mainframe performance was primarily bottlenecked by the limited bandwidth of the card punch machines used to input COBOL programs, the switching speed of the vacuum tubes and the pages per minute of the line printer used for output /s :stuck_out_tongue:

(In all seriousness, I am aware of the extremely performance-critical transaction processing application for which the dwindling but still considerable number of s390x and similar type machines handle daily; as a NASA researcher used to working with HPC clusters, I’m obligated to take a dig at my stuffy buisness colleagues, heh)

Performance wise, yeah; I don’t imagine working memory size (as opposed to latency, and to a lesser extent bandwidth) will be particularly critical for that application, and even that is minor, vastly less than that for using Python in the first place and many other tricks; for almost all real applications the choice between def and lambdas should be predicated on maintaining idiomatic usage, i.e. whether the function will be named and used elsewhere, or simply created and immediately passed to another function/method.

1 Like
18

BTW it’s perfectly feasible to run Python on both z/OS and Linux on Z. (Not sure about z/VM.)

So making Python run well on the mainframe is topical.

19

Given how easy it is to dynamically create named closures in Python along with the beauty of list comprehensions, I find myself rarely wanting lambda expressions outside of the kinds of examples that have been given, such as for comparison functions in sortings.

Often times when I find myself wanting to use a lambda expression, I realize that a list comprehension is the better approach.