In many languages trailing closures are a feature designed to improve readability when you need to pass a function a reference to another function to act as a callback, as part of constructing a DSL dynamically or part of functional programming.
Currently I believe the way Python handles these kinds of paradigm goes against the first four lines of PEP 20
Firstly, if you want a function to have a callback you have two options:
- Pass in the name of a function
def callback(*args):
print("do some thing")
other_function(callback)
In this example it’s easy to read but in practice the developer now needs to keep this function and the thing using it, co-located. If code is sufficiently complicated you now need to scroll up and down, as well as keep track of which variables hold values and which are just references to functions.
- Lambdas
other_function(lambda : print("do something"))
This is succinct and keeps it co-located. But python limits it to one line meaning you will end up with a very long line if you tried to do it just using a lambda. Even if python allowed for multiple lines you would end up with “Callback Hell” where the code ends up being hard to read as a result of the the code for the lambda being in the middle of where you are specifying the parameters for the function using that closure.
Secondly, It makes some DSLs more cumbersome to write
Imagine you need to make a decision of what needs to be built at runtime depending on some kind of dynamic value.
This often needs you to write a load of if statements / construct a list of nodes which are then passed into other nodes. Some tools such as Gradio require you to pass in a reference to the function which will update the values of those nodes.
For example here is an example of constructing a markup like document:
document = Node()]
home = document.create_node()
if has_messages:
list = home.create_node()
for message in messages:
message_node = list.create_node()
You can use context managers to assist with making it more markup like:
if has_messages:
with home.create_node() as list:
for message in messages:
message_node = list.create_node()
The problem is that the code in the with block has to be executed along side the code where it is defined, if the DSL only wants to evaluate this part of the tree (For example imagine you get a new message), it cannot. It must evaluate all of it. This is why some tools like Gradio require you to pass references to functions which update some sections of your UI.
How can this be done today
Trailing closures can already be done using Python today, but the syntax isn’t pythonic.
Firstly we must create a closure which will setup a function to accept a trailing closure:
def closure(f):
def activator(*args, **kwargs):
def wrapper(closure):
return f(closure, *args, **kwargs)
return wrapper
return activator
We now apply that decorator to our filter function:
@closure
def filter(closure, array):
yield from [i for i in array if closure(i)]
With this closure decorator decorating our “filter” function now get a new decorator function which will accept arguments to be passed to the original decorated function. When it’s used to decorate the closure, it will pass the closure to and call the original function
names = ['James', 'Josh', 'Micheal']
@filter(names)
def filtered_names(name: str):
return name.startswith('J')
print(list(filtered_names))
Now wie’ve decorated our closure function with the decorated function which accepts the trailing closure. Our original function will now call our closure when it needs.
Due to how python works, when a decorator returns a value, that value will be set at a variable with the same name as the decorated function. Hence why we can print the filtered names as if it was originally a variable.
Unfortunately it’s not very pythonic as unlike @property, it’s not obvious to the use why the decorated function has turned into a filtered list.
However this implementation does have good symmetry with the existing functional methods python has like filter since closure is passed in as the second argument like it is today. In fact since filter is a decorator we can do the same
filter(lambda: False, names)
The Proposal
My proposal is that I would like to make this a first-class element of python’s syntax, I’ve hopefully demonstrated that this can be already supported in python with minimal changes or having to introduce large changes to how the python language functions.
This proposal will introduce, zero new keywords (That’s right!) and hopefully wouldn’t need a closure decorator as proposed above.
I propose that a trailing closure can be indicated by using “with” after a function’s call site. The trailing closure would be passed in as a reference (like functions and lambdas) as the first argument for functions or the second for class methods.
Meaning all the function signature needs to do is to accept a function as it’s first argument (like python’s functional methods such as filter and map already do)
The syntax would work like this, a function call, the “with” keyword followed by the lambda syntax for describing args (and types). The body works with same rules as a python function (Use return to return, yield to yield etc etc)
filter(names) with item: str -> bool:
return item.startswith("j")
And thats it, we would be going from:
filter(lambda item: item.startswith("j"), names)
or
def j_only(item)
return item.startswith("j")
filter(j_only, names)
to just this
filter(names) with item: str -> bool:
return item.startswith("j")
Much easier to read and follow the control flow of the program. In other words more pythonic!
Backwards Compatability:
Existing code would still work since previous Python Versions the functions would just be standard functions which accept a reference to a function or a lambda as their first argument.
This would just be some syntactical sugar for the new python.
