# Backquotes for deferred expression

**URL:** <https://discuss.python.org/t/backquotes-for-deferred-expression/70577>\
**Category:** Ideas\
**Created:** [November 8, 2024, 6:41pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577 "2024-11-08T18:41:11Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [November 8, 2024, 6:41pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/1 "2024-11-08T18:41:11Z")

</div>

Back quotes have been freed from a syntactic sugar for `repr` since the first version of python3, I think it will be perfect to be repurposed for deferred expressions.

### Use cases:

#### 1. Late-bound function arguments

```python
def fn(x: int, y: int, z: int = `x + y`):
    print(z) # expr will be evaluated here

fn(1, 2) # 3

def fn(el: object, z: list = `[]`):
    z.append(el)
    print(z)

fn(1) # [1]
fn(2) # [2]

def task1(next_task=`task2`): ...
def task2(next_task=`task1`): ...
# No NameError!

```

### 2. A new approach of declaring zero-argument lambdas

P.S. This is fundamentally how it behaves: a lambda function with zero argument and auto-evaluates itself upon observation (i.e. no empty parenthesis).

```python
a, b, c = 1, 2, `a + b`

print(c) # 3

a, b = 3, 4
print(c) # 7

```

### 3. A non-backdoor approach for typing forward references

```python
type X = `A | B`

class A: ...
class B: ...

```

## Previous Attempts

1. PEP671 proposed a new operator `=>` that works identical in use case 1. Unfortunately, this proposal have received too many objections and is currently stalled (thanks @Rosuav for sharing the information!).

2. A working prototype of `defer` keyword has been proposed in related topics 2 (see below). The `defer` keyword works analogous to the proposed back quotes. But it (1) takes a longer form and (2) may cause precedence problems for operators and separators _(I might be wrong on this one)_.

3. My own proposal of “deferred arg list” - this does not raise much interest in the community. And the keyword decorator I proposed lacks flexibility and will cause confusions on its behavior.

4. `functools.lazy` and other 3rd party wheels for lazy evaluation (thanks @dg-pb for brining them up!) - they covers most of the cases above. But lack of interpreter support exposed some operations that cannot be properly proxied (such as `is` and `type`). In addition, lack of syntax sugar makes them cumbersome to spell:

## Implementation Details:

1. Collapsible?

2. Observation - namely the `is` and `type` keyword:

3. Typing

* * *

Related Contents:

1. [PEP671](https://peps.python.org/pep-0671/) - Syntax for late-bound function argument defaults
2. [Defer Expressions](https://discuss.python.org/t/defer-expressions/43490) - A working prototype of `defer` keyword.
3. [Another idea on deferred arg list](https://discuss.python.org/t/dynamic-evaluation-of-function-argument-list-initializer/69106) - Dynamic evaluation of function argument list initializer
4. [TypeExpr-specific constructors](https://discuss.python.org/t/why-cant-we/69841/23) - Where this idea originated.
5. [`functools.lazy`](https://discuss.python.org/t/dynamic-evaluation-of-function-argument-list-initializer/69106/9) - A discussion post bringing up proxy tools provided by both stdlib and 3rd party.

> **P.S.**
>
> I know this topic have been revisited many times. I myself stated in my original post “it might be too late to even bring this up for discussion”. However upon second thought I think this idea deserves to be presented to the community - even if it will very likely stall again.

---

<div class="post-metadata">

**Author:** ![Nineteendo](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/nineteendo/32/19122_2.png) [@Nineteendo](https://discuss.python.org/u/Nineteendo)\
**Post date:** [November 8, 2024, 6:56pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/2 "2024-11-08T18:56:52Z")

</div>

How do we write an expression that spans multiple lines?

```python
def fn(el: object, z: list = `[1,
                               2,
                               3]`):
    z.append(el)
    print(z)

```

**OR**

````python
def fn(el: object, z: list = ```[1,
                                 2,
                                 3]```):
    z.append(el)
    print(z)

````

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [November 8, 2024, 6:57pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/3 "2024-11-08T18:57:45Z")

</div>

I see no problem in these examples. Does it break any existing feature?

---

<div class="post-metadata">

**Author:** ![Nineteendo](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/nineteendo/32/19122_2.png) [@Nineteendo](https://discuss.python.org/u/Nineteendo)\
**Post date:** [November 8, 2024, 6:58pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/4 "2024-11-08T18:58:59Z")

</div>

No, but given Python’s convention of 79 character lines, I frequently need to wrap lines.  
So, if the deferred expression is too long, it won’t fit on the line and my IDE will complain.

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [November 8, 2024, 6:59pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/5 "2024-11-08T18:59:50Z")

</div>

I guess it’s at least shorter than `functools.lazy` ?

---

<div class="post-metadata">

**Author:** ![Nineteendo](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/nineteendo/32/19122_2.png) [@Nineteendo](https://discuss.python.org/u/Nineteendo)\
**Post date:** [November 8, 2024, 7:04pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/6 "2024-11-08T19:04:49Z")

</div>

My question is simply: _“if it would be supported or not?”_ \[1\] Regardless of how long something is, you’ll have to wrap at some point. \[2\]

* * *

1. “no” is an acceptable answer 

2. that’s why I import names from modules as often as possible instead of importing the module

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [November 8, 2024, 7:12pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/7 "2024-11-08T19:12:03Z")

</div>

Got it. I am not familiar to AST related stuff. I will leave it as an open question.

---

<div class="post-metadata">

**Author:** ![dg-pb](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dg-pb/32/16248_2.png) [@dg-pb](https://discuss.python.org/u/dg-pb)\
**Post date:** [November 8, 2024, 7:16pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/8 "2024-11-08T19:16:39Z")

</div>

I have been thinking about this for a fair while now. 1.5 years to be precise.

See my early attempt to start (to be more precise to continue on previous discussions) discussion on this: [Deferred Evaluation Initial Proof of Concept](https://discuss.python.org/t/deferred-evaluation-initial-proof-of-concept/48421)

My current take on this is that convenience syntax decisions can be left for the very end.

The major issues of this direction is not in syntax, but in concept and implementation.

Same as with many other `syntax affecting` proposals, my take is that it is best to explore whether good enough solution can be devised via implementing and combining a set of tools that use existing capabilities.

And question “whether `syntax conveniences` are needed” will be much easier to answer once there is a clear idea of what the functionality exactly is and its usefulness.

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [November 8, 2024, 7:27pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/9 "2024-11-08T19:27:22Z")

</div>

> [@dg-pb](#):
>
> The major issues of this direction is not in syntax, but in concept and implementation.

Can you name the obstacles on the implementation side, or point me to a post on them?

The behavior of the backquote syntax has been mostly defined in the OP. I can even write you a python pre-processor that replaces the back quote syntax (`back-qoute<expr>`) into lazy + lambda (`functools.lazy(lambda: <expr>)`) and it will cover a subset of what’s supported by the new syntax.

---

<div class="post-metadata">

**Author:** ![dg-pb](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dg-pb/32/16248_2.png) [@dg-pb](https://discuss.python.org/u/dg-pb)\
**Post date:** [November 8, 2024, 7:33pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/10 "2024-11-08T19:33:22Z")

</div>

The link in that thread doesn’t work for some reason.

> <https://gist.github.com/dg-pb/17005959e487556961ba92e586f8a467#file-pep_deferred_eval_concept-md>

I have briefly explored syntax there too (including backticks). Although I was also initially pro-backticks, but there was feedback of some non-trivial historical complications for such. Also, if multiline expressions are supported, then backticks are awkward and it would be much more pythonic to have a statement-like pythonic-indentation syntax. Such as:

```python
local_var = 1

a = lazy local_var:
    b = local_var + 1
    return b

```

But regardless, this syntax could just be syntactic convenience for:

```python
local_var = 1

def foo(lazy a):
    b = a + 1
    return b

a = functools.lazy(foo, (local_var,))

```

But not necessarily of course. Alternatively implementation could be designed with more weight on parser, but I just don’t think it is a good idea.

---

<div class="post-metadata">

**Author:** ![dg-pb](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dg-pb/32/16248_2.png) [@dg-pb](https://discuss.python.org/u/dg-pb)\
**Post date:** [November 8, 2024, 7:39pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/11 "2024-11-08T19:39:19Z")

</div>

> [@zhangyx](#):
>
> Can you name the obstacles on the implementation side

These are hard to give without at least 70% baked concept.

Also, read-up the thread after reading the document. By now it is quite clear that “late bound argument defaults” and “deferred evaluation” are orthogonal concepts and are best not to be mixed up.

---

<div class="post-metadata">

**Author:** ![dg-pb](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dg-pb/32/16248_2.png) [@dg-pb](https://discuss.python.org/u/dg-pb)\
**Post date:** [November 8, 2024, 7:44pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/12 "2024-11-08T19:44:39Z")

</div>

> [@zhangyx](#):
>
> I can even write you a python pre-processor that replaces the back quote syntax (`back-qoute<expr>`) into lazy + lambda (`functools.lazy(lambda: <expr>)`) and it will cover a subset of what’s supported by the new syntax.

Agreed. It would be good to see robust `functools.lazy` implementation first.

I have done some initial drafts in Python and think it is possible, but the complexity of implementing such in `C` is greater by the order of magnitude. Have a look at proxy objects in `wrapt` package. And higher level of robustness, consideration for various nuances, etc, would be needed when implementing this in CPython stdlib.

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [November 8, 2024, 7:47pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/13 "2024-11-08T19:47:24Z")

</div>

> [@dg-pb](#):
>
> ```python
> local_var = 1
> 
> a = lazy local_var:
> b = local_var + 1
> return b
> 
> ```

A decorator can convert a vanilla function into a deferred variable for you:

```python
def defer_expr(fn):
    return `fn()`

local_var = 1

@defer_expr
def a():
    b = local_var + 1
    return b

```

This eliminates the need of a keyword. I don’t think the C++ style “[capture clause](https://learn.microsoft.com/en-us/cpp/cpp/lambda-expressions-in-cpp?view=msvc-170#capture-clause)” will help with performance nor memory footprint.

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [November 8, 2024, 7:51pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/14 "2024-11-08T19:51:32Z")

</div>

I need a bit more time to finish reading, there are a lot of contents and posts that you pointed to. I might comeback with more questions when I am finished. (Somehow I missed the thread completely when doing my research writing this post)

Form what I’ve seen, you have already done a ton of impressive work on this. Really appreciate that you brought them up here!

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [November 8, 2024, 8:05pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/15 "2024-11-08T20:05:13Z")

</div>

> [@zhangyx](#):
>
> This eliminates the need of a keyword. I don’t think the C++ style “[capture clause](https://learn.microsoft.com/en-us/cpp/cpp/lambda-expressions-in-cpp?view=msvc-170#capture-clause)” will help with performance nor memory footprint.

If you really need a snapshot reference to a variable:

```python
local_var = 1

@defer_expr
def a(x=local_var):
    b = x + 1
    return b

```

Early-bound function defaults will work for you.

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [November 8, 2024, 8:20pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/16 "2024-11-08T20:20:19Z")

</div>

> [@dg-pb](#):
>
> implementing such in `C` is greater by the order of magnitude

I am super interested. Unfortunately I do not have the expertise to do this alone. I would really appreciate if someone with enough knowledge can guide me through this and make it happen.

---

<div class="post-metadata">

**Author:** ![dg-pb](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dg-pb/32/16248_2.png) [@dg-pb](https://discuss.python.org/u/dg-pb)\
**Post date:** [November 8, 2024, 9:07pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/17 "2024-11-08T21:07:37Z")

</div>

Lets take your use cases:

1. As I said, late-bound-defaults is not part of this.

```python
def foo(a, b, c=`a+b`):
    ...

```

would not work, because `a` and `b` are not visible when `c` default is assigned. This is a separate concept as per PEP671.

1. A new approach of declaring zero-argument lambdas

This is a weak motivation as `zero-argument lambdas` can be done using `lambda` with no arguments. In other words, effort required for this is much greater than this benefit.

Or to expose its usefulness, it would be good to see real life applications of this that show the benefit.

1. A non-backdoor approach for typing forward references

I don’t use typing, but this one seems useful to me.

* * *

* * *

Personally, I am going in a slightly different direction than your proposal.

However, a possible common denominator is `functools.lazy`, which from my POV would be a good starting point.

And it is much easier sell than full proposal with new syntax.

And given it proves to be useful, combined with the fact that implementation exists and is functioning well, selling syntactic convenience would be much easier.

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [November 8, 2024, 9:15pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/18 "2024-11-08T21:15:33Z")

</div>

> [@dg-pb](#):
>
> ```python
> def foo(a, b, c=`a+b`):
> ...
> 
> ```
> 
> would not work

Good catch. I did not elaborate on this since it will open up too much complexity. I probably shouldn’t use it to sell this proposal at all.

> [@dg-pb](#):
>
> This is a weak motivation as `zero-argument lambdas` can be done using `lambda` with no arguments. In other words, effort required for this is much greater than this benefit.

I agree. Instead of a selling point, this section is more about how to “describe” it - a “mental model” to help explain the new syntax to people who knows python but does not know this syntax.

In addition, if this proposal is lucky enough to get a chance for prototyping, I bet it will reuse a lot of infrastructures made for lambda functions.

---

<div class="post-metadata">

**Author:** ![Rosuav](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/rosuav/32/3429_2.png) [@Rosuav](https://discuss.python.org/u/Rosuav)\
**Post date:** [November 8, 2024, 10:13pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/19 "2024-11-08T22:13:39Z")

</div>

> [@zhangyx](#):
>
> Back quotes have been freed from a syntactic sugar for `repr` since the first version of python3

They weren’t really. They were listed in PEP 3099 as something that wasn’t going to be used. So you’ll have to explain what’s changed since then that means that they are now available for use.

But as with every proposal for magical deferred expressions, the key is semantics. Let’s look at yours:

> [@zhangyx](#):
>
> ```python
> def fn(x: int, y: int, z: int = `x + y`):
> print(z) # expr will be evaluated here
> 
> ```

Does it simply insert the text of the expression\[1\] at that point? What variable scopes are available to it?

> [@zhangyx](#):
>
> ### 2. A new approach of declaring zero-argument lambdas
> 
> P.S. This is fundamentally how it behaves: a lambda function with zero argument and auto-evaluates itself upon observation (i.e. no empty parenthesis).

Does it capture variables or doesn’t it? If it doesn’t, this is so completely different from a lambda function that it’s only going to cause confusion to talk about them in parallel. If it does, it won’t work for function defaults.

And the biggest problem with every “magical deferred” proposal: WHEN does it get evaluated? What are the semantics of this “deferred object”? For example, if I do this:

```python
x = `a + b`
print(x)
func(x)

```

what happens? Does it evaluate more than once? Does the first one print out the DeferExpr but the second one evaluate it? Does it capture the values of `a` and `b` from the caller’s scope, or does it reach into the callee’s scope to fetch them? This is a HUGE area that you will need to explain in detail. The biggest downfall of every “deferred object” proposal I’ve seen has been the inability to precisely define these semantics.

This is why PEP 671 never said anything of the sort. There is no deferred object. It is a syntactic structure that looks at the lack of an argument and executes some code.

* * *

1. or more precisely, its syntax tree - we don’t want C-style macro insanity

---

<div class="post-metadata">

**Author:** ![zhangyx](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/zhangyx/32/23808_2.png) [@zhangyx](https://discuss.python.org/u/zhangyx)\
**Post date:** [November 8, 2024, 10:23pm UTC](https://discuss.python.org/t/backquotes-for-deferred-expression/70577/20 "2024-11-08T22:23:12Z")

</div>

> [@Rosuav](#):
>
> Does it simply insert the text of the expression[[1]](#footnote-205137-1) at that point? What variable scopes are available to it?

> [@Rosuav](#):
>
> Does it capture variables or doesn’t it?

The expression is immediately parsed and should be converted to byte code just like any other code. Suppose you have syntax error inside it, then it should be raised during syntax evaluation time, not runtime.

The expression will capture the locals and globals at **where it’s declared**. It’s covered in the OP under Implementation Details 2.1.

> [@Rosuav](#):
>
> Does it evaluate more than once?

This has been addressed in Implementation Details 1 (_collapsible_). I made it clear that it remains to be discussed, but favored no-collapsing and provided solutions for corner cases.

[Next page](https://discuss.python.org/t/backquotes-for-deferred-expression/70577.md?page=2)