# PEP 701 – Syntactic formalization of f-strings

**URL:** <https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046>\
**Category:** PEPs\
**Created:** [December 19, 2022, 5:53pm UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046 "2022-12-19T17:53:22Z")\
**Posts on this page:** 20\
**Page:** 4

<div class="post-metadata">

**Author:** ![guido](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/guido/32/21_2.png) [@guido](https://discuss.python.org/u/guido)\
**Post date:** [December 22, 2022, 2:36am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/62 "2022-12-22T02:36:48Z")

</div>

It feels oddly asymmetrical that the opening quote is present, but the closing quote is implicit. Does it really have to be that way?

---

<div class="post-metadata">

**Author:** ![pablogsal](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pablogsal/32/32246_2.png) [@pablogsal](https://discuss.python.org/u/pablogsal)\
**Post date:** [December 22, 2022, 2:42am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/63 "2022-12-22T02:42:06Z")

</div>

> [@guido](#):
>
> Does it really have to be that way?

No, but it makes the current implementation easier and is not visible (at least from the C tokenizer).

In any case, the tokens in the PEP are there just for reference to understand what they do but they are an implementation detail. For the tokenizer module, we can obviously think of something that makes more sense as a public API.

---

<div class="post-metadata">

**Author:** ![barry](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/barry/32/42_2.png) [@barry](https://discuss.python.org/u/barry)\
**Post date:** [December 22, 2022, 3:45am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/64 "2022-12-22T03:45:42Z")

</div>

> [@achhina](#):
>
> even less so with syntax highlighting

I think we need to be careful relying on syntax highlighting as a cure for confusing syntax. I use and rely on syntax highlighting, but what about visually impaired programmers or other folks for which highlighting might not help or be available?

I do think it gets worse with nested f-strings, as in this PEP example:

```auto
f"{f"{f"infinite"}"}" + " " + f"{f"nesting!!!"}"

```

That’s incredibly difficult for a human to parse IMHO. If this is the price we have to pay for consistent implementation, and some arguably useful features \[1\], then we should ensure that the tools, from the Python parser, to editors, to linters, are aligned about how to help the user understand these complex f-strings.

* * *

1. although TBH I’ve very rarely been hit by such f-string limitations and they are usually quite easy to work around

---

<div class="post-metadata">

**Author:** ![steven.daprano](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steven.daprano/32/1083_2.png) [@steven.daprano](https://discuss.python.org/u/steven.daprano)\
**Post date:** [December 22, 2022, 4:35am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/65 "2022-12-22T04:35:59Z")

</div>

> [@](#):
>
> Gotchas in basic features, however, are not one of them.

Really? How about `i + 1` in Python compared to C, where `i` is an int in both languages?

In case the gotcha isn’t obvious, what if `i = 2147483647`?

The current restrictions on f-strings are not what I would call a gotcha. If anything, they are (minor?) annoyances – things that cause an explicit error when you try them, not things which silently do the wrong thing.

> [@](#):
>
> I don’t think anyone would disagree that mutable defaults gotcha is a regrettable part of Python

I disagree.

I think that Python absolutely made the right choice here. If a language provides only one of early-bound and late-bound default values for functions, I state that Python’s choice of early binding is _by far_ better. But that’s an argument for another discussion 🙂

Bringing the discussion back on topic:

> [@](#):
>
> Banning re-use of quotes is just that, a gotcha.

Can you give an example of code which silently does the wrong thing instead of raising an exception because of the lack of re-use of quotes?

If it really is a gotcha, causing code to silently misbehave, that may push me from “oppose re-use of quotes” to “support re-use”. (Depending on just how contrived or realistic the gotcha is.)

---

<div class="post-metadata">

**Author:** ![stoneleaf](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/stoneleaf/32/88_2.png) [@stoneleaf](https://discuss.python.org/u/stoneleaf)\
**Post date:** [December 22, 2022, 6:10am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/66 "2022-12-22T06:10:44Z")

</div>

> [@monk-time](#):
>
> I don’t think you’re asking this question in good faith

Then don’t engage with me. I certainly have no desire to argue my case with someone who thinks so little of me.

> [@monk-time](#):
>
> I don’t think anyone would disagree that mutable defaults gotcha is a regrettable part of Python

I disagree. Mutable defaults serve a valuable purpose.

> [@monk-time](#):
>
> Banning re-use of quotes is just that, a gotcha.

How? Misusing mutable defaults can be hard to track down, but trying to misuse f-strings by resusing the quotes is hardly difficult to find and fix.

---

<div class="post-metadata">

**Author:** ![markshannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/markshannon/32/72_2.png) [@markshannon](https://discuss.python.org/u/markshannon)\
**Post date:** [December 22, 2022, 10:05am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/67 "2022-12-22T10:05:11Z")

</div>

Personally, I found the prohibition on reusing the same quote mildly annoying.  
`f"You have the following in your basket: {", ".join(items)}."` seems perfectly fine to me.

But I think I would find the restriction much more annoying if I knew that it was unnecessary, and that extra work had been put in just to stop me.

---

<div class="post-metadata">

**Author:** ![malemburg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/malemburg/32/50_2.png) [@malemburg](https://discuss.python.org/u/malemburg)\
**Post date:** [December 22, 2022, 11:12am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/68 "2022-12-22T11:12:08Z")

</div>

Couldn’t resist…

```auto
def bart(foo):
    f"""{(

    foo := foo + 1,
    print('Finally, curly brackets in Python ;-)'),
    [
        print('Teacher: You should not use full Python in f-strings.')
        for i in range(foo)
    ],
    print(f'But you can... {foo}-)')

)}"""

bart(7)

```

…

```nohighlight
Finally, curly brackets in Python ;-)
Teacher: You should not use full Python in f-strings.
Teacher: You should not use full Python in f-strings.
Teacher: You should not use full Python in f-strings.
Teacher: You should not use full Python in f-strings.
Teacher: You should not use full Python in f-strings.
Teacher: You should not use full Python in f-strings.
Teacher: You should not use full Python in f-strings.
Teacher: You should not use full Python in f-strings.
But you can... 8-)

```

This is already possible in 3.10, but I strongly doubt that it was ever intended. Should we really head on in the same direction even more ?

Just to be clear: I like the PEP and support it, since it simplifies the implementation and removes some annoying bits (e.g. the backslash limitation), but I don’t like the removal of the string literal quoting rules and support for arbitrary nesting of f-strings. By keeping the quote restriction, arbitrary nesting can be prevented.

---

<div class="post-metadata">

**Author:** ![arhadthedev](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/arhadthedev/32/6420_2.png) [@arhadthedev](https://discuss.python.org/u/arhadthedev)\
**Post date:** [December 22, 2022, 4:13pm UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/69 "2022-12-22T16:13:23Z")

</div>

> [@markshannon](#):
>
> `f"You have the following in your basket: {", ".join(items)}."` seems perfectly fine to me.

I guess the problem arises on scanning code swiftly, diagonally jumping from one quote/bracket/colon to another in a hot potato style, reading everything else with a side vision.

Meanwhile, reading the code sequentially, line by line left to right causes absolutely no problem.

So I guess that the quote reuse punishes some reading styles that perfectly worked before.

---

<div class="post-metadata">

**Author:** ![dumbpotato](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dumbpotato/32/10305_2.png) [@dumbpotato](https://discuss.python.org/u/dumbpotato)\
**Post date:** [December 22, 2022, 6:26pm UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/70 "2022-12-22T18:26:00Z")

</div>

Having read through everything here, I still believe that the mere possibility of having deeply nested f-strings(and the re-use of quotes) is not a good enough reason to disallow it on the parser level.

Ultimately(in my opinion), such restrictions belong in linters and code formatters rather than in the language grammar itself.

---

<div class="post-metadata">

**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\
**Post date:** [December 22, 2022, 8:10pm UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/71 "2022-12-22T20:10:56Z")

</div>

> [@holdenweb](#):
>
> It’s a putative language development. Editor vendors are used to having to track those - you might almost say it comes with the territory.

This is a bit of a silly excuse. To take an adversarial example, C++ is so complicated to parse that the only reasonable way to parse it reliably is to reuse an existing compiler frontend. Which is a concrete impediment to the development of an ecosystem of third-party source-level analysis (and/or transformation) tools.

---

<div class="post-metadata">

**Author:** ![cfbolz](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/cfbolz/32/23913_2.png) [@cfbolz](https://discuss.python.org/u/cfbolz)\
**Post date:** [December 22, 2022, 8:32pm UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/72 "2022-12-22T20:32:19Z")

</div>

Wearing my PyPy hat: Overall I like the PEP from a parsing implementation point of view, because it removes a lot of complexity in the current f-string implementation, and likely reduces bugs around position information of the expressions within f-strings as well. Well done @pablogsal, @isidentical and @lys.nikolaou!

My comment is about the changes to tokenizing (I’m staying out of the language design questions). Given that some of the complexity of implementing f-strings moves there, I think this part of the PEP:

> How the lexer emits this token is **not specified** as this will heavily depend on every implementation

is taking the easy way out. I agree that it doesn’t make sense to prescribe details about the tokens (like whether the starting/ending quotes should be part of them). But I think the PEP should add some detail about how this can be implemented in a tokenizer, because the new approach to tokenizing f-strings is another Python feature that requires leaving classical regular-language-based tokenization.

One way to do this is either to add some detail about how it works in CPython’s real tokenizer. Another option would be to add some Python code to the PEP that shows how it could be done in the pure-Python tokenization implementation in the `tokenize` module.

Speaking of the pure-Python `tokenize` stdlib module. From what @pablogsal wrote [above](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/61) (and the following comments) it sounded like the plan is to decide what to do about `tokenize` independently of this PEP. If that’s the plan, the PEP should mention it.

---

<div class="post-metadata">

**Author:** ![replabrobin](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/replabrobin/32/8686_2.png) [@replabrobin](https://discuss.python.org/u/replabrobin)\
**Post date:** [December 23, 2022, 10:20am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/73 "2022-12-23T10:20:17Z")

</div>

Will the change to peg parsing improve the speed of f strings. I notice that in python311/312 I see a speed difference between simple % formatting and f strings. As example here are some timeit outputs for several pythons

$ for v in 37 38 39 310 311 312; do echo -n “python$v %: “;python$v -mtimeit -s’a,b,c=1,2.1,“ABCDEF”’ '”%s %s %s”%(a,b,c)‘;echo -n "python$v f: ";python$v -mtimeit -s’a,b,c=1,2.1,“ABCDEF”’ ‘f"{a} {b} {c}"’;done  
python37 %: 500000 loops, best of 5: 446 nsec per loop  
python37 f: 500000 loops, best of 5: 454 nsec per loop  
python38 %: 500000 loops, best of 5: 411 nsec per loop  
python38 f: 500000 loops, best of 5: 427 nsec per loop  
python39 %: 1000000 loops, best of 5: 396 nsec per loop  
python39 f: 500000 loops, best of 5: 410 nsec per loop  
python310 %: 500000 loops, best of 5: 407 nsec per loop  
python310 f: 500000 loops, best of 5: 408 nsec per loop  
python311 %: 1000000 loops, best of 5: 302 nsec per loop  
python311 f: 1000000 loops, best of 5: 368 nsec per loop  
python312 %: 1000000 loops, best of 5: 307 nsec per loop  
python312 f: 1000000 loops, best of 5: 363 nsec per loop

so it seems that recently % formatting speeded up, but f formatting didn’t so much

---

<div class="post-metadata">

**Author:** ![malemburg](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/malemburg/32/50_2.png) [@malemburg](https://discuss.python.org/u/malemburg)\
**Post date:** [December 23, 2022, 10:58am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/74 "2022-12-23T10:58:54Z")

</div>

> [@dumbpotato](#):
>
> Having read through everything here, I still believe that the mere possibility of having deeply nested f-strings(and the re-use of quotes) is not a good enough reason to disallow it on the parser level.
> 
> Ultimately(in my opinion), such restrictions belong in linters and code formatters rather than in the language grammar itself.

That’s a fair point.

Perhaps it would be enough to rephrase the PEP to not encourage such use and instead refer to it as an implementation detail. At the moment, it reads in a way which promotes reusing the same quotes inside f-string expressions as a feature.

---

<div class="post-metadata">

**Author:** ![pablogsal](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pablogsal/32/32246_2.png) [@pablogsal](https://discuss.python.org/u/pablogsal)\
**Post date:** [December 23, 2022, 7:24pm UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/75 "2022-12-23T19:24:58Z")

</div>

Thanks, everyone for your feedback and for raising your concerns. Your thoughts are fundamental to us and we want to ensure they are correctly reflected in the document and considered when decisions are made. I have prepared [a first PR](https://github.com/python/peps/pull/2939) to incorporate some of the important points raised so far. Apologies if I am missing something, I will try to make another pass at a later date. Also, feel free to open PRs yourself if you have any enhancements in mind to the document and we can discuss them in the PR.

Thanks a lot for your help and your insights!

---

<div class="post-metadata">

**Author:** ![sweeneyde](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sweeneyde/32/3689_2.png) [@sweeneyde](https://discuss.python.org/u/sweeneyde)\
**Post date:** [December 23, 2022, 8:11pm UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/76 "2022-12-23T20:11:24Z")

</div>

Make sure you compare `'%s'` to `'{x!s}'` or compare `'%r'` to `'{x!r}'`. In 3.11-3.12, the `dis` module is showing that these typically lead to identical bytecode, so they should have exactly the same performance. As far as I know, this PEP will result in no bytecode changes and thus have no runtime performance effect on existing code, though it might be worth spelling that out explicitly.

---

<div class="post-metadata">

**Author:** ![pablogsal](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pablogsal/32/32246_2.png) [@pablogsal](https://discuss.python.org/u/pablogsal)\
**Post date:** [December 23, 2022, 8:17pm UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/77 "2022-12-23T20:17:48Z")

</div>

> [@sweeneyde](#):
>
> might be worth spelling that out explicitly.

Will do 👍

---

<div class="post-metadata">

**Author:** ![mem\_dixy](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/mem_dixy/32/10316_2.png) [@mem\_dixy](https://discuss.python.org/u/mem_dixy)\
**Post date:** [December 24, 2022, 3:58am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/78 "2022-12-24T03:58:39Z")

</div>

Personally I dislike the idea of nesting f-strings. Preventing it should be easy. Just check that you don’t see another FSTRING\_START tag before seeing a FSTRING\_END tag. Otherwise we might end up with code like this:

> `print(f"({print(f"[{print("print")}]")})")`

But I do like the idea of making the parser simpler. And the code golf people might like it. So maybe it will have to be one of those “Just because you can, doesn’t mean you should!” kind of things?

---

<div class="post-metadata">

**Author:** ![UltimateLobster](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ultimatelobster/32/9727_2.png) [@UltimateLobster](https://discuss.python.org/u/UltimateLobster)\
**Post date:** [December 24, 2022, 8:47am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/79 "2022-12-24T08:47:08Z")

</div>

This PEP looks great! I would be happy to see it implemented 😃

However, I think that the abillity to reuse quotes is problematic. I’ve seen people here mention potential security/IDE maintainability issues. But I think that just from the simple readability aspect, this can be quite confusing and unintuitive (Which I believe to be the most important aspect to consider).

The `{}` syntax in f strings just doesn’t “feel” like it should be able to do it. For example, if the original syntax for f-strings would consist of the backtick character instead of the brackets, I would say otherwise since the backtick “feels” quote-like, it feels like we’re using a different kind of quote and therefore it should be possible. But the bracket character does not have this intuitive property (at least for me).

---

<div class="post-metadata">

**Author:** ![TeamSpen210](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/teamspen210/32/1004_2.png) [@TeamSpen210](https://discuss.python.org/u/TeamSpen210)\
**Post date:** [December 24, 2022, 8:57am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/80 "2022-12-24T08:57:28Z")

</div>

Preventing nesting entirely wouldn’t be backwards compatible, f-strings can already be nested if you use different quote styles for each level. I wonder if a bit of a compromise might be possible though? Infinite nesting could be allowed, but with the restriction that any quotes can’t reuse the immediately surrounding quote type. The switching quotes might make it easier to match each up:

```nohighlight
>>> f"{f'{f"{f'{f"{f'{1+1}'}"}'}"}'}"

```

---

<div class="post-metadata">

**Author:** ![replabrobin](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/replabrobin/32/8686_2.png) [@replabrobin](https://discuss.python.org/u/replabrobin)\
**Post date:** [December 24, 2022, 9:37am UTC](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046/81 "2022-12-24T09:37:23Z")

</div>

Good idea; the timings seem much closer for the later pythons when we explicitly specify the f format.

[Previous page](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046.md?page=3)

[Next page](https://discuss.python.org/t/pep-701-syntactic-formalization-of-f-strings/22046.md?page=5)