PEP 822: Dedented Multiline String (d-string)

Do you think that sentence is sufficient to explain the need to describe the idea of keeping only the newline without content, while already describing the idea of allowing content after the opening quote?

Allowing content after opening quote is more consistent.
And the reason that idea is rejected would just repeat most of the reasons for rejecting the idea of allowing content immediately after the opening quote.

After reading the open PR, I think it is the same topic. I’d like it to be clearer that this is a question of being consistent with other multiline strings or not.

I’m not totally satisfied with the emphasis in the new text, but the topic is covered, which is the main thing I care about.

I do see use cases for writing on the first line of the d-string. It’s more compact, more similar to the way we currently write docstrings, easier to switch…

But I’m convinced linters will fix those problems before they ever get to me. ā€œAdd a new line to the start of a d-string if it doesn’t start with a new lineā€ doesn’t seem like a hard rule to add.

OTOH, I think keeping the door open for annotated d-strings, something in the direction of

d"""#SQL
select statement from table;
"""

is valuable.

(Maybe there’s other better things to use that first line for. I think we’ll have a better view on that in the future regardless.)

1 Like

I’ve been trying to exhaustively envision the use cases of d-strings… and at the end I think there are two ways they can be used cleanly : a ā€˜vertical flavor’ and a ā€˜horizontal flavor’…

The vertical flavor wants to copy paste snippets of text into indented code and add the quotes over and under, at the end it wants to concatenate vertically (thus the last newline is actually convenient)

... :
     snippet = d"""
       -  Lorem ipsum
           Bla bla bla
     """

text += snippet

The horizontal flavor wants to define blocks and incorporate them afterwards in a template, it probably wants to keep the number of lines minimal, and does not require the indentation level to be defined by the closing quotes (because it wants to reindent in the template) :

... :
    statement = d"""#some comment
    do_something()"""

... :
    instructions = df"""
    for in in range(n):
        {statement}
    """"

(Note ; Here the remaining problem is the multiline reindentation of {statement} in the template.)

I don’t think there are really transverse ways (besides the vertical and horizontal flavors) for using d-strings properly.
→ If I’m right, keeping the last newline should actually be more convenient… and ā€˜transverse flavors’ that would require to remove it would be anecdotical and clumsy, thus discouraged intrinsically by the syntax, and the python neatness will be preserved.

1 Like

I feel torn on this PEP for a few reasons:

  1. I very much prefer the leading \n removal, but I’m unsure of the leading-but-not-trailing \n removal
  2. I sometimes wish I could embed a multi-line string within another multi-line string and have both automatically dedented (I made a t-string-powered library to demonstrate this wish) and I hoped this PEP might accomplish this but it does not (deliberately it seems, as there’s a complexity trade off)
  3. I teach Python and this seems like a syntax I would want my students to know about but the mental model for this doesn’t seem nearly as intuitive to teach as traditional multi-line strings (both due to point 1 and 2 and because dedenting is difficult to reason about in general)

All that said, I really like the idea of improving the dedenting mechanisms in Python, whether through a string method, a new syntax, or just an enhancement to textwrap.dedent

On leading & trailing newline removal

I find myself copy-pasting these two functions around between various projects:

This dedent version that removes the leading newline only:

def undent(text):
    return dedent(text).removeprefix("\n")

And this version which strips a trailing newline as well (like inspect.cleandoc):

def undent(text):
    return dedent(text).removeprefix("\n").removesuffix("\n")

It seems that I use the first version most of the time but I use the second version (to remove both prefix and suffix \n) about one-third of the time.

It would be nice to avoid using .rstrip() with d-strings, but that would make my primary use case more awkward.

On dedenting and re-indenting replacement fields

Given this string:

code = r"""
def strip_each(lines):
    new_lines = []
    for line in lines:
        new_lines.append(line.rstrip("\n"))
    return new_lines
""".strip("\n")

I would love it if this:

example = d"""
    Example function:

        {code}

    That function was indented properly!
"""

Resulted in this:

>>> print(example)
Example function:

    def strip_each(lines):
        new_lines = []
        for line in lines:
            new_lines.append(line.rstrip("\n"))
        return new_lines

That function was indented properly!

But I understand that this could complicate the mental model even further, especially raising questions of ā€œwhat about replacement fields that are mid-lineā€.

On teaching this to beginners

The ā€œHow to teach thisā€ section doesn’t currently address teaching this to new Python programmers.

I suspect that I would teach this most often to folks who have never heard of textwrap.dedent and who may never hear of it (if d-strings become successful enough to supersede its use).

My main concern with teaching this to beginners is explaining how it works. The prefix/suffix newline removal feels a bit magical and I’m suggesting even more (likely unfeasible) magic above, but dedenting is also a bit magical, especially if a {...} replacement field includes a string that contains a newline.

I hope some of the above concerns may be useful to consider. I know I’ve repeated/overlapped at least a bit with previously expressed concerns.

Thanks for pushing this idea forward in various forms over the years @methane! I’ve found what I’ve read from the previous threads inspiring.

9 Likes

Hello @methane !

The Python Steering Council has reviewed PEP 822. While we’re generally positive about the PEP, we think it would be best to defer this feature until Python 3.16, to give it more time to solidify.

The main feedback we have is that this feature is baking into the language specific formatting style and expectations, which would be very difficult to change if we get some of the corner cases wrong. We note that the discussions about this PEP still express different opinions and we feel like consensus and convergence hasn’t been reached. Rather than prematurely approve this PEP, we want to give it more time and think that a Python 3.16 target release provides that confidence that we’re making the right choices.

Some details:

  • We’re not sure whether the semantics regarding newline stripping are going to be more useful or more of a wart that people will have to workaround.
  • Are the special rules that people have to learn (e.g. newline only after a d-string’s opening triple quoted string, the asymmetry between single quoted and triple quoted strings) easy to learn and remember, or hard to remember and easy to get wrong.
  • We’d like to see how useful d-strings would be in the standard library. We’re not saying that the stdlib should be migrated to use d-strings, but we think that would be a useful exercise to learn if the choices the PEP makes are the best they can be.
  • Adding a new string prefix is a high price, perhaps worth paying if the value it provides is large, but we don’t feel that bar has been reached yet.

Can we also find some projects where d-strings would help a lot and get some testimonials from their maintainers? We think that experience would help a lot in establishing confidence that we’re adding a language feature that truly helps, or just another way to do something similar to the alternatives, but a bit different.

The PSC thanks you for your work on this and urges you to continue to let this idea bake. We look forward to its resubmission for Python 3.16.

19 Likes

IMO it’s easy, since, if I have to dedent a multistring, also the first line must be indented.

My 0.0.1 cents.

PS: I’m happy the PEP is deferred and not rejected. :slight_smile:

1 Like