PEP 822: Dedented Multiline String (d-string)

The more idiomatic way to do this is to call the splitlines method instead:

letters = '''\
a
b
c
'''.splitlines()

And len(letters) would be 3 as expected.

3 Likes

A left-aligned multiline string means exactly what you say it means. I’m not good at English, so I’m writing English sentences using AI translation. If there is a clearer term, please teach me it.

Your sample code only has one level of indentation, so left-aligned string is not a problem.
But as I wrote in the Motivation section, “When writing multiline string literals within deeply indented Python code”, it can be matter.

class MyClass
     ...
    def method(self, ...):
        ...
        try:
            ...
            for ... in ...:
                ...
                if ... :
                    html = Markup("""
<html>
    <p>
        I don’t start from the code’s indentation level
    </p>
</html>
                    """.strip())

Some people doesn’t code like this. That’s why textwrap.dedent() used much in CPython testcode.

1 Like

分かりました、ありがとうございます。I think the term works, I never thought of a way to call it before so I was unsure I got the right meaning, but as far as I know everyone else understood it, so it’s fine.

I don’t think deeply indented, left-aligned strings are a big issue, but I do support the idea of a d-string, because I know left-aligned strings aren’t very popular (my colleagues at work for example don’t like them). Specifically, I support them with a syntax like the suggested d""" and would not want to see completely new syntax to support them.

IMO the ground has been trod pretty thoroughly here. Does it make sense for the trailing newline discussion to go into a Rejected Ideas section? If people agree that their position is either adopted by the PEP or fairly represented in the Rejected Ideas, then it’s up to the SC to decide whether to adopt the PEP as-is or request to switch.

Along those lines, I think the from __future__ alternative should mention its benefits in addition to its reasons for rejection. “This could help simplify Python’s grammar in the future” is the smallest of them, IMO. There are already two interpretations: regular multiline strings and docstrings. Further, the PEP itself is evidence that there is a desire for another interpretation. Using from __future__ would allow a transition to a single consistent interpretation, which benefits users, teachers and students. I do think that “a transition is practically impossible” is a valid argument against it (although I disagree), so it’s fine for the PEP to take the position that three interpretations is the best we can hope to achieve.

It would also be worth noting when citing the vote for new syntax over __future__, the poll did not permit multiple acceptable choices:

Thanks for the PEP!

5 Likes

About this, since the opening triple quotes needs to be followed by a newline character, usind d-srtrings in docstrings would look like

def complex(real=0.0, imag=0.0):
    d"""
    Form a complex number.

    Keyword arguments:
    real -- the real part (default 0.0)
    imag -- the imaginary part (default 0.0)
    """
    ...

which violates PEP 257:

Multi-line docstrings consist of a summary line just like a one-line docstring, followed by a blank line, followed by a more elaborate description.

Since leading whitespace is already stripped from docstrings anyway (since 3.13), I suppose they can be considered “implicit d-strings” and thus the d prefix is not needed?

The docstring use case is not currently mentionned in the PEP, I think it should (to either discourage its usage, or explain how it plays with PEP 257).

3 Likes

PEP 257 is so widely violated that I consider it a failure — or at best a mere suggestion.

I like the idea of using d-strings for docstrings, and maybe the d-semantics should implement whatever tools people use to strip whitespace from docstrings today.

8 Likes

This seems right. We can add to the doc that d-strings are transforming string just like doscstrings implicitly do.

In this case it might be possible to allow not having the initial line return, but this might be discouraged for esthetical reasons.
The ‘header line’ indentation level should be considered being at the same level as the closing triple quote. For example :

def function():
    return d"""#comment
    with contextmanager:
        statement()
    """

The function would return

#comment
with contextmanager:
    statement()

I mostly agree. I’m sure @methane has thought about this plenty between the Ideas thread and authoring the PEP. And we can definitely trust the SC review process to examine the options carefully.

The choice of which rules are used should be explained in the PEP, and I agree that Rejected Ideas is a great place to do most of this. But I don’t find “intuitiveness” to be a great justification, since it’s very subjective on its own. Plus, I find the proposed rules a bit unintuitive.

So I think it is worth challenging this a bit, if only to produce a better document which covers the options well.

I like the idea of matching the dedent rules which are used for __doc__ in theory, but I don’t think it works. inspect.cleandoc[1] is too aggressive about stripping whitespace for general use:

All leading whitespace is removed from the first line. Any leading whitespace that can be uniformly removed from the second line onwards is removed. Empty lines at the beginning and end are subsequently removed. Also, all tabs are expanded to spaces.

__doc__ assignment does a lot less. But __doc__ ignores leading whitespace on the first line, which can produce some odd things:

def f():
    """\
    hello
      world
    """  # does not preserve a 2-space indent for 'world'

If the rules are “matches __doc__ except for X, Y, Z”, I’m not sure how convincing that is. But matching __doc__ exactly seems like it wouldn’t be acceptable, since that leading line behavior is guaranteed to confuse people.


  1. Assuming that’s the standard to follow? ↩︎

7 Likes

Of course. I am currently working on updating the reference implementation before updating PEP, so it will take some time until the PEP update.
The PEP update I’m considering is as follows:

  • Add the “removing last newline” in the rejected ideas.
  • Allow closing quote after non-empty line to remove last newline.
  • Remove “common longest indent” instead of indent of closing quote, like Julia does.
6 Likes

I like those changes. In particular, although using the indent of the closing quotes as a dedentation level sounds like a cool feature, I can’t really think of a good real-world scenario where I actually need to define a multiline string with a certain level of indentation for all lines. Removing this feature also allows the usage to follow the more prevalent styling convention of indenting the content of a multiline string with one more level than the starting statement and the closing quotes:

letters = d'''
____a
____b
____c
''' # letters == 'a\nb\nc\n'

as opposed to the weirder styling we’d have to adopt if this feature makes it to d-string:

letters = d'''
a
b
c
''' # letters == 'a\nb\nc\n'

or:

letters = d'''
____a
____b
____c
____''' # letters == 'a\nb\nc\n'

Like Julia, I will calculate indent including closing quote line. So you still need to write

letters = d'''
____a
____b
____c
____''' # letters == 'a\nb\nc\n'
4 Likes

Ah OK so I misread your third point. I’m fine with the proposal with or without this feature.

This is the PR to update this PEP. Please review it.

Note that PR is used only for review. Please keep using this thread for discussion of the PEP.

4 Likes

I think “Keep opening newline” should be added to rejected ideas, just like you added “Removing newline in the last line”.

The benefit of the idea of removing the leading newline character, as opposed to the trailing one, is almost self-evident.

On the other hand, the benefits of removing the initial line break are unclear. I understand that you think it is more symmetrical and consistent, but I do not agree. If you could write some more objective benefits that anyone would understand, I will add them to Rejected Ideas section.

Sorry if I missed it (I looked back through the thread and didn’t see any obviously relevant post), but why was the PEP now switched from the Swift/C#/KDL approach where the closing delimiter determines indentation to Julia’s approach where the least indented line determines indentation?

EDIT: oh, I see, it’s in order to support this:

s = d"""
  Hello
  World!"""
# equivalent to
s = "Hello\nWorld!"

Maybe the old approach (closing delimiters determine indentation) could be added to rejected ideas at least?

2 Likes

A very incomplete survey of what existing tools do.

inspect.cleandoc

cleandoc’s behavior is kind of interesting:

>>> from inspect import cleandoc
>>> cleandoc("""
...    a
...    b
...  """)
'a\nb'
>>> cleandoc("""
...    a
...    b
...    """)
'a\nb'
>>> cleandoc("""
...    a
...    b
...     """)
'a\nb\n '

It ignores the last line for indentation computation and strips the last newline but only if the closing delimiter has less or equal indentation than the rest.

sphinx

See the implementation here. It seems to take into account the last line when determining common indentation (in contrast to cleandoc) and it always adds an empty line at the end (even if there was none in the string), for reasons to do with reST.

pyright

First removes common indentation and then removes all empty lines and strips the last newline.


So, the situation looks to be inconsistent, unfortunately.

How do I create a d’string’ where each lines has some indent now?
I have often created fragments that drop into, say a makefile like file, that needs the indent.

1 Like

Unlike textwrap.dedent(), the line containing closing quote is always considered when calculating common indent, even if only whitespaces before the closing quote.
So you can write like this:

# having trailing newline.
s = d"""
    foo
    bar
  """ 
assert s == "  foo\n  bar\n"

# if no trailing newline
s = d"""
    foo
    bar\
  """ 
assert s == "  foo\n  bar"

# if you don't want to use line continuation
s = dr"""
    foo
    bar
  """.rstrip()  # or [:-1]
assert s == "  foo\n  bar"
1 Like

Since it’s not part of the PR, I just want to reiterate that I would appreciate it if the PEP would discuss the benefits of from __future__, in particular:

Full comment:

I would be happy to write some text, if you like.