Note that t-string too. Since t-string is not actual string literal, you cannot call str.dedent() nor textwrap.dedent() for it.
>>> world = 42
>>> s = t"Hello, {world}"
>>> s.trim() # I use str.trim() but this demonstrates we cannot use str.dedent() if we have it.
Traceback (most recent call last):
File "<python-input-6>", line 1, in <module>
s.trim()
^^^^^^
AttributeError: 'string.templatelib.Template' object has no attribute 'trim'
>>> from textwrap import dedent
>>> dedent(s)
Traceback (most recent call last):
File "<python-input-9>", line 1, in <module>
dedent(s)
~~~~~~^^^
File "/Users/inada-n/work/python/cpython/Lib/textwrap.py", line 433, in dedent
raise TypeError(msg) from None
TypeError: expected str object, not 'Template'
I’d like to share a thing that I’m doing because I think it helps explain why I’m against adding the d prefix – and favor either doing nothing or str.dedent().
I have a code fixer. If it sees adjacent strings on a single line with implicit concat, it joins them.
# before
"/ a nice small poem "
"/ so nicely aligned by me "
"/ it shows what happens"
# after black/ruff
"/ a nice small poem " "/ so nicely aligned by me " "/ it shows what happens"
# after my fixer
"/ a nice small poem / so nicely aligned by me / it shows what happens"
Skipping some of the special cases around f-strings and mismatched quote chars, here are the rules for which strings are safe for my fixer to slam together:
no prefixes, matching quote chars
prefixes are a mix of {"fr", "rf"}
one has no prefix and the other has "f"
one has a prefix of "fr" or "rf", the other has a prefix of "r"
prefixes are a mix of {"rb", "br"}
I don’t even try to handle "u", which would make this list of rules even more fun.
Adding "d" as a prefix char adds not just one new case for me to handle here, but actually many!
“Is it safe to concatenate these two d-strings and this f-string into a single literal, so long as I mix the prefixes together?”
Now, as a thought experiment replace “my program” with “the developer”. Will the developer know what combinations are valid, invalid, and which ones are syntactically valid but semantically different?
The string prefix chars are totally fine if you don’t look at them too hard. But as soon as you examine them closely as data on the screen for the developer try to process and work with – and importantly not in isolation from one another – I think the DX problem they create becomes clear. I’m happy with where the language is and is going (especially assuming we get the t-strings fixed to disallow concat), but I’m not a fan of new string prefixes.
(Also, unlike t-strings, which are not strings and can therefore disallow concat, the proposed d-string would not have that option.)
Does your fixer add extra braces to force the non-f-string to be interpreted as a literal?
ISTM your fixer is already taking a lot of liberties here, which is fine as long as you know that its assumptions are valid, but it’s no different from the assumptions and consequent liberties that are needed for d-strings.
This was one of the things I left out under “Skipping some of the special cases around f-strings”, but that might have been misinterpreted as nerd-sniping, sorry!
I don’t allow any auto-concat if the non-f-string part contains any curly braces at all; and secondarily the same concat case is handled by a lint rule. It’s sufficiently rare in practice that very occasionally getting a lint failure and having to manually handle it is fine.
My point in bringing it up was to write out some of the rules you need for a relatively trivial operation – removing excess quotes in a single string literal formed by implicit concat.
Those rules are already in many of our heads – e.g., that "fr" and "rf" are equivalent – but they got there by us learning them. And I’m not sure I think it’s a great idea for all of us to start learning what a frd"string" means.
I understand that we’re all jealous of the French devs for getting their country code included in the language, but getting de"strings" and dk"strings" and so forth is clearly a non-goal.
Maybe I’m just salty because I got the string prefix question wrong in that knowledge quiz, though! And that’s in spite of having gone pretty deep on these string transforms not that long ago.
Still, I stand by my original point: if it’s okay to special-case parts of the f-string concat (by checking the contents), is it a problem to say “d-strings and non-d-strings don’t get concatenated”? You could keep all your existing rules, and simply require that both sides have the same d-ness. (Is that a thing? If you prefer: Require either that both sides be dedented, or that neither side be dedented.)
An alternative is possible, without defining a new string type, but a new ‘dedenting’ escape sequence. It might be less elegant but might also provide more flexibility.
For example, using\# as the dedenting escape sequence :
def func():
def funcc():
instructions = """
\#for i in range(N):
\# do_some_op(i)
"""
And in the IDE, this escape sequence might be colored as the comments, while the remaining part of the line remains of the string color (so you have a two colors in the string line).
Basically the escape sequence would not remain after creation of the string, that is created with \# acting like “this line begins here”.
Note: Aligning the text this way would therefore be strictly equivalent:
instructions = """
\#for i in range(N):
\# do_some_op(i)
"""
I don’t think it changes my opinion (it might be better or worse for other reasons; I don’t know). It’s still adding the possibility of
df"""this"""
du"""and this"""
dr"""and that"""
Plus, developers will have to learn which of these is valid
f" One"
r" Two"
d" Three"
It still adds to special rules that developers need to memorize.
I’m not worried about my tools. This is not a hard thing to teach tools about. But thinking about tools can help us think about people, especially beginners, because we have to start writing down the rules we know.
So, I’m still not in favor and prefer str.dedent().
Since thinking about it more, I have actually become a very big fan of str.dedent(), I think that’s what we should do.
Only if it becomes the unconditional behavior in the future. I’d be okay with that, but I think it’s a very big change.
Imagine it being planned for 3.18, for example. Users will maybe panic in 3.17 and heavily petition the SC to delay. You can imagine the bad scenarios around this.
So in this case, I also/still prefer str.dedent(). But for very different, unrelated reasons.
Maybe it would be nicer if the language always worked this way. But is it beneficial enough, vs the cost, to be worth doing now? I don’t know.
This is what also the OP proposes and what I would support too.
In Python 3, the "u" has no effect whatsoever since unicode is the default. So, the simplest rule would be to just remove it everywhere. It’s actually that simple.
Agreed, but it again depends on the exact semantics. JEP 378 rejects literals if there is any non-whitespace on the first line after a triple quote. If this would ever become the standard behavior in Python, quite some code would be rejected. If “just” the default dedentation rules would change, there would be just changes in the whitespace characters that are included in the output. If you create HTML pages for example, it doesn’t really matter. So, a __future__ that will become the default eventually, could be ok as well. For __future__, one would have to decide if also raw strings are dedented.
Fair enough. It adds another pass to the fixer, which with libcst is costly, so it’s still useful for my fixer to not care about it – but that is all OT.
The inflation in complexity for u is definitely minimal, but it is another rule that we’re memorizing: the rule that it does nothing and we can or should remove it the moment it’s inconvenient.
I voted for the second option, where the behavior of triple-quoted string literals itself changes to auto-dedent, because it truly is my favorite option. I’m willing to live with a __future__ import for at least another 5 years, as I do think it’s the best outcome in the long term.
However, I’m aware of how significantly harder it will be to get a breaking change like that accepted over one that adds a non-breaking new feature, so my hypothetical second vote would be for a d-string literal instead.
Let’s discuss that __future__ import for a moment. I personally would prefer the prefix, but I would be okayish with the change to triple-quoted semantics, as long as it can be done smoothly.
Future imports affect one module at a time. How does this affect functions that look at strings that come from other modules? For example, help(func) needs to look at the function and display its docstring. How will it handle the possibility that the string might have come from either type of compilation? Will there be an attribute on the string saying whether it’s been dedented or not? Or will everything need to cope with both possibilities?
Doesn’t help dedent docstrings anyway? And wasn’t there a recent change so that docstrings are dedented by the compiler for storage in .pyc files to save space?
Yes, and I’m not certain whether its behaviour would be affected by automatic dedenting. That one might be okay, since dedenting would be idempotent? I think? Others, perhaps not.
Ah, in that case help() is probably already fine, as is anything else that uses docstrings specifically (eg clize which uses them to configure argparse). So I don’t have any good examples. It’ll still potentially be relevant though, if anyone knows of something that would need to change its behaviour based on some other module’s use or non-use of the future directive.
I want to point out an issue with how str.dedent() composes with f-strings.
If you have a program like this:
def generate_instructions():
epilog = """
Output the equation followed by the result.
Example:
2 * 2 * 3 = 12
Example:
2 + 3 + 4 = 9
""".dedent()
if mode == "multiply":
return f"""
You'll be given a list of numbers.
Multiply the input numbers together.
{epilog}
""".dedent()
# elif mode == "add" ....
print(generate_instructions())
Then the output will be:
You'll be given a list of numbers.
Multiply the input numbers together.
Output the equation followed by the result.
Example:
2 * 2 * 3 = 12
Example:
2 + 3 + 4 = 9
The first part of the output is indented because the f-string substitution happened before the dedenting. The second dedenting had no effect, because the substitution introduced lines with no indents at all, so the minimum indent was 0.
The right order of operations is to dedent first, then format.
For that reason, I think a d"" string prefix would be better, that way, when composed with f-strings, Python can ensure the ordering is dedent then substitute. Then the above program looks like
def generate_instructions():
epilog = d"""
Output the equation followed by the result.
Example:
2 * 2 * 3 = 12
Example:
2 + 3 + 4 = 9
"""
if mode == "multiply":
return df"""
You'll be given a list of numbers.
Multiply the input numbers together.
{epilog}
"""
# elif mode == "add" ....
print(generate_instructions())
and would output:
You'll be given a list of numbers.
Multiply the input numbers together.
Output the equation followed by the result.
Example:
2 * 2 * 3 = 12
Example:
2 + 3 + 4 = 9
Which is far more likely to be what the user intended.
edit: i see now that this was noted by @methane and @pf_moore above. I’ll leave the point here as I do feel that it’s important how language features compose with each other, and I come across this pretty often myself, when writing code that generates LLM prompts.
I find the example above an incorrect .dedent() usage. Only the string to be printed should be unindented or the two multiline parts should be dedented separately and then joined. Let me explain below why I came to this conclusion and let me start by saying that this example nicely shows that .dedent and d"string" are not equivalent. (I like it a lot when we get examples to test new ideas like this.)
The syntax some_string.dedent() indicates that the unindentation method is applied on the final form of string, i.e. afterward. A memory/runtime saving is possible only if the parser will handle "<some_literal_text>".dedent()specially.
OTOH d"string" syntax says that this is a special form of a string and the parser must treat it specially before it can be used in the program.
Have you actually tried these? Neither of them work with f-strings (minus one special case where the two literals are written at the same indentation and you .strip() the inner string). The best we have at the moment is dedent("""string 1""") + dedent("""string 2""") + ... which is a real fight to keep legible in real world cases, despite being nothing more than mashing a few strings into a template.
Out of curiosity, how does complexity of having new string prefixes scale? People talk about there being so many combinations you can make out of r", f", rf", rfd", t",… but are there actually places (in CPython or linters/formatters/…) where each permutation is its own code branch (O(2n) complexity) rather than just per modifier (O(n) complexity)?
would need 2 arguments for this to be functional, such as:
string = """
Yes?
Maybe.
""".dedent(1, -1)
Then can do as following:
def generate_instructions():
epilog = """
Output the equation followed by the result.
Example:
2 * 2 * 3 = 12
Example:
2 + 3 + 4 = 9
""".dedent(1, -1)
if mode == "multiply":
return f"""
You'll be given a list of numbers.
Multiply the input numbers together.
{epilog}
""".dedent(1, 5)
# elif mode == "add" ....
print(generate_instructions())
Although, I think this would allow the above to work, I wouldn’t structure the code this way, but rather:
def generate_instructions():
epilog = ["""
Output the equation followed by the result.
Example:
2 * 2 * 3 = 12
Example:
2 + 3 + 4 = 9"""]
if mode == "multiply":
epilog.append("""
You'll be given a list of numbers.
Multiply the input numbers together.""")
# elif mode == "add" ....
return ''.join([part.dedent(1) for part in epilog])
print(generate_instructions())
Also, I would like .indent() as well, but in the form of prefix_lines…
