Add `!R` conversion to f-string and `str.format()` to format error messages

Core Development
1

Hi,

I propose adding !R conversion format to f-string to format error messages and quote properly str subclasses. Example:

class StrSubclass(str):
    def __repr__(self):
        return '<override>'

abc = StrSubclass('abc')
print(f'{abc!R}')

The output is 'abc', whereas f'{abc!r}' (current !r conversion) returns <override>.

The use case is to replace f"no attribute '{attr}'" with f"no attribute {attr!r}" to handle correctly attr which contains quote characters (' and ").

Do you think that it would be a good idea? Or do you think that it’s overkill to handle str subclasses differently?

Well, an alternative is to not bother with quotes and leave the current code as it is :slight_smile: So for example, continue using '%s' in error messages to format an attribute name.

Problem

While reviewing a PR changing AttributeError, I noticed that strings including a quote character are not formatted properly. Example:

>>> getattr(object(), 'abc')
AttributeError: 'object' object has no attribute 'abc'

>>> getattr(object(), "a'c")  # the attribute name contains a quote
AttributeError: 'object' object has no attribute 'a'c'

The ouput string 'a'c' didn’t escape the quote character correctly.

The AttributeError error message is created (in C) using "'%s' object has no attribute '%s'" % (type_name, attr_name). I would prefer to format the attribute name using repr(). Something like:

>>> type_name = '<object>'; attr_name = "a'b"
>>> print("'%s' object has no attribute %r" % (type_name, attr_name))
'<object>' object has no attribute "a'b"

The attribute name is now formatted using double quote ("): "a'b". The new problem is that str subclasses can override __repr__() which isn’t formatted as expected:

class MyStr(str):
    def __repr__(self):
        return '<custom repr>'

print('object has no attribute %r' % MyStr('abc'))

Output:

AttributeError: 'object' object has no attribute <custom repr>

The error now contains <custom repr> instead of 'abc'. The problem is that getattr() tries to get the object.abc attribute, so the error message is not correct.

Proposed solution

A solution for this problem would be to add !R conversion to f-string and str.format(): similar to !r which calls repr(), but don’t call __repr__() on str subclasses.

In the C API, add also %#R format to PyUnicode_FromFormat(): similar to %R (call repr()), but don’t call __repr__() on str subclasses.

See also

2

The !R conversion would be added to t-string as well. Hum, it would be nice to add alt=True optional keyword-only parameter to repr(). Otherwise, non-trivial code should be added in each project handling t-strings, like the repr_alt() function in this example:

class StrSubclass(str):
    def __repr__(self):
        return '<custom repr>'

def repr_alt(obj):
    if isinstance(obj, str):
        # Don't call obj.__repr__() on str subclasses
        return repr(str.__str__(obj))
    else:
        return repr(obj)

print(repr_alt(StrSubclass('abc')))
3

What would it do if the argument is not a string? If raise TypeError, then you can simply use unbound method str.__repr__().

My proposition is C API only (for now), but more general, it affects all character and string formatting codes: %c, %X, %s, %ls, %S, %U, %V, etc. Quotes are not included, so you can for example compose the full qualified name from the module name and the qualname as '%#U.%#s'.

Open question is what to do with quoting quotes? Should we always quote both quotes or add modifier specifying what quote to quote, e.g. %#'s? In the latter case we can drop the # modifier which conflicts with %#N and %#T, but some formats will look quite cumbersome ('%'s' or \"%\"s\").

4

By the way, the PyUnicodeWriter_WriteStr() function added to Python 3.14 has a similar issue with str subclasses which overrides __str__() (not __repr__()): see the issue.

5

I checked the stdlib and I found some AttributeError created with str() and some with repr() (it’s not consistent). Examples from two different Python modules:

# Module 1 using str() + quotes
def __getattr__(name):
    ...
    raise AttributeError(f"module '{__name__}' has no attribute '{name}'")

# Module 2 using repr()
def __getattr__(name):
    ...
    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

My proposition is to move from str()+quotes towards repr() (without adding quotes, str._repr__() adds quotes). If the argument is not a string, repr(obj) just calls obj.__repr__() is called.

I don’t understand your remark about TypeError. Can you give an example? What would raise TypeError?

6

The old version 1 is already wrong in the presences of str subclasses that override __str__.

I don’t think this is an issue worth worrying about. There is a contract for subclasses, especially subclasses of builtins, and IMO such dunders violate this contract and it producing garbage output is not unexpected nor avoidable.

1 It actually turns out that the current[1] version of AttributeError doesn’t directly use __str__, which means your python code equivalent doesn’t reflect the current behavior.

  1. 3.14.4 ↩︎

7

In C code, it’s possible to format an error message without calling overridden __str__() or __repr__(). The problem is that in Python, it’s not possible. For example, in Python, "%s" % obj, f"{obj}", f"{obj!s}" and f"{obj!r}" call overridden __str__() or __repr__() method.

In Python, str.__str__(obj) can be called to format a str subclass as C code does, but I would prefer to not have to write such code.

There are legit use cases to inherit from str and overrides __str__() and/or __repr__(). Examples with enum:

from enum import StrEnum, auto, Enum
import types

class Color(StrEnum):
    RED = 'r'

class StringEnum(str, Enum):
    COLOR = "color"

print("StrEnum:")
print("- str:", str(Color.RED))
print("- repr:", repr(Color.RED))
print()
print("StringEnum(str, Enum):")
print("- str:", str(StringEnum.COLOR))
print("- repr:", repr(StringEnum.COLOR))
print()

# Use str subclasses with getattr/setattr
mod = types.ModuleType("mymodule")
try:
    getattr(mod, Color.RED)
except AttributeError as exc:
    print(f"getattr(mod, Color.RED) fails with: {exc}")
try:
    getattr(mod, StringEnum.COLOR)
except AttributeError as exc:
    print(f"getattr(mod, StringEnum.COLOR) fails with: {exc}")
setattr(mod, Color.RED, 1)
setattr(mod, StringEnum.COLOR, 2)
print("- mod.__dict__:",
      {key: value for key, value in mod.__dict__.items()
       if not key.startswith('_')})
print(f"- {mod.r=}")
print(f"- {mod.color=}")

Output:

StrEnum:
- str: r
- repr: <Color.RED: 'r'>

StringEnum(str, Enum):
- str: StringEnum.COLOR
- repr: <StringEnum.COLOR: 'color'>

getattr(mod, Color.RED) fails with: module 'mymodule' has no attribute 'r'
getattr(mod, StringEnum.COLOR) fails with: module 'mymodule' has no attribute 'color'
- mod.__dict__: {<Color.RED: 'r'>: 1, <StringEnum.COLOR: 'color'>: 2}
- mod.r=1
- mod.color=2

StrEnum overrides __repr__(). StringEnum overrides __str__() and __repr__().

To format AttributeError error message, types.ModuleType doesn’t call __str__() or __repr__(). C code: PyErr_Format(PyExc_AttributeError, "module '%U' has no attribute '%U'", mod_name, name).

8

Yes, but I don’t think that means we need to work around this. Unless the repr/str is deliberately confusing, i.e. it looks like a different string it will almost always be clear what the string being looked up is. This is the case for both Enums.

Sure, so keep this behavior/call str.__repr__. I don’t see the motivation for a new formatting code. It’s not currently directly possible to emulate the C code behavior without explicitly calling str.__str__ and I don’t see many complains. Maybe it’s worth adding a note in the docs somewhere that it’s suggested to format strings used as attributes like that, but I don’t see the need to add a new python level formatting option that would require handling by all consumers. (which especially with template strings will have gone up)

9

Then we return to the same issue

    def __repr__(self):
        return '<override>'

I supposed that you will make !R to reject non-strings. This is the way to avoid executing overrided __repr__.

10

I think this is a wrong premise.
getattr tries to get the MyStr object as an attribute, not the string 'abc'. The issue is that the MyStr class is misleading – its __repr__ doesn’t match the string contents that getattr uses. Either the author wanted to make it misleading, or it’s a bug in MyStr.

Adding !R won’t solve other cases of misleading subclasses. For example, the following will AFAIK still fail with 'int' object has no attribute 'real', while it’s “really” looking up otherattr:

class MyStr(str):
    def __eq__(self, other):
        return other == 'otherattr'

    def __hash__(self):
        return hash('otherattr')

real = getattr(3, 'real')
print(f'{real=}')
real = getattr(3, MyStr('real'))  # 'int' object has no attribute 'real'
4 Likes