Repr output of t-strings

Core Development
1

I really like the new t-strings in Python 3.14 and am experimenting with them.

However what seems problematic to me is the repr output of t-strings for two reasons.

For example:

x = 1
y = 2
t = t"{x} + {y}"
print(repr(t))

prints

Template(strings=('', ' + ', ''), interpolations=(Interpolation(1, 'x', None, ''), Interpolation(2, 'y', None, '')))

repr output is supposed to show the exact Python source that recreates the object. If this is not possible the <...> for is used. The actual output seems to show a constructor call, but templatelib.Template has no keyword arguments strings and interpolations. I actually tried recreating the t-string with:

from string import templatelib
t = templatelib.Template(
	strings=('', ' + ', ''),
	interpolations=(
		templatelib.Interpolation(1, 'x', None, ''),
		templatelib.Interpolation(2, 'y', None, '')
	)
)

but got

TypeError: Template.__new__ only accepts *args arguments

So the repr output is misleading.

The second problem is that the output makes it really hard to reconstruct the structure of the t-string in my head, since the parts of the template are output in “scrambled” order.

An output that shows the parts in the order that they had in the template would be much more useful, i.e.

Template(Interpolation(1, 'x', None, ''), ' + ', Interpolation(2, 'y', None, ''))

This has three advantages:

  1. This is very close to the expression that created the t-string (in this case it really is the correct expression).
  2. It show the parts in the order that they are in the t-string.
  3. It is shorter, since the keyword names are gone, and the empty strings between the interpolations are omitted.

I realize that what the repr currently displays it how the parts of the template are stored internally, which makes the implementation really short:

static PyObject *
template_repr(PyObject *op)
{
    templateobject *self = templateobject_CAST(op);
    return PyUnicode_FromFormat("%s(strings=%R, interpolations=%R)",
                                _PyType_Name(Py_TYPE(self)),
                                self->strings,
                                self->interpolations);
}

But IMHO the repr output for Template objects should be changed to show the real order of the parts.

Servus,
Walter

15 Likes
2

I don’t think this is a rule, and I don’t think it’s useful. In my mind, the important thing about repr output is that it be useful to developers, and part of that is being unambiguous. Python source that can recreate the object meets those criteria, but it’s not the only way to meet them.

I agree: this looks like a useful output that meets the repr criteria.

4 Likes
3

It’s even worse for nested t-strings:

table = "tablename"
fields = {
	"field1": "foo",
	"field2": "bar",
}

condition = t"field1={fields['field1']} and field2={fields['field2']}"
query = t"select * from {table:l} where {condition:l}"

The default repr gives:

Template(
	strings=('select * from ', ' where ', ''),
	interpolations=(
		Interpolation('tablename', 'table', None, 'l'),
		Interpolation(
			Template(
				strings=('field1=', ' and field2=', ''),
				interpolations=(
					Interpolation('foo', "fields['field1']", None, ''),
					Interpolation('bar', "fields['field2']", None, '')
				)
			),
			'condition',
			None,
			'l'
		)
	)
)

the new repr would give:

Template(
	'select * from ',
	Interpolation('tablename', 'table', None, 'l'),
	' where ',
	Interpolation(
		Template(
			'field1=',
			Interpolation('foo', "fields['field1']", None, ''),
			' and field2=',
			Interpolation('bar', "fields['field2']", None, '')
		),
		'condition',
		None,
		'l'
	)
)
7 Likes
4

Since templates are immutable they can never reference themselves directly, but if a template is part of an interpolated mutable value it can. For this the current repr gives:

x = []
t = t"{x}"
x.append(t)
print(repr)

Template(
	strings=('', ''),
	interpolations=(
		Interpolation(
			[
				Template(
					strings=('', ''),
					interpolations=(...)
				)
			],
			'x',
			None,
			''
		),
	)
)

With the new repr and proper recursion detection we get:

...
print(repr)

Template(
	Interpolation(
		[
			Template(...)
		],
		'x',
		None,
		''
	)
)
5

It’s never been a universal rule, though some objects are specified to round-trip via repl/eval. str being one of them.

In this case, I do think it’s useful. Not necessarily that repr(t) is identical to the original string, but certainly that eval(repr(t)) is identical to the original Template object (which allows for differences in whitespace, escaping, etc.).

Probably just requires someone to implement it, though. I can’t imagine we’d reject such an addition unless it’s merely wasting memory by keeping the unparsed string around. Regenerating the string on request (i.e. someone has invoked __repr__) is the way.

6

I do have a patch that implements that (that’s how I generated the outputs for the new repr ;))

If there are no objections I can open an issue and prepare a pull request for that.

My last patch was ages ago, so I’m no longer familiar with the current workflow.

7

It’s certainly easier to discuss something like this over an actual pull request, rather than in generalities.

The process is basically a standard GitHub process - create an issue (you can link here, but put context into the issue), then create the PR against main, link to it from here and give it a couple days for the relevant/interested committers to take a look.

8

OK, I’ve created the issue Better `repr` for `string.templatelib.Template`. · Issue #151099 · python/cpython · GitHub

9

And here’s the pull request: gh-151099: Better repr output for template strings. by doerwalter · Pull Request #151104 · python/cpython · GitHub

10

After some discussion on the issue Better `repr` for `string.templatelib.Template`. · Issue #151099 · python/cpython · GitHub it is unclear what the best repr output for template strings is.

Currently

>>> x = 1
>>> y = 2
>>> t = t"{x} + {y}"
>>> print(repr(t))

outputs (1):

Template(strings=('', ' + ', ''), interpolations=(Interpolation(1, 'x', None, ''), Interpolation(2, 'y', None, '')))

With the proposed patch it outputs (2):

Template(Interpolation(1, 'x', None, ''), ' + ', Interpolation(2, 'y', None, ''))

(1) looks evaluatable, but isn’t, (2) is evaluatable, but that only holds as long as the interpolated values are themselves output in evaluatable form (and only after from string.templatelib import Template, Interpolation). So the first question is whether the repr output should include the fully qualified class name (like datetime.datetime does) nor not. This would look like this (3):

string.templatelib.Template(string.templatelib.Interpolation(1, 'x', None, ''), ' + ', string.templatelib.Interpolation(2, 'y', None, ''))

Another option is to forgo evaluatable output in constructor form and output the t-string in its literal syntax (4):

t'{x} + {y}'

but that hides the values of the interpolated expressions.

Or we could drop evaluatable output altogether and use the <...> form like this (5):

<Template t'{x} + {y}', x=1, y=2>

which looks good when the interpolated values are simple variable references, but not so good, if they are literals, e.g. t"{[1, 2, 3]} + {'foo'}" would look like this:

<Template t'{[1, 2, 3]} + {"foo"}', [1, 2, 3]=[1, 2, 3], 'foo'='foo'>

What do people think?

1 Like
11

This is the default restriction when outputting evalutable reprs, so these should not be considered drawbacks. Option (2) is always better than Option (1) and (3).

Option (4) looks the most useful for debugging, although I think inlineing the value could also be considered:

<Template t'{x=1} + {y=2}'>

That would prevent confusion of the exact same expression is used multipe times (although this is hopefully very rare):slight_smile:

<Template t'(rand()=4} + {rand()=5}'> compared to `<Template t’{rand()} + {rand()}‘, rand()=4, rand()=5>’

OTOH, Option (4) has the benefit that the template string can be copied out of the terminal and is then valid code, assuming you define the variables. So I think I prefer (4).

12

I don’t have a strong opinion among the choices, but I’ll re-iterate: I don’t believe it’s important for the output of repr() to be evaluatable. How often does anyone actually evaluate the output of repr? (rarely). And if they do, how often does it fail because some class somewhere isn’t evaluatable? (often).

The important thing about repr() output is that it be unambiguous and useful to a developer to understand the data. Evaluatability is one way to achieve that, but not the only way.

5 Likes
13

From the __repr__ docs:

If at all possible, this should look like a valid Python expression that could be used to recreate an object with the same value (given an appropriate environment)

I don’t see much point in relitigating that. Even if we did, I’d imagine we’d come to the conclusion that in this case, an evaluable repr is more information-dense that the alternatives.

1 Like
14

The literal thing is kind of a game-breaker for me. It’s very confusing to see an invalid identifier on the LHS of an equal sign.

2 and 4 are both much better than 3 IMO; the fully qualified imports seem verbose, and t-strings can have a lot of parts. 2 is somewhat better than 4 since it preserves context; I don’t think we should use a repr that privileges names over values since some applications will care more about the latter than the former.

15

TBH, I’d be happy to update that section of the docs to clarify the goals of repr(). I don’t think it’s ever been common to literally re-evaluate the output of repr(). Evaluatability is a means to an end: unambiguous information-rich strings useful to developers.

1 Like
16

Do you have any actual evidence for this or is this just your opinion? I regularly make use of this, especially for dataclasses. Primarily when trying to check edge cases for short scripts.

I don’t think a long established policy should be abolished because you feel like it.

5 Likes
17

It’s also already the case for all other string types, but I think the implication of the text that’s currently there is that it should be very familiar to a Python developer, to disallow “if we base 64 encode the original bits then it can be perfectly recreated” which is obviously silly, but sometimes we need to write down guidelines to save ourselves from politely explaining how silly some ideas are.

I kind of like this one, though I’m also happy without the inlined values. The “given an appropriate environment” note from the repr docs I think gives us justification for omitting them, but at least for reasonable cases it’s definitely more useful to have them.

18

I’m not talking about forbidding making it evaluatable. I’m talking about clarifying the goal. It wouldn’t change how dataclasses are repr’d. There are many classes now that are not round-trippable through repr/eval, so clearly it’s not a policy. It’s a common way to achieve a goal. If it works to achieve the goal, then make your class evaluatable.

Understanding the goal will help people write better reprs.

1 Like
19

I think the output of repr() should be unambigous and should show all relevant parts of the object, so the users know what they’re dealing with. An object with a different value should have a different repr. This is a problem for option (4). If a logfile contained t’{x} + {y}’ relevant information (the values of x and y) would be lost. This is probable the most valuable part of the t-string, since it’s the only variable part, all other parts are found literally in the source code. Of course this is limited: I wouldn’t expect the repr output of an XML element node to dump the whole XML tree. (But then again a large JSON compatible data structure consisting of nested dicst an list would dump the complete tree).

So IMHO for t"{x} + y{}" it’s either:

Template(Interpolation(1, 'x', None, ''), ' + ', Interpolation(2, 'y', None, ''))

or

<Template t'{x=1} + {y=2}'>

And of course we would need to output the conversion and format spec, so it would probably be:

<Template t'{x=1!r:02} + {y=2:x}'>

for

t'{x!r:02} + {y:x}'

For expressions that are not variables or literals like

t'{x+y} {'foo'} {[1,2,3]}'

we would get:

<Template t'{x+y=3} {'foo'='foo'} {[1,2,3]=[1, 2, 3]}'>

This looks a bit strange. Option (2) would give

Template(Interpolation(3, 'x+y', None, ''), ' ', Interpolation('foo', "'foo'", None, ''), ' ', Interpolation([1, 2, 3], '[1,2,3]', None, ''))

which is much longer but makes it clear, that the interpolated value is whatever type it is, and the expression is a string.

3 Likes
20

I like

Template(Interpolation(1, 'x', None, ''), ' + ', Interpolation(2, 'y', None, ''))

most, because we see at a glance both

  • the internal reprensentation of the t-string (notably the fact that it’s NOT a string, which can be not obvious to newcommers; I dislike option 4 t'{x} + {y}' for this reason);
  • the global shape of the string (unlike the current template= and interpolations=, where we need to mentally intertwine them).
3 Likes