PEP 813 - The Pretty Print Protocol

PEPs
61

Thanks for the detailed thoughts RDM[1]. We’ll bikeshed on the methodname, and I think the salient point for where I envision this PEP is this last paragraph. All the suggestions you make would be quite useful, but is a more more elaborate proposal than what’s going on here. Which of course doesn’t mean it’s bad, just that it’s unlikely something I’d want to champion[2].

So let’s focus in on the corner-painting and make sure that nothing we’re doing here can prevent such future directions, should someone want to pursue them.

  1. and for the subtle callback to the email module for motivation! :smiley: ↩︎

  2. though I won’t speak for my co-conspirator ↩︎

1 Like
62

Except that formatting isn’t what the magic method is doing either. Rather, it’s returning the components that the “framework” (as embodied by the pprint module) uses to build a prettier representation of the object.

2 Likes
63

This argument doesn’t work for me. This proposal does more than just tie some things together. The existing pprint is a module whose behavior is encapsulated and local to that module. It’s a builtin module, yes, but it’s still a module. This PEP elevates that to the level of fully built-in behavior in the form of a dunder and an interaction with the builtin print function. In my view that requires substantially more justification than just “well it’s what the pprint module already does”.

I understand the urge to limit the scope of the PEP, but I think in the end it won’t be the best choice if we wind up with this behavior ensconced in the Python language just because it was already there in a stdlib module.

Instead I think we need to ask ourselves “if we are going to have pretty-printing behavior built in to Python, how should it work?” The answer to that may well be different from what pprint does, and I think a lot of the replies here already raise important questions about that[1]. In fact I think it would make more sense to first think about how the pprint module could be improved, and then think about how or whether to add its behavior to the core language. Otherwise we are potentially locking in a suboptimal design.

  1. customising the behavior for builtins, wrapping in a class-name “template”, color, etc. ↩︎

12 Likes
64

GUI debuggers (or GUIs in general) would probably be better off with something like the _repr_mimebundle_ protocol so that they can provide non-plain-text display.

For example here is how a Polars dataframe displays under Jupyter, vs. how it may display if using only a plain text pretty-printing (emulated here via converting it do dict):

jupyrich
jupyrich691×775 41.2 KB

5 Likes
65

Honestly, this seems like an unnecessary footgun. I think it would better to prohibit not using tuples and instead require having None as the name when providing a positional argument - I think this falls under “explicit is better than implicit” and “refuse the temptation to guess”. It’s a slight downgrade in the egernoumics with the benefit of far more reliable semantics.

I’ve been using it for a few years, and very occasionally do I make that mistake. When it happens, the output is obviously wrong, but doesn’t throw any exceptions.

I don’t recall any feedback about that particular footgun. I guess it doesn’t blow your entire foot off.

The ratio of __rich_repr__ methods I write, versus the fraction of times I do that, makes me prefer the improved ergonomics.

66

And this is precisely what I was pointing out - the pretty argument is at the wrong granularity, you don’t want every argument to be pretty printed in this case (and I’d argue, in every case where a literal string is one of the print arguments).

Also, if some of the arguments are large enough that pretty printing them takes multiple lines, concatenating the pretty-printed representation of the individual arguments is likely to result in bad line breaks:

>>> long = [12345] * 20
>>> short = [1] * 8
>>> print("Long =", pformat(long), "and short =", pformat(short))
Long = [12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345,
 12345] and short = [1, 1, 1, 1, 1, 1, 1, 1]

It’s not terrible, but it’s not ideal, and it’s very data dependent how good it is.

What I’m trying to say is that I’m not sure that an option which works well for a single positional argument, and sort of works in some cases for multiple arguments, but in many multiple-argument cases gives suboptimal results, is good enough to be made part of the language spec.

Conversely, the !p conversion specifier is at the right granularity - it modifies how precisely one value is formatted.

I disagree. Making the functionality of the pprint module more accessible doesn’t in itself make it “fully built in behaviour”. It’s still just as possible to change and improve the behaviour of the pprint module, and that will be reflected in the built in behaviour. (As a side note, the PEP should be clear on how the !p conversion spec will be affected if someone were to monkey-patch or otherwise replace the stdlib pprint module - it would be a really odd special case if doing so didn’t change the behaviour of !p).

What’s more questionable is the __pprint__ protocol. And I think that’s where we need to be careful. If we treat __pprint__ as nothing more than an enhancement to the pprint module, using a dunder not as a language-level protocol but as an implementation mechanism for the stdlib module (much like object.__copy__ is an implementation detail of the copy module), then it remains “behavior [which] is encapsulated and local to that module”. That’s fine, but we’ve now started to look at enhancing the pprint module - and if we’re doing that, we can’t really argue that including features like customising the display of builtin types, or supporting custom container formatting, is “out of scope”. And at the very least, a __pprint__ protocol that isn’t designed in a way that can be extended in the future to such use cases, is a potential backward compatibility issue when we do want to look at them. While that doesn’t make the protocol a language-level behaviour, it does mean that it introduces many of the same concerns that language-level changes have.

So while I disagree with your claim that the PEP makes pprint a “built-in behaviour”, I do think that it makes pprint a much more important part of the stdlib, and as a consequence, we need to be more careful about the implications - right now, changing pprint is a fairly low-impact matter, but if we make the behaviour of print and the !p conversion specifier depend on it, such changes become a lot more significant[1].

  1. Not only in terms of consequences, but also of demand - we’ll see more requests for better pprint behaviour once people start using !p and/or pretty=True… ↩︎

6 Likes
68

Indeed.

But if there are alternatives that are objectively (by consensus and latest developments in PyPI) prettier, then such tie might not be the best path forward.

If pprint is lacking, maybe the first step would be to improve and polish it via some parameterisation to the degree so that it would be the best candidate to be the only default for endeavour at hand?

1 Like
69

If confusability is the only concern with a new built-in, pretty_print is less confusable and still shorter than print(..., pretty=True).

A new built-in can also be restricted to printing a single value the same way pprint.pp and pprint.pprint are, so the problems @pf_moore pointed out regarding rendering of multiple values (including string literals) can be avoided.

3 Likes
70

And this is precisely what I was pointing out - the pretty argument is at the wrong granularity, you don’t want every argument to be pretty printed in this case (and I’d argue, in every case where a literal string is one of the print arguments).
[…]
Conversely, the !p conversion specifier is at the right granularity - it modifies how precisely one value is formatted.

What is the actual motivation for the pretty=True argument? What does
it buy us that “!p” doesn’t? print(myvar, pprint=True) is longer
than print('{myvar!p}'), so I for one will reach for the latter before
the former. I suppose that adding pretty=True to an existing print
statement is easier, but how often is that going to come up in practice?
I’m almost always using {myvar=} in my debug prints these days, so I
don’t see “adding pretty=True to an existing print” a compelling use
case.

I’d say the only thing the pretty argument buys is the ability to pass
an alternate pretty printer. If that is what gets applied to “!p”
formatting (I haven’t checked the code) then it is worth while. But the
behavior of applying pretty printing to each individual argument does
not seem all that useful, given that “!p” exists.

The current PEP implies that a passed in pretty printer is not used by
“!p”, that that seems really surprising. This should be clarified
regardless.

2 Likes
71

!p would have no access to the value of print’s pretty parameter, since the f-string’s work is done before print is called. There would be the possibility of the format spec that is used with !p could contain some additional information, like what pretty printer to use, but I don’t have a proposal for that (yet).

72

Heh. You just do {mypformat(expr)} in that case, no other magic
needed, I think.

So, print(args, pretty=callable) is a replacement for
callable(args), and its benefit is that you can also specify file=
in the print call even if callable doesn’t support that. And I guess
callable only ever needs to worry about a single argument. This boils
down to a reduction of the API a pretty printer needs to support in
order to be usable to print multiple arguments in one print
call…which could also be accomplished by calling it on each argument
you actually wanted pretty printed.

Is this ‘pretty’ argument worth it? My gut is saying no.

It feels like this parameter to print will raise more questions than it
will answer. Like mine about it applying to !p…the answer is
obvious in retrospect, but not intuitive…and Barry’s example where
strings get surrounded by quotes even if that wasn’t what you meant.
It does feel like pretty= is operating at the wrong level of
granularity.

4 Likes
73

Looking at this as data destructuring in support of pretty printing, it feels very similar to match_args.

Do the two really need to be different, or does __match_args__ give us everything we need? (once you stringify the values) Maybe there’s a good default for objects which define __match_args__?

74

It does not; __match_args__ approximately gives us all positional arguments. It doesn’t include kwargs, and it can include extra stuff (e.g. a special $self fake attribute for true ValueTypes). It also doesn’t include default values.

75

Would it be possible to have a “compact” option? I use something l like this frequently.

The “:” all line up, the key and values texts are truncated if they get to long, and the value type is shown.

It is prettier? and and the type keeps me out of trouble.

edit. sorry, Had to cut and paste a picture to show the intended formatting…

image
image713×317 10.8 KB

76

Could this be used to print out the contents of a namespace?

Like those from: from types import SimpleNamespace

or a defined tuple

It is nice to have a listing of all the stuff that is available after the “.” in a pretty format

77

I find that pretty[1] compelling actually[2]. I’ll chat with Eric and we’ll see where that goes.

  1. pun intended! ↩︎

  2. In the short history of this idea, print(..., pretty=True) predates !p ↩︎

1 Like
78

I would welcome the addition of a protocol that produces a structure that would allow, for example, the pprint module or rich or a custom callable to display an object nicely!

But I, like many commenters before me, am not totally convinced that the protocol proposed in the PEP is future-proof.

To me, this looks like an AST. (One could even use a subset of the ast expressions, like Dict, Tuple, Call, … but that might be overkill.).

from dataclasses import dataclass
import typing

### To be defined in the standard library
@dataclass
class Call:
  """
  A structured representation of a call to a constructor or a function.
  """
  name: str
  args: tuple
  kwargs: dict

StructuredRepresentation = list | tuple | dict | Call

def render_simple(obj: object) -> str:
  """
  A simple renderer for structured representations.
  
  This is just an example and can be extended to handle more complex cases, for example multi-line representations, indentation, etc.
  """
  if isinstance(obj, Call):
    args_str = ", ".join(render_simple(arg) for arg in obj.args)
    kwargs_str = ", ".join(f"{k}={render_simple(v)}" for k, v in obj.kwargs.items())
    all_args = ", ".join(filter(None, [args_str, kwargs_str]))
    return f"{obj.name}({all_args})"
  
  if isinstance(obj, dict):
    items_str = ", ".join(f"{render_simple(k)}: {render_simple(v)}" for k, v in obj.items())
    return f"{{{items_str}}}"
  
  if isinstance(obj, (list, tuple)):
    items_str = ", ".join(render_simple(item) for item in obj)
    return f"[{items_str}]" if isinstance(obj, list) else f"({items_str})"
  
  if hasattr(obj, "__structured_repr__"):
    return render_simple(obj.__structured_repr__())
  
  return repr(obj)

class StructuredRepresentable(typing.Protocol):
  """
  A protocol for classes that can provide a structured representation of themselves.
  """
  def __structured_repr__(self) -> StructuredRepresentation: ...

### Examples of user-defined classes that implement __structured_repr__
class SomeMapping(dict):
  # This could even be in collections.abc.Mapping!
  def __structured_repr__(self):
    return Call(type(self).__name__, (), {k: v for k,v in self.items()})

class SomeTuple(tuple):
  def __structured_repr__(self):
    return Call(type(self).__name__, self, {})
  
class SomeOtherClass:
  def __structured_repr__(self):
    return Call(type(self).__name__, (), {'foo': 'bar'})
  
### Example usage
print(render_simple(SomeMapping(a=1, b=2)))  # Output: SomeMapping({'a': 1, 'b': 2})
print(render_simple(SomeTuple((1, 2, 3))))      # Output: SomeTuple(1, 2, 3)
print(render_simple(SomeOtherClass()))  # Output: SomeOtherClass(foo='bar')

If one wanted to omit default arguments, like rich does, one could extend the Call class (e.g. with a default_kwargs field).

Concerning some other points:

  • I like f”{obj!p}”. It should just use the default renderer.
  • I don’t like print(…, pretty=True). The level of granularity feels wrong to me. You can just use print(f"{obj!p}") and achieve the same benefits (e.g. the file parameter). Others have said the same.
  • If I want to use my own special renderer, I can do f”{my_special_renderer(obj)}”.
  • If I want all print statements to do some special rendering of their arguments application-wide, I could, for example, builtins.print = rich.print.

So, in conclusion: I’m looking forward to a well-designed protocol (and minimal implementation) in the standard library that can be adopted as one standard by the multiple solutions that already exist for “pretty printing”.

1 Like
79

@ericvsmith and I thank you all for your insightful feedback. We’ve now published an update, which adopt many of the suggestions, and defers or explicitly rejects others.

The big TL;DR is that we’re dropping the changes to built-in print(). @bitdancer and others were pretty convincing that the granularity of the proposed pretty argument was wrong[1] and that !p for f-strings will do what we need. After working with @pablogsal to flesh out the implementation and convince ourselves that adding !p:expression format specs (only for f-strings!) would work, we felt happy with this change.

We’ve updated the examples[2], tightened up the __pprint__() protocol description, and explicitly proposed adding PyObject_Pretty() to the Limited C API.

Enjoy!

  1. something that we knew in our gut ↩︎

  2. with hilarity, since Pygments didn’t like us using !p in the f-string examples! ↩︎

3 Likes
80

How is this expression evaluated? Is this special syntax that is connected to !p? Or does expression get stored as a format specifier string and then get evaluated to an object later on?

81

The idea is that it is special logic attached to !p, which would store the expression to be evaluated. It would not be stored as a string.