Presenting progress for pyproject.toml-based builds

This is mostly intended as an open ended discussion, that might lead to a meaningful user experience improvement.

Currently, if you try to do something like pip install amazing-package and the package requires a pyproject.toml-based build, there’s no mechanism for the build-backend to indicate to the frontend that it needs to present some output. This means that a user gets effectively no output when they have a long running build within pip, unless they pass -v.

Is this something that we’d want to improve, adding some way to show progress while a build is happening?

x-ref: Better reporting on output from PEP 517 build backends · Issue #6634 · pypa/pip · GitHub

1 Like

In principle, this would be good, and having some sort of callback that backends can use to say “I’m X% done” might be nice.

In practice:

  1. Do backends know how far they are through the process? If a backend calls a C compiler, it would rely on the compiler returning progress data.
  2. The PEP 517 hooks get called in a subprocess, so simple callbacks wouldn’t work, we’d need some form of IPC.

This could be prototyped by having a backend support being passed something in config_options that acts as a progress reporting channel. Pip could then set that option to interact with the backend. It would be clumsy and not production-ready, but it would allow us to experiment with the idea before trying to standardise an interface.

My instincts say that it would be a major user interface improvement to have this, but almost impossible to actually achieve. I’d be happy to be proved wrong on the latter part of that statement, though :wink:

The way I handle it in tox 4 (and the attached pyproject-api project GitHub - tox-dev/pyproject-api: API to interact with the python pyproject.toml based projects) is to mirror the output of the commands called in a subprocess if the -v flag is specified. Arguments related to the verbosity of the backend can be controlled via the build args :thinking: Do we need anything more complicated?

I think the most sensible solution is to just show stdout/stderr. IMO they should be recorded and presented if the build fails. How does pip currently handle this?

Pip shows backend stdout/stderr when -v is specified. We stopped showing it for normal (non-verbose) invocations because users complained the output was too noisy.

It’s still captured and presented on error.

Actually, it might be possible to do something like “show last 4 lines” or something like that… Would that be an improvement over status quo?

Sorry, I oversimplified.

With rich, it’s certainly possible to do this - have a renderable that displays the last 4 lines of the captured output, scrolling older lines off the top of the widget. See Renderable showing last N lines of output · Discussion #1514 · Textualize/rich · GitHub for an example.

How does that behave with pipes?

1 Like

You mean, when the output is to a pipe not a console? No idea, I’ve not tried it. Probably about the same as a rich progress bar does. It might need some fixing up, but in my experience rich is pretty good at dealing with that sort of thing.

1 Like

The Docker buildx UI does something similar. Under a normal run, it shows the last few lines of the log for each step. If a flag is passed, it no longer use clever terminal scrolling and outputs everything directly. That way you can see output but not be overwhelmed by it unless you ask for that.

2 Likes

One thing to consider is adding something like cargo’s cargo:warning=MESSAGE, which allows build scripts to show warnings to the user even if the output is otherwise hidden.

A lot of modern build tools (e.g. ninja and cargo) show single line progress bars, but I don’t know how easy it is to extract that information and pass it back to pip. For cargo at least it involves parsing json and counting yourself, while for ninja I don’t know about any structure progress output. Again I think writing a specially formatted message to stdout/stderr could be a nice way to communicate that.


I also love the docker buildx output, even though I’m not sure if that much should be shown by default or only with -v.

Pip doesn’t show any progress indication on pyproject.toml builds? Not even the simple output-driven spinner that it shows for setup.py builds?

There’s a spinner, but we’ve gotten a few people asking for more granular output from build backends; ranging from being able to surface warnings to being able to show progress.

I’m not sure if either is worth the effort, but I’m open to ideas. :slight_smile:

The fundamental question here is probably whether we want to amend the PEP 517 hooks to include progress reporting, or to simply try to do something with the existing stdout/stderr streams from the backend (which would be essentially just a QOL improvement for pip).

Personally, my impression is that most of the reporting people are interested in is compiler output, which we (neither the frontend nor the backend) have no control over, so I suspect there’s limits to what we can usefully do. Including a scrolling display in pip of the last N lines of backend output seems like the most feasible option that’s come up here so far. Would it help? I’m not sure…

Me neither. I’ve gone ahead and closed the open requests on pip’s issue tracker for this stuff, directing folks to here so that they can provide feedback here instead of on the issue tracker – I think anything we do here is bound to affect more than just pip (at the very least, some build backends will care about how pip’s output presentation changes).

For setuptools there’s no progress indication in stdout and its output tends to be very noisy, so there it won’t help. Many of the most build-heavy projects will be using a build system like CMake or Meson in the future (or now already), and those know the number of targets they are building and output a counter like [127/1500] target-name-being-compiled. So the last N lines (can be short, even N=1) would help quite a bit.

1 Like

I did a very quick test:

from rich.progress import Progress, SpinnerColumn, ProgressColumn
from time import sleep

class Col(ProgressColumn):
    def __init__(self):
        super().__init__()
        self.cols = []
    def render(self, task):
        return "\n".join(self.cols[-5:])


col = Col()
progress = Progress(SpinnerColumn(), col)

with progress:
    for n in progress.track(range(100)):
        col.cols.append(f"Line {n}")
        sleep(0.1)

It displays the last 5 lines, scrolling, just as I think we’re discussing. And when redirected to a file, it shows the final display, but nothing else. It does the same when piped into something like less. Which may not be exactly what we’d want, but it’s certainly a reasonably friendly, and understandable behaviour.

2 Likes