We pretty much have that already in sys.stdout.isatty(). Thatâs what I use in my SO answer to detect interactive vs non-interactive output.
Iâll drop the class I defined for that answer inline, as it sits at the level of complexity where I think the stdlib could reasonably play:
class ProgressBar:
"""Display & update a progress bar"""
TEXT_ABORTING = "Aborting..."
TEXT_COMPLETE = "Complete!"
TEXT_PROGRESS = "Progress"
def __init__(self, bar_length=25, stream=sys.stdout):
self.bar_length = bar_length
self.stream = stream
self._last_displayed_text = None
self._last_displayed_summary = None
def reset(self):
"""Forget any previously displayed text (affects subsequent call to show())"""
self._last_displayed_text = None
self._last_displayed_summary = None
def _format_progress(self, progress, aborting):
"""Internal helper that also reports the number of completed increments and the displayed status"""
bar_length = self.bar_length
progress = float(progress)
if progress >= 1:
# Report task completion
completed_increments = bar_length
status = " " + self.TEXT_COMPLETE
progress = 1.0
else:
# Truncate progress to ensure bar only fills when complete
progress = max(progress, 0.0) # Clamp negative values to zero
completed_increments = int(progress * bar_length)
status = (" " + self.TEXT_ABORTING) if aborting else ""
remaining_increments = bar_length - completed_increments
bar_content = f"{'#'*completed_increments}{'-'*remaining_increments}"
percentage = f"{progress*100:.2f}"
progress_text = f"{self.TEXT_PROGRESS}: [{bar_content}] {percentage}%{status}"
return progress_text, (completed_increments, status)
def format_progress(self, progress, *, aborting=False):
"""Format progress bar, percentage, and status for given fractional progress"""
return self._format_progress(progress, aborting)[0]
def show(self, progress, *, aborting=False):
"""Display the current progress on the console"""
progress_text, progress_summary = self._format_progress(progress, aborting)
if progress_text == self._last_displayed_text:
# No change to display output, so skip writing anything
# (this reduces overhead on both interactive and non-interactive streams)
return
interactive = self.stream.isatty()
if not interactive and progress_summary == self._last_displayed_summary:
# For non-interactive streams, skip output if only the percentage has changed
# (this avoids flooding the output on non-interactive streams that ignore '\r')
return
if not interactive or aborting or progress >= 1:
# Final or non-interactive output, so advance to next line
line_end = "\n"
else:
# Interactive progress output, so try to return to start of current line
line_end = "\r"
sys.stdout.write(progress_text + line_end)
sys.stdout.flush() # Ensure text is emitted regardless of stream buffering
self._last_displayed_text = progress_text
self._last_displayed_summary = progress_summary
Key features:
- bar length and output stream can be set per instance.
- display elements (such as the prefix and the status text messages) can be customised via subclassing (and you could easily add
PERCENTAGE_PRECISION, BAR_SYMBOL_COMPLETE and BAR_SYMBOL_INCOMPLETE to the set of elements customisable that way)
- only updates interactive streams if output changes (repeatedly writing same output is a common SO answer bug that is slow on any stream, and super spammy for non-interactive ones)
- only updates non-interactive streams if the progress bar state or status text change (limits number of messages to the number of progress notches when non-interactive, rather than potentially emitting 1000 messages if every possible percentage value is emitted)
- seeks to the beginning after each line, so other output will appear over the top of the progress bar rather than starting off to the right of screen
- if other output is emitted between calls to
show(), then the progress bar will simply emit a new output line once the progress status changes
- the
format_progress method allows the formatting behaviour to be used without tying it directly to the IO operation
One thing this class doesnât do, but a stdlib version should is to make stream either a read-only property, or else a read/write property that resets the output history when modified.
For anything beyond the level of what this class offers, Iâd put a few recipes in the standard library, and then mention the availability of third party progressbar implementations, either in libraries dedicated to that purpose, or in more general command line utility libraries.
The recipes Iâd suggesting including would be:
- using a subclass to customise the output
- calling
bar.show(progress) from a callback function in an API like shutil.copytree (see below)
- using it to make an iterator wrapper that updates the progress bar (see below)
Example of integrating with a callback API:
import os
from shutil import copy2, copytree, ProgressBar
def count_files(src, ignore=None):
"""Recursively count files in a tree (respecting a `copytree` `ignore` filter)"""
total_files = 0
for this_dir, dirnames, filenames in os.walk(src):
if ignore is None:
total_files += len(filenames)
continue
ignored_names = ignore(this_dir, dirnames + filenames)
# Don't count ignored files
total_files += sum(1 for name in filenames if name not in ignored_names)
# Don't iterate over ignored directories
dirnames[:] = [name for name in dirnames if name not in ignored_names]
return total_files
def copytree_with_progress(src, dst,
symlinks=False, ignore=None,
copy_function=copy2,
ignore_dangling_symlinks=False,
dirs_exist_ok=False):
"""Display a console progress bar while copytree is running"""
progress_bar = ProgressBar()
total_files_to_copy = count_files(src, ignore)
def copy_with_progress_update(src, dst, *, follow_symlinks=True):
nonlocal files_copied
result = copy_function(src, dst, follow_symlinks=follow_symlinks)
files_copied += 1
progress_bar.show(files_copied / total_files_to_copy)
return result
progress_bar.show(0.0)
copytree(src, dst, symlinks, ignore, copy_with_progress_update,
ignore_dangling_symlinks, dirs_exist_ok)
progress_bar.show(1.0)
Wrapping an iterator or iterable:
from shutil import ProgressBar
def iter_with_progress(iterable, *, max_iterations=None):
"""Display a progress bar while iterating over an iterable"""
if max_iterations is None:
# Iterable must define __len__ if max_iterations is not given
max_iterations = len(iterable)
progress_bar = ProgressBar()
progress_bar.show(0.0)
items_processed = 0
for item in iterable:
yield item
items_processed += 1
progress_bar.show(items_processed / max_iterations)
if max_iterations is not None and items_processed = max_iterations:
break # Terminate now even if the underlying iterator isn't complete
# Passing in a sequence means the maximum progress value is determined automatically
from pathlib import Path
all_files = list(path for path in Path(input_dir).rglob("*") if path.is_file())
processed_files = [process_file(fpath) for fpath in iter_with_progress(all_files)]
# Alternatively, the maximum progress value can be passed in explicitly
num_files = sum(1 for __ in Path(input_dir).rglob("*") if path.is_file())
iter_files = (path for path in Path(input_dir).rglob("*") if path.is_file())
processed_files = [process_file(fpath) for fpath in iter_with_progress(iter_files, max_iterations=num_files)]
As far as âWhy in shutil?â goes, part of my motivation is that shutil has some example use cases for a progress bar (copytree now, and maybe someday make_archive), and the rest is that it and argparse are the only real contenders, and argparse has never really been a general purpose CLI utility library, while shutil already has get_terminal_size().
There would then be several things I would declare as explicitly out of scope and leave them to third party libraries (yes, this is inspired directly by Serhiyâs list of the more complex cases that arise):
- displaying multiple concurrent progress bars
- displaying hierachical progress rather than linear progress
- displaying anything other than a simple completion percentage against a fixed target
- displaying more complex output than a fixed length bar that changes from one text character to another
For all those use cases, the recommendation should be to reach for a specialised third party library, since they need more advanced console manipulation tools than a simple \r character in the output stream. (edit: according to tqdmâs readme, Iâm wrong about that requirement. It still needs more complex code than this to achieve it, though)
The one other utility that I think might reasonably live in the stdlib is a basic rotating -\|/ activity indicator with some text after it that can be used when you donât have any useful way to estimate progress, but do want to indicate that the process is still doing something when running at an interactive console.
Iâve got an untar script which