Ergonomics of new pathlib.Path.scandir()

barneygale · November 21, 2024, 8:08pm

If we decide to revert the addition of Path.scandir() then I’m OK with that, for what it’s worth.

I am interested in adding an abstract version of pathlib.Path, and tbh this was my main motivation for adding a scandir() method. The abstract base class already exists in cpython, but it’s private:

github.com

python/cpython/blob/09c240f20c47db126ad7e162df41e5c2596962d4/Lib/pathlib/_abc.py#L412-L425


      
          class PathBase(PurePathBase):
              """Base class for concrete path objects.
          
              This class provides dummy implementations for many methods that derived
              classes can override selectively; the default implementations raise
              UnsupportedOperation. The most basic methods, such as stat() and open(),
              directly raise UnsupportedOperation; these basic methods are called by
              other methods such as is_dir() and read_text().
          
              The Path class derives this class to implement local filesystem paths.
              Users may derive their own classes to implement virtual filesystem paths,
              such as paths in archive files or on remote storage systems.
              """
              __slots__ = ()

Previously the PathBase.iterdir() method was abstract, but this hampered performance for the PathBase.walk(), glob() and copy() methods, which were implemented using self.iterdir(). These methods are now implemented using self.scandir(), which improves performance a great deal. And it improves performance of Path.copy() too, because Path inherits the PathBase.copy() implementation.

But the pathlib ABCs are still private, and so it’s possible to conclude that none of the above matters. I’m sympathetic to this view and I’m open to adding scandir() only if we decide to make PathBase public, which would require a PEP but not much more technical work.

Some other non-ABC motivation for adding a scandir() method can be found in a previous post of mine