Incrementally move high-level path operations from shutil to pathlib

I’m never quite sure whether to motivate each change in isolation or focus on the big picture.

Short term, I’m only looking to make pathlib.PurePath and pathlib.Path subclassable. You can already subclass the Windows and Posix variants, but you get an AttributeError for the generic classes when you try to instantiate them. This was originally reported as a bug in 2015 and personally I still consider it a bug rather than a feature request. The PR does not introduce a notion of an abstract filesystem. It fixes an exception by simplifying the pathlib internals.

Maybe I should put together a youtube video for the longer-term motivation. In my day job I’m regularly mirroring and munging files between the local filesystem, tarballs, S3, Artifactory, etc. I firmly believe these DevOps-y tasks are major and growing use cases for Python, and that a common object-oriented path interface would be a “killer feature” on par with pathlib’s original introduction.

8 Likes

I’m in the process of converting all my programs to use pathlib. For shell operations, I typically use the sh module, which make it easy to interface with the shell without the complexity of using the SubProcess module. Not having to use os, os.path, sh, shutil and pathlib would simplify the process of updating several hundred programs a great deal; reducing the modules to two, sh and pathlib, would be a delight. I’m not certain that I’d use AbstractPath, but I can think of at least a few sections of code where I would at least examine it in detail to see if AbstractPath could simplify things.

Barney makes a good case to move the implementation of path operations into pathlib; I’d agree that shutil shouldn’t be removed, and I’d think that implementing shutil as a bunch of thin wrappers around pathlib would be ideal as a way to make the transfer simple without breaking existing code (of which I’m sure there is a vast amount).