Request for review: pathlib._PathBase

barneygale · August 28, 2023, 12:35am

Hello! Would anyone be able to help review this PR?

github.com/python/cpython

GH-89812: Add `pathlib._PathBase`

python:main ← barneygale:gh-89812-omgtarpath

opened 06:13PM - 02 Jul 23 UTC

barneygale

+687 -161

Add private `pathlib._PathBase` class. This will be used by an experimental PyPI… package to incubate a `tarfile.TarPath` class. ## Implementation notes In `pathlib.py`: - Split `Path` into `_PathBase` and `Path` - Implement `_PathBase.is_junction()` and `is_mount()` using `stat()`. In the latter case, restore the implementation deleted in #31458 - Implement `_PathBase.resolve()` using `absolute()` and `readlink()` - Implement `_PathBase._scandir()` using `iterdir()` In `test_pathlib.py`: - Add `PathBaseTest` to exercise the `_PathBase` class directly - Split `PathTest` into `DummyPathTest` and `PathTest`. The former exercises a representative subclass of `_PathBase`. - Add `DummyPathWithSymlinksTest` to exercise a similar subclass, this time with symlink support. ## Questionable bits In `pathlib.py`, the `_PathBase.is_junction()` and `_PathBase.resolve()` methods are new code. The latter is very performance-sensitive and may need more work. ## Future work - Make `pathlib._PathBase` public. There's still at least a couple of things to do first: - ~~Make `PurePath._flavour` public~~ - Figure out what to do with `_PathBase.__hash__()`, `__eq__()`, and other comparison methods. --- * Issue: gh-89812

The new class sits between PurePath and Path in the pathlib class hierarchy. After extensive discussion elsewhere on the forum and in previous PRs and issues, I’m pretty confident in the overall design. However, I don’t feel comfortable merging until these aspects have had some review:

_PathBase.resolve(), which uses absolute() and readlink() to implement pathname resolution.
_PathBase._scandir(), which uses iterdir(). We might want want to adjust how iterdir() raises exceptions - see discussion on PR.
_PathBase.is_junction(), which uses stat() and feels a bit artificial compared to the other is_*() methods.

Would anyone be willing to look over the code and share their thoughts? Many thanks

ofek · August 28, 2023, 4:28am

This will be great! I had difficulty finding the right magic to satisfy type checkers when subclassing: https://github.com/pypa/hatch/blob/hatch-v1.7.0/src/hatch/utils/fs.py#L14-L19

barneygale · August 28, 2023, 12:52pm

That’s a slightly different thing! It wasn’t possible to directly subclass PurePath and Path (issue here), but it’s fixed in Python 3.12 (PR here).

This feature is about providing a base class that can be used to implement things like TarPath, FTPPath, etc, similar to how io.IOBase helps users to implement custom file objects.

RonnyPfannschmidt · September 11, 2023, 11:35am

a nice addition would be if we had acessor paths, where the path object is implemented in terms of accessor

an accessor could be something like posix, windows, windows+unc, tarfile, zipfile, “another path”