# Protocol for virtual filesystem paths

**URL:** <https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753>\
**Category:** Ideas\
**Created:** [March 2, 2025, 12:53pm UTC](https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753 "2025-03-02T12:53:23Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![barneygale](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/barneygale/32/1882_2.png) [@barneygale](https://discuss.python.org/u/barneygale)\
**Post date:** [March 2, 2025, 12:53pm UTC](https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753/1 "2025-03-02T12:53:23Z")

</div>

ISTM that `os.PathLike` is doing double-duty:

1. Majority usage: a **local** path with a string/bytes representation
  - e.g. when passed to `io.open()`, `os.mkdir()`, etc

2. Minority usage: **any kind** of path with a string/bytes representation
  - e.g. when passed to `os.path.join()`, `os.path.split()`, etc

In some situations the distinction might matter. For example, it’s perfectly legitimate to take the `posixpath.basename()` of a zip file member path, but much less advisable to call `posixpath.islink()` in the same context. Yet the function signatures don’t reflect the difference. Another such case is `pathlib.PurePath`, which “provides path-handling operations which don’t actually access a filesystem” according to its docs, yet provides a ` __fspath__ ()` method that enables those filesystem-accessing operations:

```python
>>> Path('...').unlink() # success
>>> PurePath('...').unlink() # error: no such method.
>>> os.unlink(Path('...')) # success
>>> os.unlink(PurePath('...')) # success... wait PurePath supports I/O?!

```

IMHO the last case here should (eventually) be an error, but to do that we’d need to deprecate/remove `PurePath. __fspath__ ()` (and make it a `Path`-only thing), and if we did _that_, then purely lexical function like `os.path.join()` would begin rejecting `PurePath` objects for no good reason.

This suggests to me we need something like `os.fspath()` / `os.PathLike` / ` __fspath__ ()` but for pure/virtual paths. I humbly suggest `os.vfspath()` / `os.VirtualPathLike` / ` __vfspath__ ()`, where `VirtualPathLike` is a superclass of `PathLike`. The `os.path` functions listed in the first half of [this GH issue](https://github.com/python/cpython/issues/119671) could be made to call `os.vfspath()`, as they do only lexical work.

I bring this up because, in the pathlib ABCs I’m developing ([discussion](https://discuss.python.org/t/make-pathlib-extensible/3428), [pre-PEP](https://docs.google.com/document/d/18XlclCGWjWnHO5JZSAA7eXaWNERKYENKoEvDP-_MIcc/edit?tab=t.0)), I declare [`JoinablePath. __str__ ()`](https://github.com/python/cpython/blob/214562ed4ddc248b007f718ed92ebcc0c3669611/Lib/pathlib/_abc.py#L62-L66) as an abstract method. I can’t use ` __fspath__ ()` because this type shouldn’t be accepted by `os.unlink()` etc. But ` __str__ ()` doesn’t feel quite right either - for one thing, users might want a string representation that isn’t just a path - perhaps they’d like to use a URL instead for example. I believe what I really need is a ` __vfspath__ ()` dunder to serve as an abstract method in `JoinablePath`.

Thank you for reading. I’d love to hear your thoughts/concerns/ideas!

P.S. Deprecating `PurePath. __fspath__ ()` in favour of `PurePath. __vfspath__ ()` might not be practical even if we agree it’s desirable from a typing perspective (and I’m not taking that for granted either!). I bring it up as a class that will be familiar to many readers; my immediate concern is with `JoinablePath` though.

---

<div class="post-metadata">

**Author:** ![juancer](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/juancer/32/26511_2.png) [@juancer](https://discuss.python.org/u/juancer)\
**Post date:** [March 2, 2025, 10:15pm UTC](https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753/2 "2025-03-02T22:15:59Z")

</div>

I agree with the conflation of lexical and filesystem paths—`PurePath. __fspath__ ()` allowing I/O is indeed misleading. A separate `os.vfspath()` could help, but how would be handled cases where a path can be both lexical and filesystem-backed, like a symlink resolution in virtual filesystems?

---

<div class="post-metadata">

**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\
**Post date:** [March 3, 2025, 8:41am UTC](https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753/3 "2025-03-03T08:41:01Z")

</div>

> [@barneygale](#):
>
> I declare [`JoinablePath. __str__ ()`](https://github.com/python/cpython/blob/214562ed4ddc248b007f718ed92ebcc0c3669611/Lib/pathlib/_abc.py#L62-L66) as an abstract method.

that is:

```python
    @abstractmethod
    def __str__ (self):
        """Return the string representation of the path, suitable for
        passing to system calls."""
        ...

```

I don’t think this can stay. “Passing to system calls” only applicable to _local filesystem_ paths. A JoinablePath does not, in general, have a representation you can pass to system calls.  
You can “downcast” to a local filesystem path, and that could be implicit – “we trust the programmer to only call ` __str__ ` if it makes sense”\[1\]. It would be good to be explicit about where the downcast happens, conceptually, between the JoinablePath and the syscall. Does ` __str__ ` do that (as the docstring above suggests), or do the `os.unlink`/`open` do it?

In my mental model, it happens when an `os` function interprets its argument. That is:

> ```python
> >>> os.unlink(PurePath('...')) # success... wait PurePath supports I/O?!
> 
> ```

No, PurePath does not support I/O. You’re calling `os.unlink`, which interprets its input as a path on the real local filesystem.  
If you do `open(secrets.token_hex(), "w")`, the token doesn’t support I/O either.

* * *

> it’s perfectly legitimate to take the `posixpath.basename()` of a zip file member path

Yes, but taking the `os.path.basename` of it would be, conceptually, wrong.\[2\]  
The reason you can use `posixpath.basename()` is because you know an extra detail about zip files: they behave enough like Posix paths. You can’t use the function if you are working with truly generic JoinablePaths.

Sounds like zip paths can use ` __str__ ` as defined above. Then, zip paths would be special (relative to JoinablePath) in that their ` __str__ ` gives you a posix-path-shaped string.  
Or maybe they should get a ` __posixpath__ ` method, to make this explicit for type-checkers (and brains trying to construct a mental model).  
But I don’t think it would help to add a generic ` __vfspath__ `: that sounds like it would give you a string you can use with a specific “parser” module, but it would erase info about _which_ module to use.

Or did you mean ` __vfspath__ ` to remain tied to the path’s parser module? The class itself would pass it to its parser, but the _user_ shouldn’t call it (since they shouldn’t use the path’s parser directly, there’s nothing they can do with an internal string representation)?

* * *

1. This often works well – if your path is just an alphanumeric ASCII filename, you can treat it as any flavor of path, and implicit conversions simplify the work! 

2. For what’s conceptually correct, it helps to think about a hypothetical platform whose `os.path` uses, say, `$` for path separators. (And assuming all the places where Python assumes `/` & `\` were properly ported.)

---

<div class="post-metadata">

**Author:** ![barneygale](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/barneygale/32/1882_2.png) [@barneygale](https://discuss.python.org/u/barneygale)\
**Post date:** [March 3, 2025, 1:51pm UTC](https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753/4 "2025-03-03T13:51:10Z")

</div>

Thanks Petr. Indeed the `JoinablePath. __str__ ()` is misleading. I’m sorry I didn’t spot this before posting! I guess it should be along these lines:

```python
    @abstractmethod
    def __str__ (self):
        """Return the string representation of the path, suitable for
        passing to lexical path functions like os.path.join()."""
        ...

```

> [@encukou](#):
>
> If you do `open(secrets.token_hex(), "w")`, the token doesn’t support I/O either.

`open(secrets.Token(), "w")` is presumably a `TypeError` though?

> [@encukou](#):
>
> Yes, but taking the `os.path.basename` of it would be, conceptually, wrong.[[2]](#footnote-231279-2)  
> The reason you can use `posixpath.basename()` is because you know an extra detail about zip files: they behave enough like Posix paths. You can’t use the function if you are working with truly generic JoinablePaths.
> 
> Sounds like zip paths can use ` __str__ ` as defined above. Then, zip paths would be special (relative to JoinablePath) in that their ` __str__ ` gives you a posix-path-shaped string.  
> Or maybe they should get a ` __posixpath__ ` method, to make this explicit for type-checkers (and brains trying to construct a mental model).  
> But I don’t think it would help to add a generic ` __vfspath__ `: that sounds like it would give you a string you can use with a specific “parser” module, but it would erase info about _which_ module to use.
> 
> Or did you mean ` __vfspath__ ` to remain tied to the path’s parser module? The class itself would pass it to its parser, but the _user_ shouldn’t call it (since they shouldn’t use the path’s parser directly, there’s nothing they can do with an internal string representation)?

I don’t have good answers for these, but thank you for giving me things to ponder 🙂

---

<div class="post-metadata">

**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\
**Post date:** [March 3, 2025, 4:21pm UTC](https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753/5 "2025-03-03T16:21:26Z")

</div>

AttributeError, since `secrets.Token()` doesn’t exist.  
I wanted to point out that right now, _any_ string can be interpreted as a local path.

---

<div class="post-metadata">

**Author:** ![barneygale](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/barneygale/32/1882_2.png) [@barneygale](https://discuss.python.org/u/barneygale)\
**Post date:** [March 3, 2025, 7:28pm UTC](https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753/6 "2025-03-03T19:28:37Z")

</div>

> [@encukou](#):
>
> I don’t think this can stay.

I wonder if a `segments` abstract property might help? We could define `_JoinablePath. __str__ ()` as a stub method like:

> **(expand code)**
>
> ```python
> class _JoinablePath(ABC):
> """Abstract base class for pure path objects."""
> __slots__ = ()
> 
> @property
> @abstractmethod
> def parser(self):
> """Implementation of pathlib.types.PathParser used for low-level
> path parsing and manipulation.
> """
> raise NotImplementedError
> 
> @property
> @abstractmethod
> def segments(self):
> """Sequence of raw path segments supplied to the path initializer.
> """
> raise NotImplementedError
> 
> @abstractmethod
> def with_segments(self, *pathsegments):
> """Construct a new path object from any number of path segments.
> Subclasses may override this method to customize how new path
> objects are created from methods like `iterdir()`.
> """
> raise NotImplementedError
> 
> def __str__ (self):
> """Return the string representation of the path."""
> if not self.segments:
> return ''
> return self.parser.join(*self.segments)
> 
> ```

Might just be moving the problem around.

---

<div class="post-metadata">

**Author:** ![gerardw](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/gerardw/32/8739_2.png) [@gerardw](https://discuss.python.org/u/gerardw)\
**Post date:** [March 3, 2025, 8:33pm UTC](https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753/7 "2025-03-03T20:33:34Z")

</div>

> [@barneygale](#):
>
> but much less advisable to call `posixpath.islink()` in the same context.

Why is this? Mac / Linux zipfiles support symbolic links.

---

<div class="post-metadata">

**Author:** ![encukou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/encukou/32/2461_2.png) [@encukou](https://discuss.python.org/u/encukou)\
**Post date:** [March 4, 2025, 9:48am UTC](https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753/8 "2025-03-04T09:48:54Z")

</div>

> [@barneygale](#):
>
> Might just be moving the problem around.

Seems like we need to move the problem around if we don’t want a load-bearing ` __str__ `, and there’s a choice to make between ` __vfspath__ ` and `segments`.

---

<div class="post-metadata">

**Author:** ![barneygale](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/barneygale/32/1882_2.png) [@barneygale](https://discuss.python.org/u/barneygale)\
**Post date:** [March 4, 2025, 6:25pm UTC](https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753/9 "2025-03-04T18:25:49Z")

</div>

> [@gerardw](#):
>
> > [@barneygale](#):
> >
> > but much less advisable to call `posixpath.islink()` in the same context.
> 
> Why is this? Mac / Linux zipfiles support symbolic links.

`posixpath.islink()` checks for a link on the _local_ filesystem, so it would be an error to pass in a zip file member path (`ZipInfo.filename`) and expect it to query the zip file data.

---

<div class="post-metadata">

**Author:** ![gerardw](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/gerardw/32/8739_2.png) [@gerardw](https://discuss.python.org/u/gerardw)\
**Post date:** [March 4, 2025, 8:55pm UTC](https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753/10 "2025-03-04T20:55:04Z")

</div>

Still not getting it. You’re discussing how you’re updating Python to work in the future, right?  
_help(posixpath.islink)_ says

```python
Help on function islink in module genericpath:

islink(path)
    Test whether a path is a symbolic link

```

so a future (3.14) zipfile library could implement using the os.path interface instead something like

```python
  # Check if the file is a symlink (using Unix permissions)
  is_symlink = (zipinfo.external_attr >> 16) & 0o120000 == 0o120000

```

---

<div class="post-metadata">

**Author:** ![pf\_moore](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pf_moore/32/35_2.png) [@pf\_moore](https://discuss.python.org/u/pf_moore)\
**Post date:** [March 4, 2025, 9:18pm UTC](https://discuss.python.org/t/protocol-for-virtual-filesystem-paths/82753/11 "2025-03-04T21:18:54Z")

</div>

> [@gerardw](#):
>
> so a future (3.14) zipfile library could implement using the os.path interface instead something like

I thought the discussion was about future implementations of the _pathlib_ interface (because that’s what’s being designed as extensible). Adding extra implementations of `os.path` functions isn’t in the scope of the changes @barneygale is making (as I understand it).