Add cross-platform user directories to pathlib/standard library

The platformdirs package, and its predecessor appdirs, have been fulfilling a very useful purpose for a long time – they return the paths to the platform-specific data, cache, config, temp, etc. directories, which differ between OSes.

On Linux for example the module provides the output of the various XDG environment variables, but on macOS and Windows it provides the appropriate equivalents.

pathlib already provides Path.home(). And Path objects help a lot with platform-agnosticism. Given platformdirs’s similar mission to pathlib, its relatively small footprint, and its simple nature, would it be reasonable to think about incorporating it into the stdlib, perhaps as from pathlib import PlatformDirs?

7 Likes

OP also raised this on the platformsdirs issue tracker, where the maintainer at least wasn’t against it on principle Adding to stdlib/pathlib? · Issue #265 · platformdirs/platformdirs · GitHub

They mentioned their that their understanding was that the stdlib is trying to get smaller. I wouldn’t quite say that that is true, it just has to be a large enough benefit-to-maintance burden tradeoff. If platformsdirs is stable as a project and decently widely used, adding it to the stdlib I think could have a chance of being done.

But for that, the maintainers best mentioned here how stable the project/concept/ecosystem is. Is it foreseeable that at some point within the next few years a backwards incompatible change becomes necessary? I would doubt that because OSes generally try to be stable across similar timeframes.

Platformdirs is a great project!

The usual question: would end users benefit from it being in the stdlib, with only one feature release per year, rather than on PyPI, with releases as often as they like?

I see 20 releases in 2023 at platformdirs · PyPI. Ignoring patch releases, 14.

In the stdlib, that would be reduced to one per year, plus with no feature changes allowed for the last 5 months before release (first beta to official release).

8 Likes

I agree with Hugo. The problem might appear settled and stable, but we never know when a change in the operating systems or specifications will require adaptations in the lib, which would be very slow to come with Python or not come at all for security branches.

4 Likes

Reasonable concerns of course!

It looks to me like platformdirs uses some sort of automatic release process that results in many small releases, even days apart. Many of 2023’s releases are just for updates to the docs, changes to supported Python versions, linter changes, and so on.

So in reality I don’t think the pace of change is that fast. OS structure and file conventions don’t change much from year to year, after all. It would cause a lot of problems to a lot of software if they did.

The maintainers will have a better idea than me, an outsider, but it feels like once a year would be often enough for feature additions, if any are even necessary.

Sure. Currently though, without using platformdirs, a programmer has to hard code the paths. The consequences of changes to the way an OS does things results in similar things in each case (stdlib integration vs hard-coded):

Non-breaking changes to the recommendations for an OS: people’s code doesn’t break in either case. For the hard-coded case, the programmer would have to pay attention enough to find out about it themselves and actively change the code to meet the new recommendations, but more likely the program would just continue forever to follow the outdated convention. For the stdlib case, it would just be one of the things in the “list of changes in Python 3.1x” to be aware of when updating the Python version used for a project.

Changes to the way an OS does things that breaks the functionality (e.g. permissions changes): every program would be broken in both cases. If using the stdlib, only broken until a bugfix is released. With hard-coded paths, permanently broken until the programmer fixes it. Such a manual fix would also always be an option for anyone using the stdlib anyway. So the inconvenience of it being in the stdlib would be no greater, I would say.

2 Likes

But there’s a third option: use the third-party package. In that situation the fix can come much faster.

/cc @barneygale

Is the likelihood of breakage much higher than other OS-dependent modules in the stdlib like tempfile?

I wasn’t saying it is or it isn’t, just pointing out that your comparison left out the status quo. If there ever were any breaking changes, having a package is better than both of the options you described.

1 Like

I would think at this point that platformdirs is relatively stable with very few behavioral changes over time (As a user of it). There have continued to be convenience methods added over time that could otherwise be received by users more frequently, but the actual platform-specific behavior has not changed frequently (the only cases I’m aware of are in 2021 relating to the original fork from appdirs, fixing appdirs getting something wrong and another fix in 2023 which was a fix, but did change one of the paths returned for macos

Adding this to the standard library could be done in a very similar to the situation to dataclasses, where attrs and the standard library’s dataclasses are two separate namespaces, one updated more frequently than the other, but the standard library covers the base functionality that is the core need, with considering to roll things from platformdirs back into the standard library on a case by cases basis, but that’s probably something worth discussing with platformdirs.

At the same time, I’m not sure the opportunity cost is high here to leave it outside the standard library. This is a pure python dependency with a rather small and easily reviewable code base that does something specific and narrow in scope, and does it well. I would be surprised if there were issues for anyone including this as a dependency even in some of the strictest environments.

OK, then this could be something nice to have in the stdlib, mostly for scripts that want to do the right thing for config / cache / data dirs but have no dependency. I am not opposed to the proposal, but I think all new modules require a PEP now (the last one without was graphlib IIRC).

4 Likes

Would it make more sense to add a new module or to add its functionality (or a subset of it) to pathlib? And would the latter also require a PEP?

No, this is not related to object-oriented abstraction for path manipulation.
It would be its own module, like tempfile is.

4 Likes

I agree with Éric that this doesn’t belong in pathlib. To add an another reason: pathlib only distinguishes between Windows and POSIX, whereas platformdirs has specific algorithms for macOS IIUC.

I quite like the idea of including platformdirs somewhere in the stdlib – it’s very useful but slow-moving and self-contained, similar to tempfile or os.path.expanduser(). But I’m a newish core dev with a small area of responsibility, so take a handful of salt.

8 Likes

I don’t think this can be added. We discussed it in 2010 and never reached a consensus: Define a standard location and API for configuration files · Issue #51424 · python/cpython · GitHub