There is already a standardized data directory for packages, I really really do not understand why you need to be able to install to arbitrary locations for your use-cases.
For package configuration, like the use-case you describe, there is also a more pythonic way, having some API to query things.
If you need something more dynamic, you also have entrypoints. Packages can register entrypoints with an arbitrary key, which you could then easily discover.
If there isn’t any good enough API for you to interact with these mechanisms, importlib.metadata could be extended, but I wholeheartedly think that allowing packages to install to arbitrary locations is a big mistake.
This discussion has included the fact that the data directory in wheel’s package-1.0.data/data/ is only there because there was a same-named category in distutils. The category named data is not “a useful place to put your data” because, by mistake, virtualenv and non-virtualenv installations place those files in different locations.
Hence my suggestion for importlib.metadata to be extended. But even ignoring its existence, you can just install the data as a Python package, and you have a pretty good API to access it (importlib.resources), I really do not see the issue. The only new use-case covered by the proposals on this thread is installing to arbitrary locations.
I just wanted to note that this is a problem for the Roundup Issue Tracker as well.
I just spent a couple of hours changing code to try to make it work with a wheel
format install and I am still not close to done.
We have 3 types of files that should not be installed under site_packages:
man pages
locale translations
template files
Setuptools supports entry_points which puts command line programs
on a normal user path. These other items need to be handled similarly IMO.
Thanks to entry_points the commands are available to the users, but the man pages for the commands are buried under site-packages/usr/share/man/… Where no man (command) has ever gone before. (Yes, the user can change MANPATH but there is no way for me (AFAIK) to advise the user that they need to do this when they run pip install roundup.)
Also gettext._default_locale points to a useless directory as the locale files aren’t installed there. The application has to search to find the locale files. Distutils installed them (as data files) in the _default_locale (sys.prefix/share/locale…) dir
so it “just worked”.
Also roundup has template files for the different tracker use cases. These are
useful for the user to view. Having to grovel into .../python3.10/site-packages/.../share/roundup/templates rather than being available under /usr/share/roundup/... where other packages install similar materials fails the principle of least surprise to say the least.
I am trying to figure out if I can detect when it’s being installed in a container or virtualenv so I can allow a wheel install. This will put the three types of files above in an isolated environment. Outside of those environment, use a standard install to make template, locale and man pages available in expected locations.