Should there be a new standard for installing arbitrary data files?

Packaging
63

Fedora Python maintainer here. I need to pitch in as somebody who fundamentally disagrees that desktop files or manual pages “don’t work under sys.prefix” or that installing stuff to an arbitrary location under sys.prefix has an increased potential to create file-conflicts. It is matter of perspective. Let me describe a matrix of the following things I could think of:

Python modules in {sys.prefix}/.../site-packages

Python modules: sys.prefix is /usr

  • The modules work naturally because they are by definition in sys.path.
  • There is a potential of file-conflict between different Python packages.
  • There is a potential of file-conflict between pip-installed packages and distro-package manager.

Python modules: sys.prefix is /usr/local or ~/.local

  • The modules work naturally because they are by definition in sys.path.
  • There is a potential of file-conflict between different Python packages.

Python modules: sys.prefix is within Python virtual environment

  • The modules work naturally because they are by definition in sys.path.
  • There is a potential of file-conflict between different Python packages, albeit the chances are very limited, as virtual environments tend to be one-purpose and the package set usually does not grow without bounds.

Python modules: sys.prefix is another arbitrary location including Windows

  • The modules work naturally because they are by definition in sys.path.
  • There is a potential of file-conflict with different Python packages.

Commands (scripts) in {sys.prefix}/bin

Commands: sys.prefix is /usr

  • The commands work naturally because /usr/bin is (almost) always on $PATH.
  • There is a potential of file-conflict between different Python packages.
  • There is a potential of file-conflict between pip-installed packages and distro-package manager.
  • There is a potential of file-conflict between pip-installed packages and other language stack package managers that would also install scripts into this location, or manually created content.

Commands: sys.prefix is /usr/local

  • The commands usually work because /usr/local/bin tends to be on $PATH.
  • If needed, users can extend their $PATH easily to make it work.
  • There is a potential of file-conflict between different Python packages.
  • There is a potential of file-conflict between pip-installed packages and other language stack package managers that would also install scripts into this location, or manually created content.

Commands: sys.prefix is ~/.local

  • The commands usually work because ~/.local/bin tends to be on $PATH on modern distros.
  • If needed, users can extend their $PATH easily to make it work.
  • There is a potential of file-conflict between different Python packages.
  • There is a potential of file-conflict between pip-installed packages and other language stack package managers that would also install scripts into this location, or manually created content.

Commands: sys.prefix is within Python virtual environment

  • The commands don’t work unless the virtual environment is activated: activate script adds the directory to $PATH.
  • Users can add symbolic links to scripts in a virtual environment to directories on their $PATH.
  • There is a potential of file-conflict between different Python packages, albeit the chances are very limited, as virtual environments tend to be one-purpose and the package set usually does not grow without bounds.

Commands: sys.prefix is another arbitrary location

  • The commands don’t work unless user modifies their $PATH.
  • Users can add symbolic links to scripts in arbitrary locations to directories on their $PATH.
  • There is a potential of file-conflict witch anything else.

Commands: Windows

  • This is handled differently on Windows and it seems to work.

Manual pages in {sys.prefix}/share/man

Manpages: sys.prefix is /usr

  • The manual pages work naturally because /usr/share/man is (almost) always in manpath.
  • There is a potential of file-conflict between different Python packages.
  • There is a potential of file-conflict between pip-installed packages and distro-package manager.
  • There is a potential of file-conflict between pip-installed packages and other language stack package managers that would also install manual pages into this location, or manually created content.

Manpages: sys.prefix is /usr/local

  • The commands usually work because /usr/local/share/man tends to be in manpath.
  • If needed, users/distros can extend their config easily to make it work.
  • There is a potential of file-conflict between different Python packages.
  • There is a potential of file-conflict between pip-installed packages and other language stack package managers that would also install manual pages into this location, or manually created content.

Manpages: sys.prefix is ~/.local

  • The manual pages usually work because ~/.local/share/man tends to be on manpath on modern distros.
  • If needed, users/distros can extend their config easily to make it work.
  • There is a potential of file-conflict between different Python packages.
  • There is a potential of file-conflict between pip-installed packages and other language stack package managers that would also install manual pages into this location, or manually created content.

Manpages: sys.prefix is within Python virtual environment

  • The manual pages don’t work out of the box.
  • Users can make them work by setting/extending $MANPATH.
  • If deemed useful, the activate script could be improved to set/extend $MANPATH.
  • Users can add symbolic links to manual pages in a virtual environment to directories on their manpath.
  • There is a potential of file-conflict between different Python packages, albeit the chances are very limited, as virtual environments tend to be one-purpose and the package set usually does not grow without bounds.

Manpages: sys.prefix is another arbitrary location

  • The manual pages don’t work out of the box.
  • Users can make them work by setting/extending $MANPATH.
  • Users can add symbolic links to manual pages in arbitrary locations to directories on their manpath.
  • There is a potential of file-conflict witch anything else.

Manpages: Windows

  • The manual pages are not relevant there but they don’t hurt anything.

Desktop files in {sys.prefix}/share/applications

(This also applies to their icons in {sys.prefix}/share/icons or {sys.prefix}/share/pixmaps.)

Desktop files: sys.prefix is /usr

  • The desktop files work naturally because /usr/share/applications is used by default.
  • There is a potential of file-conflict between different Python packages.
  • There is a potential of file-conflict between pip-installed packages and distro-package manager.
  • There is a potential of file-conflict between pip-installed packages and other language stack package managers that would also install desktop files into this location, or manually created content.

Desktop files: sys.prefix is /usr/local or ~/.local

  • The desktop files work naturally because /usr/local/share/applications and ~/.local/share/applications is used by default.
  • There is a potential of file-conflict between different Python packages.
  • There is a potential of file-conflict between pip-installed packages and other language stack package managers that would also install desktop files into this location, or manually created content.

Desktop files: sys.prefix is within Python virtual environment

  • The desktop files don’t work.
  • Users might be able to make them work by some configuration (I have not explored this).
  • Users can add symbolic links to desktop files in a virtual environment to the directories that work with desktop files.
  • There is a potential of file-conflict between different Python packages, albeit the chances are very limited, as virtual environments tend to be one-purpose and the package set usually does not grow without bounds.
  • If somebody wants to explore new ideas, we can have a concept of “activating a virtual environment for your desktop environment”, but I don’t think it would be that useful.

Desktop files: sys.prefix is another arbitrary location

  • The desktop files don’t work.
  • Users might be able to make them work by some configuration (I have not explored this).
  • There is a potential of file-conflict witch anything else.

Desktop files: Windows

  • The desktop files are not relevant there but they don’t hurt anything.

Static application data in {sys.prefix}/share/{app_name}

(E.g. Jupyter kernels.)

  • They always work regardless of sys.prefix.
  • There is a potential of file-conflict witch anything else that uses app_name.

tl;dr

  • The stuff works quite fine for many values of sys.prefix; the degree of “works out of the box” varies, but there is some potential for improvement as well.
  • The potential for file-conflicts is not worse than the existing potential (you can nuke a system by installing bash or sh script quite fine already even without data_files).
  • Many files are useless on Windows but I consider that OK.
2 Likes