Installing multiple versions of a package

[CAM-Gerlach]

FYI, this question has been discussed at great length by now, and it is generally concluded that there are too many hurdles to make this practically useful. See

Allowing Multiple Versions of Same Python Package in PYTHONPATH

TLDR; I wanted to get feedback on a potential feature that may be added to nixpkgs that allows multiple versions of the same python package to be installed in the same PYTHONPATH. This is a general approach that is not specific to nixpkgs and could be used in other package managers. The only nix specific part is the tooling to allow for the building of these specialized packages. All of the materials/demo is in this repo https://github.com/costrouc/python-multiple-versions. Sorry discourse preve…

for some of the most recent discussion.

@MrTomRod

pip install pandas==1.1.3 --as=pandas_1_1_3

• Which would create a folder with the name pandas_1_1_3 in PATH, rather than just pandas
• The import syntax could then be import pandas_1_1_3 as pandas

This is basically just vendoring the import package, but but exposed externally as a distribution package. This has a number of issues; listing a few off the top of my head:

• A lot of details would need to be handled and worked out (e.g. distribution metadata), often with non-trivial or non-optimal solutions.
• Non-relative imports within pandas itself would still import from whatever version of the package is installed under its standard import name, which would very likely break, unless there was some magical rewriting done (which could not handle all cases reliably)
• Code wanting to use this version of the package would need to be rewritten too in order to use the new name
• That code would also need full control over what that name is, or be manually rewritten by the user accordingly.
• What would be done about Pandas’s own dependencies? Are they just installed in the regular site-packages? What if they are incompatible with anything else installed (which is a big reason for allowing multiple installs in the first place)? If not, how or where are they installed?

On top of the issues specific to this proposal, it also runs into the same fundamental issues discussed on the above-mentioned thread—if at any point the top-level code calling pandas, or any code that code calls, or any code that code calls, ad nauseum, needs/uses a different version of pandas (and if it didn’t, you wouldn’t need multiple version support in the first place), and anything from any of that code interacts with any of the objects (functions, classes, methods, dataframes, timestamps, etc) from the other copy you’ve installed, you’re likely in all kinds of nasty trouble.

At that point, if you absolutely, positively cannot get it to work in separate virtual environments or with compatibility fixes, you may as well just vendor the packages you need (which is what this is doing anyway), which avoids many of these problems and allows you full control to reliably hack around the rest.