(I’m splitting off from the PEP 627 thread, but I’ll repeat the relevant information here.)
It is currently not clear how projects that are installed on a system can be uninstalled.
For background: installed projects are recorded .dist-info directory, as specified PEP 376 (which I hope to amend by PEP 627). The directory contains several interesting files: RECORD, a list of files with hashes, INSTALLER, the name of the tool that installed the project, and REQUESTED, present iff the project was not installed automatically/as a dependency.
There are several ways I (and others) can think of:
Projects can only be uninstalled by the tool that installed them. The INSTALLER file looks handy for this. However, if interoperability between tools is an overall packaging goal, it does not make sense to default to not touching packages installed by other tools. (AFAIK, pip nows uses this as one of its heuristics, and I believe it’s an OK thing to do in the mean time, before things are standardized.)
There are also questions related to tools delegating to each other. Tzu-ping Chung noted:
There’s actually a significant number of Conda packages out there right now packaged improperly to have their INSTALLER files say pip . I wouldn’t worry about this too much since very few people bother to complain when pip “incorrectly” uninstalls those Conda-installed packages.
Projects can be uninstalled by removing all files listed in RECORD. If RECORD is missing, then a project cannot be uninstalled (unless the tool is following other standard than PyPA: e.g. system package managers are expected to omit RECORD and uninstall in their own way). To me this seems reasonable, but it does assume things that, I guess, might not always be true:
RECORD lists all files needed to uninstall a project
deleting a bunch of files is sufficient to uninstall a project
Tools should record a shell command that will uninstall a project.
This looks viable, but I feel it could be tricky to pull off, especially if mixing virtualenvs (e.g. using a globally installed tool to install/uninstall to/from a virtualenv).
(It was suggested this be recorded in an existing file like INSTALLER or REQUESTED, but since those already exist with other content, so a new file like UNINSTALL.sh seems more appropriate.)
Or something else? Or a combination of the above?
Of course, the same questions (and more) can be asked for upgrading an installed project.
I’m strongly against (3). Tools would need to be able to write future-proof shell commands for every type of OS that they supported. There are way too many edge cases here to make this something I’d be comfortable with. And there’s also security questions - would the command be executed with shell=True? It sounds like that’s the intention, but that opens up all sorts of possible exploits.
I think the assumptions in (2) are probably reasonable. We don’t have post-install script support, so installation is nothing more than creating a bunch of files. Deleting that same bunch of files (plus any associated .pyc files) seems fair enough. We will be pretty much locking in the decision to not support post-install/pre-uninstall scripts, though.
The interoperability argument against (1) seems OK, but I feel that there might be cases where a tool wants to say “don’t mess with this, it’s mine” while still taking advantage of RECORD. And I can also see people wanting to write RECORD because it’s useful information, while still not wanting people to uninstall. Audit tools, for example, might complain if you don’t write RECORD even though they have no intention of uninstalling.
Maybe another option would be to say that tools must not uninstall projects with an INSTALLER value that specifies a different tool, but we have a special value (maybe a wildcard *) which says “any tool can uninstall this”?
That would let us go back to saying that RECORD is mandatory, so maybe this is something we should consider before accepting PEP 627?
pip does not look at INSTALLER for packages that are not pip.
The only time that pip looks at INSTALLER, is when it’s about to print the message to nudge users to upgrade it. If pip’s own INSTALLER is not pip itself, it does not print the upgrade message nudging the user to run pip install --upgrade pip.
I don’t think so.
If I understand correctly, we’d have to add at least another metadata field/file to support post-install/pre-uninstall steps. So, if/when we add such functionality, we’d be changing the uninstallation process from “remove the files list in RECORD” to “use <pre-uninstall-mechanism>, then remove the files listed in RECORD + what <pre-uninstall-mechanism> gave you”. In both cases, we have to look at the RECORD (either directly, or through whatever <pre-uninstall-mechanism> we settle on), so using that as an indicator of “you can’t uninstall this” is sufficient IMO.
For completeness, my position on these right now is (-0.5, +1, -1).
From my POV, when dealing with uninstalling projects, the question “which tool installed me”, isn’t actually the right thing to ask, but is instead being used as a crude proxy for “is the Python metadata the source of truth for this installation or not?”.
So I would lean heavily to a solution that explicitly denoted when the Python metadata is not the source of truth, and have tools not touch packages like that. Perhaps this could be something as simple as dropping a file in .dist-info called EXTERNALLY-MANAGED, whose presence tells Python tooling that we don’t own this installation, and thus shouldn’t modify it (but we can still interrogate it to display information, or as inputs into a dep solver or whatever).
Well, that’s as easy as not installing the RECORD file :)
I currently find the Audit tools might complain if you don’t write RECORD argument quite weak. Are there actually audit tools that work with packages that use PyPA metadata but weren’t installed with PyPA tools/standards? What are the use cases here? Is RECORD even the right thing for those use cases?