I was writing this up, and I’m not a 100% sure that I did not miss any major redistributor with their own package manager? Here’s the list so far:
I guess one good group to reach out to would be Homebrew / Linuxbrew. I can’t think of who else though, so thoughts on that are welcome!
It’s worth mentioning Spack I think. It works in a very similar way to Conda.
(Conda [3] is a bit of a special case, as the conda command can install much more than just Python packages, making it more like a distro package manager in some senses.
I don’t actually think about it like this at all. The conda base environment is like a Linux distro, and you should never mess with it as a user. However, user-created conda environments are conceptually similar to Python’s virtual environments, just more powerful. They do environment activation, and have a “one environment for one development task” philosophy. The environment activation does a bunch of magic, that’s why you can’t layer a virtualenv on top of an activated conda env. And a major part of the reason why you need environments is because the package manager/repository provides multiple versions of every package - just like PyPI, and unlike Linux distros and Homebrew.
The above all applies to Spack environments too I believe.
This makes the “using a virtual environment is best-practice” advice more generalizable. We have two kinds of Python installs:
base env Python, etc.The advice for all of these is: never mess with (1), always work in (2).
The NixOS Python docs page linked from the PEP is kind of illustrative in this regard: it has a python-with-my-packages where everything is in the “base” OS packages and that’s immutable so you can’t pip install anything, and then further down it explains how to get virtual/conda/micromamba/mach-nix environments - all user-level dev envs where you can install whatever you want.
I like this characterisation. Apart from being a good model for this discussion, it also leads nicely into the (completely off-topic for this discussion, but maybe worth having a separate thread on at some point) question of shipping Python “applications”, which is essentially the question of how you take your code and bundle up the essential aspects of your “dev environment” in a way that allows your code to be deployed to a “base” environment without violating the “never mess with the base environment” rule.
FWIW, I’ve started a slightly bike-sheddy topic around this PEP: Renaming PEP 668
If I remember correctly that statement was less about conda’s capabilities with the ability to create virtual environments, and more about the fact that it cannot rely solely on the Python level metadata because a number of its Packages have nothing to do with Python at all. Thus, in this case, it’s operating more like a Linux distribution in that it has its own database of installed packages that exists outside of the Python level metadata.
Granted conda itself has integration with pip, but pip doesn’t integrate with conda.
Though if I recall correctly the PEP doesn’t expect conda to write the “don’t touch me” file, just the sysconfig split.
Based on the discussion in Renaming PEP 668, I’m going to go ahead and make a PR changing the PEP title to the following.
Marking Python base environments as “externally managed”
If someone has concerns around this renaming, please bring those up in that thread.
Alrighty, are there any other folks that we should reach out to? I only have Spack and Homebrew on my list of groups to reach out to for this.
@rockobonaparte What’s the story for Chocolatey potentially implementing the bits that this PEP proposes?
There’s also ActiveState Python here, that I missed earlier!
MacPorts is a major distributor of Python, third-party Python packages, and many other open source packages on macOS using their own package manager. It was based on the FreeBSD ports package manager also still in use.
There’s another package manager for macOS called fink that distributes Python.
On Windows, Cygwin and MSYS2 might also be interested, especially with all the patching they have to do to their Python distributions.
In Nixpkgs/NixOS there is not really any difference between system packages and user packages. With NixOS and Nix in general the system and users can have profiles where they can install packages/environments, or they can open a shell with whatever packages/environments. The built packages are exactly the same and will be shared as well. Everything built with Nix is read-only. Using pip or conda outside of Nix means you have to give it a writable folder so it can work.
If I understand the PEP correctly in Nixpkgs we will add a EXTERNALLY-MANAGED file to our CPython build. For NixOS we will not add any directories for system-wide local installation because that goes against our way of working.
Link to the “official” documentation regarding Python in Nixpkgs is btw NixOS - Nixpkgs 21.11 manual.
I’ve reached out to the Spack folks, over the Slack channel that they have for the project.
Quote from @tgamblin, about Spack and this PEP:
We are trying to move people to environments so they don’t modify the existing python installation directory (as it’s likely that the python install is a dependency of many things with different needs). Ideally it’d be read-only, and they could use pip in an env. We’d probably mark the installed python as externally managed and not the python in the spack environment
So, if I’m understanding correctly, this would be useful to Spack!
@pf_moore (and others as well) Is the following list of redistributors of Python good-enough to show that this PEP is reasonably generally applicable? (this was the concern flagged in PEP 668: Marking Python base environments as "externally managed" - #6 by pf_moore)
I’d like to get a better sense if there’s any need to reach out more redistributors for establishing that this PEP would be useful+beneficial overall. Primarily because I don’t think we want to go to every potential redistributor of Python. 
Seems pretty good to me. The only other ones I can even think of are Homebrew and Pyenv, but like you say, we’re not trying to be exhaustive here.
From what I understand based on past interactions, Homebrew doesn’t specify a groups of people working on policy decisions like this. Homebrew maintainers (as in ensuring brew to work as intended) aren’t that interested in ecosystem-specific things, and generally don’t really know who to talk to either. So what I would do is to simply make the decision, and contribute the externally-managed marker to Homebrew afterwards. Hopefully “the community” will welcome the change, but if not, things continue as-is there and we won’t get complaints either.
And, as a pyenv user, I don’t think it would be adding such a marker file — pyenv doesn’t manage Python packages, but instead Python installations.
Although the [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv) plugin does manage virtualenvs. Those would not be marked as ‘externally managed’ though, as the normal packaging tools are used to maintain them.
Hi, Homebrew maintainer here. We welcome discussion about big changes in projects we distribute, especially those aimed at the distro level like this. If a project wants to talk with us, a discussion or issue or PR in our homebrew-core repo is the perfect place. We just don’t have dedicated people for parts of the packages, so we can’t point out to one person and say “this is our Python person”.
Regarding the PEP, I have read through it and I think it addresses an important issue. We will discuss how to integrate that in Homebrew in future releases, but we probably want to mark the base environment in such a way. Probably some users will complain at the beginning, as will all changes, because it disrupts some workflows, but in the long term it will be less pain for everyone.
Another Homebrew maintainer here 
I’ve volunteered to be the point-of-contact on Homebrew’s side for this PEP, to reduce the number of independent communication channels. I’ll continue to follow this thread and can offer insight into Homebrew’s current policies/work to make them compatible with this PEP.
Awesome, thanks folks. I’ll mark Homebrew down as supportive of the PEP, even if the story of “how to transition to that” is unclear.
@pf_moore do you want one of the authors to update the PEP with references to the various mentions of support, quoted from this thread with back links or is the broad expression of support in this thread sufficient?
I think this is the last piece before I ask for the PEP delegate for a decision. If someone has outstanding concerns, now’s the time to flag them.