Speaking for Debian, if it installs into /usr/local/ or /opt with binaries /usr/local/bin or similar, it shouldn’t break the system.
If it does, that’s probably a bug in a system package. We strongly encourage distro packages to use a #!/usr/bin/python3 shebag, not !#/usr/bin/env python3.
We use dist-packages precisely so that a locally installed python can have site-packages in /usr/local/ without conflicting with the distro python.
I do not understand this. Doesn’t the Debian apt-installed Python have both a dist-packages and a site-packages? If I sudo /usr/bin/python3.10 -m pip install something (which I can not recall having ever done, I could try in a container) does it get installed in /usr/lib/python3/dist-packages? I thought only apt-installed packages would be installed there, isn’t that so?
Actually, it seems off-topic, so no need to answer here, I can probably figure out by myself.
+1
I also recommend /opt. The FHS defines that /opt is used for add-on and 3rd party application software. A PSF / python.org Python installer can be considered add-on software.
I suggest /opt/pythonorg as a base directory with the same directory layout as /usr, /usr/local, and virtual envs. There would be a /opt/pythonorg/bin/python3.12 interpreter with code in /opt/pythonorg/lib[64]/python3.12. Users can then either add /opt/pythonorg/bin to their PATH or create virtual envs with /opt/pythonorg/bin/python3.12 -m venv venv. It might even be a good idea to mark the site-packages directory as externally managed, so users cannot mess up their installation and have to use virtual environments.
That’s a little painful for users (not being on the default PATH), but can be mitigated with tooling.
As soon as you are in a virtualenv, it’s not an issue any more.
This means doing a simple edit in .bash_profile right?
Yes, that’s all.
Its a pattern to use the web address like this /opt/org.python/whatever you might consider also doing this.
I was thinking about using the py launcher like on Windows, e.g. py -3.12 -m venv venv
I used to build Python from source, that was over 20 years ago. The biggest problem I encountered was not Python itself, but the extensions that required shared libraries. Version incompatibility issues were popping up frequently. Several years ago I discovered conda, now I cannot imagine a life without it. I can install multiple environments with different versions of Python, all isolated from each other and from the system’s Python, which I do not touch. Installing extensions is a breeze.
Occasionally things go awry, conda fails to sort out dependencies, but then it is (almost) trivial to rebuild the failed environment from a scratch. Recently, I switched from conda to mambaforge to speed things up. I don’t need another Linux installer.
I don’t want to be totally negative about conda. It’s an important project, that makes easy to use python for scientific purposes. But the way conda install Python and packages modules is quite unorthodox, and I personally run in some problems with conda. For example, the fact that in conda pure python packages wins against machine specific ones is completely different from the behavior of pip, and, IMHO, completely wrong.
That said, I like the idea to have a simple way to install Python on Linux in /opt simply downloading it from python.org
I don’t quite understand. Do you mean that venv inherits from the base environment? If that is so, I prefer the conda way, it gives me full isolation.
I am not against the proposal, I just do not need such capability.
I mean that if you publish on conda a py package, it gets the precedence even if you published also machine specific packages. This happens for C extensions. With pip, machine specific packages have the priority.
I’m not saying conda is not good, but it’s a very custom project, as well as the Debian python packages are very custom.
There’s a large arsenal of possibilities to determine precedence of different variants (e.g. higher build numbers, adding so-called track_features which weigh down the build for the resolver, etc.). You don’t have to like conda, but please avoid blanket statements for things you’re not familiar with.
The scenario you describe is also very unusual (having both a generic and a machine-specific build for the same package & version), and it’s impossible with C extensions, because those need to be compiled, which is never machine-independent.
When would anyone who wants this not be better off with pyenv?
PATH easilypip vs pip3 vs python3 -m pip hoo hahs)$HOME as a regular user (no urge to break everything using sudo or pip’s --user option)--enable-shared build or to include extensions like dbm.gnuThe Windows and macOS installers are each lacking quite a few of those (what I’ve come to think of as) requirements so they feel pretty deficient once you get used to pyenv.
While I think pyenv does a laudable job with a difficult task, it has a lot of user experience issues that I would not recommend until someone really has to manage lots of Python environments.
To name a few issues why I don’t recommend it by default, it’s a considerable cognitive load to understand which environment applies when (shell, global, local, virtualenv, etc.), it doesn’t used optimization flags by default (leading to users often not realising they have slower installs), and there are issues with the way the shimming works such as having to use pip install ... instead of python -m pip install ... for entry points to be available after the install.
Supposing an external option that isn’t on python.org, I’d rather point people at the reproducibly built standalone distributions here that only need to be extracted, and depending on where extracted to, optionally added to their path. These will work out of the box for more users, and come prebuilt with optimizations including having multiple downloads based on CPU, including down to x86_x64 level for x86_64 processors
Compiling from source, even with a tool that provides a recipe, is not something users impacted by the issues that have continually been raised around Linux distribution of Python tend to do (or they wouldn’t have issues like those related to Debian deciding to omit certain functionality)
The point of a Linux installer on python.org as I see it would be to guide users who aren’t going to build from source and understand all of the configuration options to a reasonable default that doesn’t have distribution-specific surprises, while still allowing distributions to provide their own configured how they may.
By default, pyenv doesn’t use link-time and profile-guided optimisation (LTO and PGO) (see pyenv/pyenv#2577), whereas the official macOS and Windows do, making them ~30% faster than pyenv.
I have to say I, as the lead Python maintainer for SUSE/OpenSUSE feel offended by this post.
Do you on OpenSUSE do what Fedora does and package lots of python versions for parallel install?
As a Fedora user I have always had lots of python versions available to test against and use. At the moment I have python 3.6 through to 3.13 available.
Edit: Oh and also python2.7
they are often slightly delayed as far as availability goes (compared to other platforms where you can easily install it on days zero) or not available at all because of the LTS nature of these.
OpenSUSE provides packages for Python-3.13.0rc1.tar.xz, and although it takes us couple of days to publish our packages, we are usually providing patches for CVEs and serious bugs before the upstream releases their release. And, of course, we maintain the same fixes for long obsolete versions for our enterprise-class distros (have you backported anything to Python 3.4 lately? I did, two weeks ago, and there is another patch to be backported after I return from holidays).