Drawing a line to the scope of Python packaging

[pf_moore (Paul Moore)]

@teoliphant

This channel is about drawing a line to the scope of Python packaging. Will pip be used to package and install Python itself? Will pip be used to package and install Java? Julia? I believe the answer should be no

I think the answer to that is clearly and self-evidently “no”. I don’t think anyone imagines otherwise - although it’s possible that not everyone draws the same conclusion from that inference that you’re suggesting.

@teoliphant

there must be language at packaging.python.org that helps people understand that you should not expect ‘pip install’ to be the only way to install every Python package. Perhaps it can be used to install the ‘python-parts’ of the package but some of the things that must be installed for the solution to work should be installed by other package managers.

I think that’s a fair suggestion in isolation. But it still avoids the question of how tools work together. If I’ve installed Python using the system package manager (on Windows, the python.org installer) and then used pipenv to set up my application development workflow, and as a consequence used pip to install several packages, and I now want to install something that “should be installed by another package manager” (which may be conda, but could be something else that sits in the same space as conda), we’re currently in a position where that’s not possible, and the developer has to unwind all the way back to installing Python with conda, and then looking for a conda-compatible equivalent to pipenv for their workflow. My contention is that doing so isn’t a practical option for the majority of people, and so we have 2 options:

1. Tell those people that they can’t use the package that they are trying to use.
2. Offer an option to use that package via pip (or any other toolchain that does work without that “rewind”) - probably with caveats that there may be integration issues, and those will be down to the user to address for themselves as their chosen toolchain doesn’t have the means to manage the issues automatically.

The problem I have is that at the moment we’re talking about helping people to make informed decisions right at the start of their projects, but glossing over the fact that the technical limitations of the tools mean that we’re expecting them to make difficult-to-change decisions before they have the information needed (specifically, what packages will they be using) to actually make those decisions.

IMO, advising new users to start with pip is only a problem if we don’t have a gradual migration path to tools that address more advanced issues. And the struggle with conda is that it doesn’t have that gradual migration path - so it looks like there’s a bias against conda, when in actual fact all there is, is an acknowledgement that new users may not need something as powerful as conda yet.

I’m afraid I think this is an issue that the conda community really need to solve themselves - how to provide a more gradual migration path for existing pip/pipenv/poetry/etc users. Once that path exists, I think that documenting and promoting it would be very easy to integrate into packaging.python.org.