Thank you @steve.dower. This is interesting feedback and in line with what I have seen as well. The fact that “pip” is the easiest thing for the new person to reach for means that all they think about is using pip for install. This invariably means that pip will be pressured to be a general-purpose packaging solution (it will slouch towards it based on what appears to be just following what users want).
The problem is that there is a space for a user-level, cross-language package manager like conda and there always will be. This channel is about drawing a line to the scope of Python packaging. Will pip be used to package and install Python itself? Will pip be used to package and install Java? Julia? I believe the answer should be no.
Then, if that is the case, because Python is used to “glue together” so many other languages there must be language at packaging.python.org that helps people understand that you should not expect ‘pip install’ to be the only way to install every Python package. Perhaps it can be used to install the ‘python-parts’ of the package but some of the things that must be installed for the solution to work should be installed by other package managers.
If we can agree on that framing (or something similar and better articulated), then we can have a conversation. Right now, what I see is that people are making “pip installable” things that make it much more difficult to actually provide a working and reproducible environment using tools that were built for that purpose. I’m not sure why people are doing that rather than build packages using tools that let them install them — other than the branding of packaging.python.org and its apparent message that everything should be “pip installable”
That won’t be able to provide what a user will expect until pip install can also install every other run-time that Python solutions glue together. For example, think about the pip install pyspark that happens right now. What does it do? Does it install Java (which is necessary for it to work)? It doesn’t as far as I can tell. Other Python packages are like this too and should be like this (they need previously installed things in order to work).
Is there a mechanism for pip to check for these previously installed things and raise an error or warning if they aren’t there? Perhaps that is a feature that could be added which would also implicitly help people understand the scope of “pip install”. All I’m suggesting is to do that plus a bit of modification to packaging.python.org in order to point to the efforts of other communities like NumFOCUS and conda-forge that are solving the general-purpose install problem.
Thanks for the feedback and help understanding other points of view. And just in case it’s not clear, I’m incredibly impressed and grateful for all the hard-work that goes into the open-source and community-centric solutions that you are all providing. Please just take my recommendations as a particular point of view from the trenches.
Thanks,
-Travis