Venv wheel package install (and sdist install w/o wheel)?

Packaging
#1

Can venv (be upgraded to) by default install the wheel package into a newly created venv? This would elegantly resolve an issue with installing an sdist into a venv on machines disconnected from internet (from PyPI).

The problem(s): I distribute a Python installer (e.g. miniconda .sh file) and my authored package as sdist to end-users who must install onto a disconnected machine (i.e. sneakernet and DVD). Upon installing sdist (/venv_path/python3 -m pip install --no-dependencies my_sdist.tar.gz), I get error: false requirements already satisfied. This is due to pip recently insisting upon building/installing a wheel from the sdist, and the wheel package is absent from the venv.

A workaround: First do: /venv_path/python3 -m pip install --upgrade wheel --no-index --find-links local_path_to/wheel-0.36.2.tar.gz, i.e. I provide the wheel package locally (with my distribution to end-user). The --no-index and --find-links are necessary since , since pip otherwise attempts reaching out over internet to PyPI for the wheel package.

Question(s):

  1. Is this the most elegant workaround short of my suggested venv upgrade?

  2. Just how did pip solve the chicken/egg problem with installing the wheel package into the venv, since it evidently first wants to build a wheel from the wheel package sdist??? My logfile says: Using legacy 'setup.py install' for wheel, since package 'wheel' is not installed. Why doesn’t it behave this way for my end-user sdist install???

#2
  1. Probably, if it’s necessary for you to distribute your package as a sdist rather than a built wheel.
  2. Pip doesn’t need to install wheel from its sdist, because there’s a wheel of the wheel package available. So we just use that (which can be installed via a simple unzip). I don’t know why you’re providing the wheel package as a sdist rather than a wheel, by the way - you might want to switch to providing the .whl.
#3

IIRC, the answer was no the the last time this was brought up, because we want people to switch to PEP 517 instead. This is as easy as adding a new pyproject.toml file with a couple of configurations to your my_sdist.tar.gz unless it’s a crazy complex package like numpy (in which case you really should distribute a wheel instead).

#4

Thanks, Paul!

Just to understand your point #2a. Where/how is the ‘wheel of the wheel package’ available to pip, if machine is disconnected? Is it in the ‘mother’ Python installation from which venv draws?

Point #2b. I’m providing wheel sdist, not having thought of your option of .whl. But I’m supporting multiplatform (MSWindows too) so sdist does seem a bit more generic. What are reasons to prefer the .whl?

Thanks again.

#5

Thanks, Tzu-ping.

Thankfully my package is pure-Python. From determining my workaround, I came upon the PEP 517/518 ferment (I’m used to the pre-517/518 setup.py approach to packaging, which was the guidance at https://packaging.python.org/tutorials/packaging-projects/ a year ago). If I update my procedures as you suggest, I have 2 questions:

  1. For machines disconnected from internet, what if anything do I need to do after creating my venv (using venv, I presume the recommended method) to insure my project builds/installs into venv?

  2. I presume to get the PEP 517 build/install behavior from pip, I need to be providing end-users a sufficiently new Python distribution. What minimal Python/pip versions are necessary?

Thanks!

#6

Wait, in the disconnected case pip didn’t solve the chicken/egg problem, what made you think it did?

No, if it’s a universal wheel (i.e., it doesn’t contain compiled binaries) the wheel is more generic than the sdist, because you don’t need anything to install a wheel (pip can do it without any additional tools) whereas you need a build backend to install a sdist.

In which case you should just build a wheel and ship that. Your users won’t need anything other than your wheel and pip to do the install.

#7

Nice advice on points 2 and 3, thanks!!

Re: chix/egg - right, I’m mystified - why didn’t it attempt to build a wheel out of the .gz (and promptly fail, of course), yet pip did process the .gz and install wheel. Any ideas to what actually happened? Actually, I guess it took the ‘legacy path’ as you describe here: Raise a warning when pip falls back to the “legacy” direct install code #8102. The #8102 thread timeframe is same/later than my Python/pip distro age, so my observed behavior reflects state of pip back then.

Thanks again.

#8

Not really, because you didn’t give enough detail, TBH. I didn’t even realise you were talking about your install in the disconnected environment at first. I assume it’s not that relevant, though, now that we’ve established that you should be providing wheels, not sdists.