Remove distutils bdist_wininst command

Hi,

I deprecated distutils bdist_wininst command in Python 3.8:

Pillow was one of the last popupar project publishing Windows installer files. Following this deprecation, Pillow stopped to publish Windows installers (.exe), since Pillow 6.2.0 (October 2019).

I now propose to remove the bdist_wininst command to ease Python maintenance:

Is there anyone who strongly want to keep this command? If yes, speak out and give your rationale. Otherwise, I plan to merge my PR at the end of the week.

2 Likes

I guess I don’t care about it too much, but we usually redirect almost all distutils bugs to setuptools these days, and fix them there instead. We have a project to remove distutils entirely from CPython. I think it’s better to leave the deprecations and removals up to setuptools, and just close bugs in distutils as wontfix.

Already the deprecation of bdist_wininst in distutils has caused problems for setuptools, because we have some tests that hit that pathway and it’s difficult to interpret deprecation warnings as errors while ignoring that specific path in that situation. I suspect that similarly trying to push these deprecations on the standard library side instead of the setuptools side will likely cause more problems for the maintenance of setuptools than it saves from the maintenance of CPython.

I recall that Tarek Ziadé had a great project for distutils and made tons of changes. During this work, it was no longer alloweds to touch distutils. Two years later, this project was abandoned and most of the work was lost. At the end, distutils was in the same sad state and no bug was fixed for 2 years.

If you want to remove distutils from Python: go ahead. But I expect practical issues. For example, Python uses distutils to build its standard library (C extensions). This issue should be fixed first. I expect many other small corner cases that should be fixed first.

I don’t care much about distutils, but I do care of the Python Maintenance. Removing bdist_wininst code should reduce the overall maintenance.

I’m suggesting that this is not true. distutils is fundamentally broken in many ways without setuptools. You have not identified anything in the BPO issue that is making Python maintenance any harder other than, “People shouldn’t use this, so we shouldn’t continue maintaining it.” But CPython is not maintaining it, and in general distutils only gets intermittent updates for things that are more or less critical problems.

I’m saying that by removing this particular module, you’re saving no one’s time on the Python module and taking time and energy away from maintaining setuptools (which has to cope with this change).

I think the right thing to do here would be to propose removing this in setuptools, and drop the idea of removing it in distutils. The same goes for bdist_rpm and all the other weird non-wheel binary distributions.

1 Like

I’m happy for bdist_wininst to be removed, but FYI it looks like pycurl may still be distributing wininst installers. See https://pypi.org/project/pycurl/#files and http://pycurl.io/docs/latest/install.html (“If you are not using pip, EXE and MSI installers are available in the download area.”

The PEP 527 has been accepted, so it should no longer be accepted, but the PEP is not fully implemented yet:

I thought we’d already had this discussion?

If we’re allowed to wontfix issues here, then I can stop reporting all the false positives when virus scanners claim that CPython is infected because of the wininst executable stubs? And not update it to detect changes to registry keys when finding Python installs?

I’ve already removed it from a few alternative distributions and nobody has noticed. Happy for it to go completely - it’s not difficult to find another project to generate better installers, such as pynsist or briefcase.

Can you point me at one such thing? It sounds like the answer is “yes” because I don’t really know that conforming to the dictates of whatever nonsense heuristics virus scanners use to detect “infections” is part of our remit, but I’m also not clear as to whether you’re saying that this is an issue with projects generated using bdist_wininst or an issue with something about bdist_wininst itself. I think we can wontfix any bug that is fixed by not using bdist_wininst.

In any case, to be clear, I am not saying we shouldn’t remove bdist_wininst, I’m saying that we should remove it from setuptools, and either not touch it in distutils at all (or at least not until a decent interval of time after it’s been removed from setuptools.

As it stands now, it was deprecated in 3.8 but not in setuptools, which means the deprecation cycle for setuptools hasn’t even started.

I suppose we could just import it wholesale into setuptools and then go through the deprecation cycle there, but like I said, that creates extra needless work that could be solved by just having started this process in setuptools and not distutils in the first place (the project that the vast majority of people actually use for their builds).

We never figured out which file was the problem in https://bugs.python.org/issue30944 or https://bugs.python.org/issue33220 but it was definitely the wininst stub in https://bugs.python.org/issue27383 (and some that were dealt with via the security list, IIRC). In general, the problem is actual malware that uses bdist_wininst for distribution, but the signatures end up being based on the stub rather than the data.

There was also https://bugs.python.org/issue37481 where I posted my support. I don’t think it was linked earlier in this thread. Deprecation in 3.8 may be too recent for hard removal in 3.9, but I’d want to see some evidence of it being in deliberate use (rather than “we’ve always done it that way”). Otherwise just get rid of it.

It seems like removing the code breaks setuptools. Maybe we can replace the code with a mock which quacks like wininst but fails as soon as you start to use it. For example, setuptools only uses wininst to check if a file looks like a Windows .exe installer. The mock just has to reply “no” and we’re done. Only “bdist_wininst” would be broken. I may be a smoother transition.

If we replace wininst implementation with a “mock” in the stdlib, setuptools would not be impacted: it would continue to work (again, until you actually use the real bdist_wininst command).

This smart idea comes from @steve.dower: https://github.com/python/cpython/pull/18329#issuecomment-582169262

1 Like

I maintain pycurl which has a C extension and I use the exe/msi artifacts for testing pycurl, as follows:

Install all supported pythons (3.5-3.8 in 32/64 bits, 8 total)
Build all pycurl packages
Pick a package (say 3.7/64)
Execute this package's installer, either exe or msi
The installer locates where the respective python version is installed and offers to install itself there, systemwide.
I can now run the respective python binary and test that the package is functional by executing import pycurl and using it.
I uninstall the package using the windows add/remove programs functionality.

The workflow described has the following two particular aspects:

It installs pycurl globally into my chosen python interpreter.
It provides a way to uninstall pycurl completely from that interpreter.

Was a workflow like this considered, and if yes what is the recommended replacement?

@oleg It should be possible to replace the workflow with wheels (built with setup.py bdist_wheel) for distribution, and pip for managing the installs.

installs pycurl globally into my chosen python interpreter.

<chosen-python> -m pip install pycurl-*.whl

uninstall pycurl completely from that interpreter.

<chosen-python> -m pip uninstall pycurl

I don’t plan to remove bdist_msi, even if wheel packages are now recommended.

We should, though. Installing a package using an MSI is worse than an EXE, as it leaves far more cruft behind if you don’t uninstall it before changing/removing the Python install.

Standalone apps should bundle everything, like pynsist or briefcase. GPO deployment should create their own MSI with everything they want in the bundle and deploy that. Perhaps someone can make an “installer” based on the py.exe launcher (which I believe supports an attached zip file) that will use pip and a local/embedded wheel.

But we should really discourage package installs that don’t support venv and/or leave cruft behind.

If you want to see it disappear, you should start by deprecating it in Python 3.9. It would be a first step.

What does the “attached zip file” part mean here? I can understand making an installer that runs py -m pip path-to-embedded-wheel, but don’t quite understated how attaching a zip file works in this context.

You can append a zip file to the py.exe binary and the combined binary py myapp.zip.

I believe you need a compile time option that’s not enabled by default in the py.exe shipped with Python, though.

Ah I see, now I remember Steve talking about this sometime before. Thanks!

Please see bpo-39586 to deprecate bdist_msi:

Once again I would suggest starting this process in setuptools, which should backport the notice to users of all Python, and will make it easier for setuptools to maintain.