PEP 594: Removing dead batteries from the standard library

CRuby has been providing default gems for a while, which is a set of pre-installed modules that can be uninstalled and also available on RubyGems, like how ensurepip currently works. Does anyone happen to know how Linux distributions are treating this?

1 Like

The cost of having these modules in the standard library is not only cost of actual changes in these modules themselves. I was supporting keeping nntplib module in the standard library, but when it broke for the fifth time CI while I was working on completely different change, I am beginning to change my opinion. Perhaps it would be really better if nntplib went to separate PyPI module and somebody else had to bother with its constant breakages.

2 Likes

Linux distros provide additional modules as packages that are installable in the same way as CPython itself. Technically, it would be possible to mark them as required by Python; some distros could mark them as recommended (installed by default, but uninstallable).
The issue in Debian is (AFAIK – I’m not a Debian expert) not technical: the idea there is that to get a tool that can install software from outside Debian, users must take an explicit action (install pip rather than just Python).
Some other reasons distros exclude parts of the stdlib are:

  • Dependencies (tkinter, IDLE, turtle etc. need GUI libraries, which don’t belong on servers)
  • Size (test is huge, and not neded in production)

The breakage was due to a change in the configuration or setup of an
external news server which the tests were relying on.

Such breakage doesn’t really have to do with the quality or usefulness
of the code in question, but rather with the stability of the external
resource you’re relying on and the assumptions made in the tests about
these.

We have, for similar reasons, moved a lot of external resources we need
in the test suite under PSF control by hosting pythontest.net for
this purpose, e.g. for testing ssl and some of the codecs.

In the case of nntplib, we should see whether we can get a
news.pythontest.net news server setup to test against. This would then
resolve the issues with external servers making changes to their setup
or not being available anymore or temporarily. Alternatively, we could
disable the live tests and have the mock tests run instead for CI.

Who is going to pay for setup, maintenance, and hosting of an NNTP server? Since it has to be a publicly accessible service, the service must be secured, monitored, and updated regularly. That’s a non-trivial task. I can think of better ways to use the limited financial capabilities of the PSF and limited time of @EWDurbin.

See Show devel:languages:python:Factory / python39 - openSUSE Build Service for openSUSE: we have three packages:

  • “python-base” which is the basic interpreter plus modules from stdlib which don’t require any external dependencies (except for OpenSSL, we had to compromise on that).
  • “python” with all its sub-packages for everything else (e.g., tkinter, dbm, curses, xml)
  • “python-doc” for documentation (because of the dependency on Sphinx)

a) I don’t think running tests against the real NNTP server makes sense, it should all be mocked anwyay,
b) if hosting NNTP server is requirement for nntplib (and I don’t think it is), then certainly that should be the reason to deprecate nntlib from the standard library.

2 Likes

I’ll help Ernest with this, no worries.

Who is Ernest? There is nobody named Ernest in our team.

Thank you for noting this Christian, Marc-André likely missed that my name has changed to Ee, since it wasn’t really distributed in a way that would specifically target the core devs as an audience <3.

2 Likes

Indeed, I missed that. Happy New Year, Ee :slight_smile:

Just chiming in to say please don’t remove urllib.request.urlretrieve. It’s a nice batteries included for a quick file download with progress reporting. I’m mentioning because I’ve used it a lot recently in little one liners and I get nervous everytime I got to the docs and see the note that it “might become deprecated at some point in the future”.

I realize it’s not on the docket right now but saying this now hopefully may influence the next round of battery removal should it come up.

1 Like

At this point, my suggestion to Christian is to decide on which proposal he wants to bring forward to the SC as the PEP author in terms of what to do with the deprecated, update the PEP, and send it to the SC to help make sure this gets into Python 3.10 and give folks extra time to potentially come up with their own PyPI release of things (if that isn’t part of the PEP).

2 Likes