CRuby has been providing default gems for a while, which is a set of pre-installed modules that can be uninstalled and also available on RubyGems, like how ensurepip currently works. Does anyone happen to know how Linux distributions are treating this?
1 Like
The cost of having these modules in the standard library is not only cost of actual changes in these modules themselves. I was supporting keeping nntplib module in the standard library, but when it broke for the fifth time CI while I was working on completely different change, I am beginning to change my opinion. Perhaps it would be really better if nntplib went to separate PyPI module and somebody else had to bother with its constant breakages.
3 Likes
Linux distros provide additional modules as packages that are installable in the same way as CPython itself. Technically, it would be possible to mark them as required by Python; some distros could mark them as recommended (installed by default, but uninstallable).
The issue in Debian is (AFAIK â Iâm not a Debian expert) not technical: the idea there is that to get a tool that can install software from outside Debian, users must take an explicit action (install pip rather than just Python).
Some other reasons distros exclude parts of the stdlib are:
- Dependencies (
tkinter, IDLE,turtleetc. need GUI libraries, which donât belong on servers) - Size (
testis huge, and not neded in production)
The breakage was due to a change in the configuration or setup of an
external news server which the tests were relying on.
Such breakage doesnât really have to do with the quality or usefulness
of the code in question, but rather with the stability of the external
resource youâre relying on and the assumptions made in the tests about
these.
We have, for similar reasons, moved a lot of external resources we need
in the test suite under PSF control by hosting pythontest.net for
this purpose, e.g. for testing ssl and some of the codecs.
In the case of nntplib, we should see whether we can get a
news.pythontest.net news server setup to test against. This would then
resolve the issues with external servers making changes to their setup
or not being available anymore or temporarily. Alternatively, we could
disable the live tests and have the mock tests run instead for CI.
Who is going to pay for setup, maintenance, and hosting of an NNTP server? Since it has to be a publicly accessible service, the service must be secured, monitored, and updated regularly. Thatâs a non-trivial task. I can think of better ways to use the limited financial capabilities of the PSF and limited time of @EWDurbin.
1 Like
See Show devel:languages:python:Factory / python39 - openSUSE Build Service for openSUSE: we have three packages:
- âpython-baseâ which is the basic interpreter plus modules from stdlib which donât require any external dependencies (except for OpenSSL, we had to compromise on that).
- âpythonâ with all its sub-packages for everything else (e.g., tkinter, dbm, curses, xml)
- âpython-docâ for documentation (because of the dependency on Sphinx)
a) I donât think running tests against the real NNTP server makes sense, it should all be mocked anwyay,
b) if hosting NNTP server is requirement for nntplib (and I donât think it is), then certainly that should be the reason to deprecate nntlib from the standard library.
2 Likes
Iâll help Ernest with this, no worries.
Who is Ernest? There is nobody named Ernest in our team.
Thank you for noting this Christian, Marc-AndrĂ© likely missed that my name has changed to Ee, since it wasnât really distributed in a way that would specifically target the core devs as an audience <3.
2 Likes
Indeed, I missed that. Happy New Year, Ee 
Just chiming in to say please donât remove urllib.request.urlretrieve. Itâs a nice batteries included for a quick file download with progress reporting. Iâm mentioning because Iâve used it a lot recently in little one liners and I get nervous everytime I got to the docs and see the note that it âmight become deprecated at some point in the futureâ.
I realize itâs not on the docket right now but saying this now hopefully may influence the next round of battery removal should it come up.
1 Like
At this point, my suggestion to Christian is to decide on which proposal he wants to bring forward to the SC as the PEP author in terms of what to do with the deprecated, update the PEP, and send it to the SC to help make sure this gets into Python 3.10 and give folks extra time to potentially come up with their own PyPI release of things (if that isnât part of the PEP).
3 Likes
Right. I worry that perfection is the enemy of the good in this case. Since weâre not actually removing anything from 3.10 (except for the parser module, which is already gone, for separate reasons) we could proceed with the current list. If there are serious objections we could have a change of heart in 3.11, or delay certain deprecations by 1-2 release cycles. But right now the inaction is killing me!
6 Likes
One of the big pains of Python is distributing software that somehow depend on Python, either because it is written in Python or have some other interfaces to Python. Iâm a maintainer of such a software.
We have recently stumbled upon problems with our Python builds, which rely on âlibnsl.so.1â for the ânisâ extension. This library is maintained together with glibc, but is no longer installed by default on new Fedora, Centos or RHEL systems (Centos/RHEL >= 8 do not install this by default).
We have no interest in NIS/libnsl, yet it is a requirement from Python. There are no easy/documented way I can build CPython without it, to achieve this I have to âhackâ the Python build system, which is not straightforward for every release.
Most of CPython dependencies have a permissive license which allows me to statically link quite a lot into the Python core, but this not the case for libnsl (if it is possible at all to create a static libnsl - I havenât checked).
libnsl seems to be struggling to die for decades, so maybe it would be possible to let it rest now. I would really see this PEP going through in some form or another. Thanks for the great programing language!
libnsl is not required to build Python. Python builds and works fine without NIS support. The build system will show a warning, but it wonât fail.
1 Like
Yes, I know. But when your build system happen to have the library and headers, because it is Centos 7, Debian Jessie or some other old system that install these as part of the bare minimal OS, there are no way to tell python âdo not build nisâ. It will be built and there will be a libnsl.so dependency. Always.
There are no --disable-nis, --disableânismodule. Then you need to hack the build system. Please correct me if I have overlooked something.
Iâd like to add our point of view as a third party software vendor that are currently reliant on asyncore.
I can fully understand the need to reduce the maintenance burden, so we are fully on board with deprecating things over time. We do have some concerns over the time frame though.
To keep things sane for our maintenance, weâd like to have the same code base for all our users. That means we need to restrict ourselves to things that are available from the oldest stuff users might have, to the newest. In order to keep this relevant we use the life cycles of the enterprise distributions as a reference.
Right now the oldest still supported enterprise distribution is SUSE Linux Enterprise 12. That includes Python 3.4. So we can only use things that were available in that version. asyncio was not, so we cannot currently port things from asyncore to asyncio.
So weâd like you to consider moving the deprecation date of asyncore until a time where asyncio is available in every supported enterprise distribution. That would mean 2024, or Python 3.14/3.15.
Itâs only asyncore that is of concern for us, but the same reasoning could be relevant for other modules as well.
Deprecation is not the same as removal, youâre welcome to keep using it even as it is marked deprecated. Just be aware that it will likely bit-rot, and if you really need it to keep working, you may need to implement workarounds.
Alternatively, feel free to just copy the module into your own application and vendor it. The license allows that, and it means you can just stop worrying about whether itâs available or not.
2 Likes
Why does asyncore have to stay in the stdlib? You could ask your vendor to package and maintain asyncore as a separate package. After all you are paying a vendor for an enterprise experience.
There is no --disable-nis configure option. You can easily work around the problem. You could package nismodule.so in a separate subpackage or rm %{buildroot}/path/to/nismodule.so in the %install block of your spec file.