PEP 594, take 2: Removing dead batteries from the standard library

https://github.com/defnull/multipart is a proper replacement for cgi.FieldStorage, used by Zope, and thus by Plone, and so here to stay til the end of time :wink:

2 Likes

FYI since Python 3.1, 13 stdlib modules have been removed: Removed modules. I removed binhex and formatter after looking at PEP 594.

The asyncore module is also used in stdlib tests. The tests for ftplib , logging , smptd , smtplib , and ssl are partly based on asyncore . These tests must be updated to use asyncio or threading.

When I removed asyncore, asynchat and smtpd, I kept them as private modules in the test package. Technically, we can continue using them in tests, until someone steps in to rewrite tests with asyncio.

Recently, I had to debug a test_ftplib issue. It’s a pain to follow asyncore class inheritance and callbacks to understand where socket events come from, how they are handled, which callback is used (methods are overriden in sublcasses by design), etc. I really prefer asyncio coroutines!

Two nits from the current pep text: lib2to3 is listed as one to keep. That is already deprecated and removal is on track in its own bpo issue. Drop all mention of that or include it with plans matching the issue. (On mobile so I won’t search the # right now)

It also suggests jumping directly to DeprecationWarning without a PendingDeprecationWarning release cycle first. Why? Cover that. I’d suggest 3.11 be Pending for things without them already. 3.12 then makes those DeprecationWarning. For removal before our :pie: release if not in 3.13 itself.

2 Likes

https://bugs.python.org/issue40360

I suggest skipping PendingDeprecationWarning and going directly to DeprecationWarning.

Even with DeprecationWarning, it’s been a problem that warnings aren’t heeded or seen, and more than once removals have been bumped to a later release.

For example, using collections.Mapping instead of collections.abc.Mapping emitted a DeprecationWarning since Python 3.3 and was set to be removed in 3.9 but was bumped to 3.10 (and also to allow projects to drop 2.7 in concert).

https://docs.python.org/3/whatsnew/3.9.html#you-should-check-for-deprecationwarning-in-your-code

More recently, some unittest removals (deprecated since 3.5) and configparser (since 3.2) were reverted from 3.11 and deferred until 3.12.

https://discuss.python.org/t/experience-with-python-3-11-in-fedora/12911/7?u=hugovk

I also recommend mentioning in the warning which Python version removal is expected. This means people who see the warning have a visible timetable to work towards, rather than some vague point in the future.

PEP 387:

  1. Add a warning. If behavior is changing, the API may gain a new function or method to perform the new behavior; old usage should raise the warning. If an API is being removed, simply warn whenever it is entered. DeprecationWarning is the usual warning category to use, but PendingDeprecationWarning may be used in special cases where the old and new versions of the API will coexist for many releases [2]. Compiler warnings are also acceptable. The warning message should include the release the incompatibility is expected to become the default and a link to an issue that users can post feedback to.
2 Likes

Thanks for the reference. The Roundup Issue Tracker uses CGI and FieldStorage/MiniFieldStorage extensively as well as cgitb.
Sadly it does mean that Roundup will no longer work with only the standard Python library which has always been a goal.

Even though CGI is inefficient, the CGI deployment mode works fine for low volume trackers on cheap hosting environments.

Hopefully we can work around the depreciation and keep Roundup alive.

1 Like

Both cgi and cgitb appear to be single-file pure-Python modules; they would be easy enough to vendor into Roundup.

Yeah, I mentioned this on the ticket I opened to address this. One tricky part is that Roundup is multilingual and runs under both python2 and 3. I would need to vendor for python 3 only. But there is already a mechanism to handle issues like this.

Do you really need to support Python 2 still? If so, perhaps the solution would be to could just stick with Python 3.10 for the next decade. :slight_smile:

2 Likes

As you know, Roundup admins write Python code to implement workflows, integrations, custom interfaces, actions etc. There are libraries written in Python 2 that integrate with third party enterprise level tools.

Rewriting/modifying that code can be a major undertaking.
I assume that was part of the calculus for not upgrading B.P.O to Python 3. Moving to GitHub Issues was a better plan given the resources available.

So yeah I hope to stay multilingual for a few years yet. A different maintainer may make a different choice though.

Thanks for your interest though.

The docs for binhex are still there: binhex — Encode and decode binhex4 files — Python 3.11.0a0 documentation . I will remove it from the PEP, though, as the module itself is gone.

They seem to still be there, e.g., https://github.com/python/cpython/blob/main/Lib/asyncore.py and asyncore — Asynchronous socket handler — Python 3.11.0a5 documentation .

Removed!

That’s our deprecation policy: PEP 387 -- Backwards Compatibility Policy | Python.org .

But it’s documented elsewhere as this is how things are deprecated, so since this is following policy I don’t think it needs to be explicitly called out.

2 Likes

Ah yes, asyncore, asynchat and smtpd are back in Python 3.11, the SC asked me to add them back: https://github.com/python/steering-council/issues/86

I just tried to explain that it is possible to remove these modules from the stdlib by hiding them into the test package. See my quote of PEP 594 in my previous message.

1 Like

It’s nice to have a core library replacement for removed modules. I have found urllib.parse.parse_qs a satisfactory replacement for cgi.FieldStorage

I’m not sure if it’s worthwhile to/how to add this to the PEP?

You could create a pull request, similar to this one:

1 Like

I have submitted the PEP to the SC for consideration: PEP 594 -- Removing dead batteries from the standard library · Issue #109 · python/steering-council · GitHub .

12 Likes

I realise I’m very late to this discussion, but scanning through the PEP, there are a couple of deprecations I was (mildly) surprised to see:

cgi: Yes, launching a new process for every request is inefficient, but there are cases where it’s good enough. It’s conceptually much easier than writing an application with a proper web framework - it’s the web equivalent of scripting. And there are (or used to be?) cheap web hosts would give you CGI support as the only way to run custom code on the server, and certainly don’t make it easy to install something from PyPI.

Reading bits of the previous discussion and making educated guesses, I think it will still be possible to write CGI scripts using only the standard library even without the cgi module, but I’m not 100% sure. If that is right, maybe it’s worth spelling it out in the PEP?

telnetlib: Telnet is old hat, but still used at times for devices on a local network. I ran into it when playing with an old e-reader: you could enable a telnet server to get shell access. A quick look on PyPI turns up two different packages to control VLC over telnet, packages for controlling network devices and other things, and a package adding SOCKS proxy support. These are all using telnetlib and released in the last couple of years.

There’s probably no particular reason this needs to be in the standard library, and I see there are already several alternatives on PyPI (telnetlib3 appears to have the most attention of the ones I found). Maybe the PEP can explain this - the current detail gives no rationale for removing the module, and its brevity implies that telnet is self-evidently useless, which doesn’t seem right.

2 Likes

Agreed, as someone who maintains an RFC-compliant Telnet server in
Python (a MUD framework specifically), I’m mildly disappointed to
see this basic implementation being dropped from the stdlib. In my
project’s case it’s not going to imply significant impact at least,
since I’m only using it for automated testing and even then only
rely on telnetlib’s client implementation and a few of its constants
for sending raw option negotiations, but as protocols go it’s no
more venerable than, say, ftplib or poplib.

Since this is for minimizing maintenance on Python itself we would need to explicitly know how widely CGI as a hosting platform is before considering this in reverting that part of the PEP.

If you have some suggested wording I would happily take a PR for that to clarify things.

No one said some of us wouldn’t like to drop those modules as well. :wink: But telnetlib is not widely used and shouldn’t be used unless you know what you’re doing due to the security risk. And everyone is going to have differing opinions for all of this as to whether is more used or not. But the key point with both cgi and telnetlib is we didn’t get enough pushback to exclude them from the PEP and we wouldn’t accept them into the stdlib today (which is what I believe Christian used as a guideline and I agree with).

4 Likes

This PEP has already gone to the SC, so it’s considered done unless the SC asks for changes.

1 Like