Experience with Python 3.11 in Fedora

Of these problems, pyupgrade can automatically resolve removal of unittest aliases and open(..., "U"), accounting for around half the packages you list. Might be useful for packaging, and/or upstream PRs?

(and as an upstream maintainer, thank you! The Fedora team’s work on advance support is a significant contribution to the health of the Python ecosystem :heart_eyes:)

7 Likes

Teyit is another handy tool for upgrading unittest asserts (including things like assertTrue(x in y)assertIn(x, y)).

I checked 20 unittest bugs, and half of them were already fixed. :slight_smile:

Summary

6 were already fixed and released upstream:

4 were already fixed upstream but not yet released:

I created 8 upstream PRs:

1 PR was already created:

1 not fixed, upstream archived:

4 Likes

Thank you both for your suggestions. Maybe it would be useful to mention those programs in the official documentation as a way how to migrate to Python 3.11. Had I known about them, I would include them in Bugzilla reports.

@hugovk thanks a lot for doing this, it’s very helpful.

2 Likes

Indeed, clearly giving people a working recipe that can automatically apply required updates to their Python codebase to run on 3.next is something we need to do more of.

Longer term it’d be ideal if a modernize tool able to easily be told to target fixing up things required in order to run on a specific version were available on PyPI by the time the CPython beta releases roll around.

2 Likes

There is the pyupgrade project for Python code.

I wrote upgrade_pythoncapi.py script in the pythoncapi_compat project for C code (C extension modules).

4 Likes

Followup:

After discussion on python-dev, the top two here have been reverted and will remain in Python 3.11 (October 2022), and instead be removed in Python 3.12 (October 2023) to give more projects time to update.

  1. removal of unittest aliases (bpo-45162) - 61 packages

Reverted in bpo-45162: Revert "Remove many old deprecated unittest features" by gpshead · Pull Request #30935 · python/cpython · GitHub

  1. removals from configparser module (bpo-45173) - 28 packages

Reverted in bpo-45173: Keep configparser deprecations until Python 3.12 by hugovk · Pull Request #30952 · python/cpython · GitHub

5 Likes

The Python 3.12 branch is now open, here’s a PR to remove the configparser deprecations in 3.12:

I checked the Fedora packages listed above for configparser, and 15/28 (54%) upstreams have fixes (or are unmaintained/removed from Fedora).

2 Likes

Until I carefully read the PR, I thought that it was actually removing the deprecations themselves, rather than the deprecated objects…it might be a good idea to clarify that PR title, heh.

3 Likes

Old thread, popping up with py 3.12 again.

Maybe a general remark from a maintainer: Some instructions like “use B instead of A” would help the transitioning a lot, and they should be in the doc of A and pointed to by the release notes. The way it is we often have to find them somewhere in the doc for A if they are there at all, or in PRs, or … From that experience, the best approach (in terms of time spent) is to ignore deprecations (if they don’t come with a guide), wait for the fall-out and hope for fixes at that point in time.

In an ideal world, we would reaction to deprecations, not to removals :slight_smile:

1 Like

I think we do it for most cases.
Which parts didn’t have “use B instead of A” document?

1 Like

The imp removal. See How do I migrate from imp?

1 Like

Sorry for sounding like “generic bashing”.

Miro mentioned a current pain point.

I remember that when I looked at SafeConfigParser deprecation first I did not know what to do (maybe because “my” package tried to be keep py 2.7 compatibility, too) - but that one was clearer when I had to revisit it now.

imp module document has many “Use XXX instead”. So it is not good example of not having “Use XXX instead” document.

As far as I read the thread, it is more difficult problem. In many case, there are no direct 1:1 mapping from deprecated API to recommended API.

Deprecated APIs may be have bad design. So it is difficult to provide 1:1 mapped new API. We need to provide case-by-case support, like the thread.

2 Likes

We could formally document those recommendations, listed by imp function, and link to that from the deprecation notice for each. For example, we could add a section to the old imp docs, (perhaps better) make a new “How-to” like e.g. “How to migrate from imp to importlib”. Of course, someone (who knows a lot more about the details of imp that I do) would have to volunteer to help provide the recommendations for each…

I’d like a single page listing current deprecations, with migration advice.

Similar to:

https://docs.pytest.org/en/stable/deprecations.html

Which I copied for:

This would be useful for users, who have a single page to see what’s deprecated, the planned removal releases/dates (if any), and how to upgrade.

And also useful for maintainers, who have a single page to see what needs removing when the time comes.

2 Likes

I like that idea.

I want to include planned incompatible changes too. For example,

  • Invalid escape sequence will become SyntaxError in the future Python. (not scheduled).
  • UTF-8 mode become default (scheduled in Python 3.15).
2 Likes

I’d expect this information to be in the “Porting notes” section of the What’s New page, e.g. https://docs.python.org/3.12/whatsnew/3.12.html#porting-to-python-3-12

That relies on people remembering to contribute those docs. Anyone is welcome to contribute to them, so if you’ve found a gap in the 3.12 docs, just file an issue about it. (I will say, just from skimming the current 3.12 section, a lot of those are not porting notes but simply changes that occurred in 3.12. The whole page usually needs a big cleanup pass before release, so I guess we’re waiting on that.)

There is a section like that: What’s New in Python 3.12: Pending Removal in Future Versions. We can add a similar sub-section under the “Porting to Python 3.12” section.

I was thinking of a single page the reader can check, instead of needing to trawl back through half a dozen (or more!) historical What’s New pages which become out-of-date, and list things that have since been removed.

A central page would be updated to reflect the current status: have deprecations with no removal date been given a removal date yet, has the planned removal date been changed, has the deprecated code been removed.

1 Like

The theory is that you would review it as you upgrade to each new release, at which time it would be up to date and you wouldn’t have to trawl back through older documents. (Note that I don’t agree with this theory.)

A single page would be great. Someone has to maintain it, and if you want it to be versionless, it needs a home that is outside the CPython docs.

1 Like