Python 3.13 alpha 1 contains breaking changes, what's the plan?

That brings to mind TeX:

Since version 3, TeX has used an idiosyncratic version numbering system, where updates have been indicated by adding an extra digit at the end of the decimal, so that the version number asymptotically approaches π. This is a reflection of the fact that TeX is now very stable, and only minor updates are anticipated. The current version of TeX is 3.141592653; it was last updated in [11] The design was frozen after version 3.0, and no new feature or fundamental change will be added, so all newer versions will contain only bug fixes.[12] Even though Donald Knuth himself has suggested a few areas in which TeX could have been improved, he indicated that he firmly believes that having an unchanged system that will produce the same output now and in the future is more important than introducing new features. For this reason, he has stated that the “absolutely final change (to be made after my death)” will be to change the version number to π, at which point all remaining bugs will become features.[13] Likewise, versions of Metafont after 2.0 asymptotically approach e (currently at 2.7182818), and a similar change will be applied after Knuth’s death.[12]

5 Likes

With all the fun we’re having about versioning schemes, I think it’s worth going back to the substance of @scoder’s original post. We made a lot of C API changes early in the 3.13 cycle. Was it too much? Should we revert these changes and use a more gradual approach to cleaning the C API?

Personally I think Cython is central enough to the Python ecosystem that it’s worth fixing Cython first before making breaking changes to CPython, even if Cython arguably is doing things it shouldn’t be doing.

8 Likes

Note that we’re at the beginning of the 3.13 development cycle so there’s ample time to fix things. Though, of course, it prevents downstream users of Cython from testing their software with the Python development branch if they want to.

3 Likes

Cython 3.0.5 was released 1 week ago with:

Preliminary support for CPython 3.13a1 was added to allow early testing. (Github issue #5767)

3 Likes

I also have a vested interest in the “let Cython do what it wants with semi-private APIs” school of thought.

I think we realise that one of the consequences of using these internal APIs is they may get changed when it’s necessary to improve Python, and that’s fair enough - we can adapt in those cases. We certainly don’t want Cython to be a blocker for improving Python.

The issue with this particular lot of API changes was that they weren’t linked to actual improvements - they were went away with no obvious benefit.

I don’t think this should be used as justification for these changes - it took (mainly Stefan) a fairly big effort to replace everything that disappeared and large chunks of the replacements are probably worse than what was there before.

6 Likes

Yes it was too much. Multiple reverts are likely to happen (a couple have, but the list of changes is long). Tracked via discussions on [C API] Meta issue to replace removed functions with new clean public functions · Issue #111481 · python/cpython · GitHub & related https://github.com/python/cpython/issues/106320 & https://github.com/python/cpython/issues/111089 which are all tied to the broader theme of recent 3.13 C API changes.

The SC, C API WG, and RM are all aware.

2 Likes

I propose to revert Python 3.13 C API incompatible changes causing most troubles right now.

1 Like

Following the announced plan, I reverted 50 private APIs which were removed in Python 3.13 alpha1. These APIs will be available again in the incoming Python 3.13 alpha2 (scheduled next Tuesday).

I planned to make Cython, numpy and cffi compatible with Python 3.13 alpha1. Well, I missed this release. With reverted changes, numpy 1.26.2 can be built successfully, and cffi 1.16.0 just requires a single change. So we should be good (or almost good) for Python 3.13 alpha2.

It’s not the first time that incompatible C API changes affect many C extensions, have to be reverted, C extensions are updated, and then the changes are made again. This method is a practical way to move the C API forward, since it’s hard to check (see/guess) how the C API is used by 3rd party C extensions. More recently, code search was used on most popular PyPI projects. But maybe we need to design a new approach to let developers experiment scheduled incompatible C API changes, changes scheduled in 1, 2 or more Python releases, without affecting everything as soon as the change is made.

This approach based on a version is being discussed at Macro to hide deprecated functions. See also the discussion on my Add again <unistd.h> include in Python.h PR: some people asked to keep the incompatible change. I reverted it anyway, to do it again later with a smoother migration path.

I was overwhelmed last weeks by the quantity of discussions and issues related to Python 3.13 C API changes. Sometimes, the tone was not pleasant. I didn’t expect so many people testing early the Python 3.13 alpha1, and I didn’t expect people to report eagerly issues asking for changes. During the Python 3.12 devcycle, it took months to get Cython (7 months after the alpha1) and numpy (11 months) ready. It seems like for Python 3.13, it will be way quicker! For me, it’s a good sign that the Python community is active, dynamic and provides feedback as soon as possible. It’s great! :heart: Thank you all! :heart:

I’m sorry if some people felt that this C API work was forced on them and their opinion was not taken in account. We heard you and we took your feedback in account. It took me time to adjust my plan according to early received feedback. I expected to have 6 months to work step by step. Well, I had 2 weeks instead :slight_smile:

I don’t expect Python 3.13 alpha2 to be perfect. We may have to revert a few more private API removals. I expect that the number of build errors will be lower and so it will be easier to test Python 3.13 on your favorite projects.

24 Likes