Extending bugfix releases schedules

This came up in a chat between release managers, and I think I feel strongly enough that we should change this.

Currently, our release schedule is such that a release gets bugfix releases for 18 months, which means 6 months after the next major release. If you look at the adoption rates of the last couple of new versions (for example @julien’s mdk/python-versions: Studying Python release adoptions by looking at PyPI downloads - python-versions - Le Gitea de l'AFPy) there is really not a lot of adoption in those 18 months. Switching to yearly releases doesn’t seem to have improved this much either (in fact, just looking at the graphs it seems to have made things worse, but there are a lot of other factors at play). That means that a lot of people, when they upgrade Python versions, they’re upgrading to a version that already doesn’t receive bugfixes anymore. It also means that if they run into bugs, it’ll be too late to fix them in that release.

I’m not sure we can significantly improve the adoption, given that we’re limited by secondary distributors and necessary compatibility fixes in common dependencies and such. I think we can do better, but I don’t think it’ll give us, say, twice the adoption at 18 months.

And I don’t think that means all the bugs will already be fixed for them. At Google we’re currently on 3.10, and it’s a lot of effort to upgrade. We used to be behind by a couple of major releases. Even so, we have always discovered bugs in Python during the upgrades. (One of the reasons we want to be closer to the latest Python version is to help fix those kinds of bugs earlier.)

Given that, I would like to extend the bugfix period for releases by at least 6 months. That would mean we stop backporting to 3.12 when 3.14.0 (final) is released, instead of around when the last 3.14 alpha is released. I wouldn’t mind extending it more, but I’m not sure what the impact would be on core dev workload and I’m not as certain that it would be worth the effort. (Whether the 3.11 schedule should be changed is up to @pablogsal.)

What do people think? Is it a reasonable support burden to take on? Is it too much work for the binary release managers (@nad and @steve.dower)? Do people really not want to bother with older versions that long after their release?

13 Likes

FWIW, as as Python user, contributor and now newly-minted core developer. I’ve always felt it would make a lot of sense to extend the bugfix window to at least the 2-year mark, i.e. supporting the last n + 1 releases rather than n + 0.5. Only supporting the previous version for 6 months with bugfixes and installers after the new feature release always felt really short to me, even as a mainly Conda user and part of the scientific Python community that has some of the fastest Python version update and deprecation cycles of any large user group.

With the currently yearly cadence, an integer-year bugfix support period would sync up much better from a core-workflow perspective and would make a number of things cleaner, more consistent and easier to reason about and communicate.

In terms of workload impact, my main focus has been documentation, of which pretty most to nearly all PRs that don’t apply just to the latest branch are backported as a matter of standard practice (unless people forget, which is not uncommon unfortunately). While it would increase workload a bit, its mostly handled by the automation, and where its non-trivial to backport to an older branch (which typically happens either because there was another intersecting change, or something wasn’t backported that should have been), it can just be skipped if necessary anyway. Overall, I would welcome it, personally. But I can’t speak as much to the impact on code-related PRs, though.

Overall, ISTM that the lion’s share of the additional individual work (besides Ned and Steve generating binaries) is on the RM, which is one reason I certainly wasn’t going to be one to suggest it myself, especially as a new core dev. But as you are the RM that it would impact, and you’re the one proposing it and willing to do the work, FWIW you have my full support (provided Steven and Ned don’t object) and I would be willing to pitch in further to help backport docs and infra PRs as necessary.

1 Like

I am in favour of extending bugfix period for the reasons mentioned. But -

I don’t think this is true. The largest burden is backporting fixes to components that have changed a lot between the versions (in the last few years internals have indeed changed a lot). We may need to be a bit more liberal about deciding that backporting something is not going to be worth the risk, given that we would be fiddling with a codebase that we haven’t worked on for a couple of years and we may not be that familiar with it anymore.

So I am in favour of extending the ability to backport fixes (without an implied promise that we would always do that).

6 Likes

I support. I think that switching to yearly releases had a negative effect of reducing the support period and slowing the adoption rate. Linux distributives need several months to test new Python version, so they can only include Python version released several months before their release. And many users wait for the first or second bugfix release to upgrade the OS. So it takes almost a year before new Python version became widespread.

Other problem, is that Ubuntu and some other distributives have a two-year release rate for LTS releases. It means, that many users skip at least one Python version. They can completely skip a deprecation period and jump directly in a breaking change. We still

There is also a problem of increasing gap between CPython and PyPy. The latest PyPy version is 3.9, but CPython 3.9 no longer accept bugfixes. We cannot fix bugs in the stdlib which are easier to reproduce in PyPy.

I think that the bugfix period for releases should be more than 2 years. At least 30 months, or maybe even 36 months.

Also, I propose that when we make several releases in a row, do it from the newer version to the older version, with a delay of one or two weeks. For example, first 3.12.0, then 3.11.5, then 3.10.13. The reason is that new bugs can be introduced in bugfix releases (it already happened in past). People which are eager to install a new version just after release are ready to take some risks. They can find bugs which were also backported to older versions which are intended for more conservative users. If we had a time to fix a bug in bugfixes, bugfix releases will be more bug free, and conservative users will adopt them faster.

6 Likes

If we extend the bugfix window by 6 months, would that reduce security-only window by 6 months (as bugfix also includes security fixes), or retain its current size?

That is, from the current:

  • 1½ year bugfix + 3½ year security-only = 5 year total support

To either of:

  • 2 year bugfix + 3 year security-only = 5 year total support
  • 2 year bugfix + 3½ year security-only = 5½ year total support

This is what I assumed and would certainly make the most sense to me, FWIW. As far as I understand, @thomas 's points above (as well as most of the others people raised) relate specifically to extending the bugfix window, not also the security-only window. And this retains the longtime status quo in terms of total support that doesn’t seem to be as much of a problem (potential or actual) as the relatively tight bugfix window, as well as keeps everything nice whole numbers which are easier to conceptualize, remember and plan for.

1 Like

I’m in favor of extending the bugfixes window an additional six months.

Regardless I do think it is worth pointing out that so many people only ever use their distro version of Python and notable big distros that most of use daily tend to not bother shipping our upstream Python bugfixes. Some apply patches to their stable releases piecemeal based on their own desires. So many users may never see them no matter what we do so long as the most convenient permitted path for them is distro Python rather than upstream Python.

But it is explicitly not our problem what distributors do. So let’s do the right thing regardless and add six months.

3 Likes

I mean, my preferred change is still to adopt my alternative proposal to the 1-year release schedule from back when we were deciding that :smiley: But I guess we’re not yet concerned enough about lack of adoption to go there.

Adding six months of maintenance and reducing the security-only period is fine to me.

The one idea that I don’t like is Serhiy’s suggestion to stagger the simultaneous releases. From the POV of discovering issues, it definitely makes a lot of sense (though I suspect releasing the older versions first would find more). However, I feel it would seriously overcommit the release team to being available simultaneously on too many days to do releases.

2 Likes

I am definitely in favor of extending the bugfix period, although I would prefer 3 years of bugfix and 2 years of security only.

A bit of an extended aside, but FWIW, as far as I’m aware this only really applies to many/most (though certainly not all) big non-rolling-release-cycle Linux distros, not most distributions for Windows, macOS or cross-platform. I can confirm that Homebrew, MacPorts, Nix and Anaconda/Conda-Forge (Anaconda/Miniconda/Maxiconda/Miniforge/Mambaforge), as well as of course the first-party Python.org and Windows store installers (until the end of bugfix support) are all up to date or nearly so with the latest point version of each feature release. And while all those mentioned carry the latest feature release (3.11), many don’t ship it by default (in distributions where there is one), which speaks to @thomas 's point.

This covers pretty much all the major non-embedded distribution channels for Windows and macOS aside from the Python Apple ships as an optional developer tools install (not really sure how that version is managed). Combined with rolling-release distros like Arch, installs from tools like like pyenv, pre-commit and tox/nox (which use the upstream releases directly), alternative Linux distribution mechanisms like deadsnakes, and running on CI (using setup-python and similar), all of which provide and generally default to the latest or nearly the latest point version on any given feature branch (but don’t always offer the latest feature branch, it seems fairly likely to me that a majority of direct/non-embedded-Python users are using it from a distribution that does regularly ship upstream bugfix versions.

4 Likes

That’s fine with me, too. Steve has largely automated the Windows manufacturing process and we’re getting close with the macOS one, as well.

1 Like

I think that’s good as that sends a signal that you can pick a Python version and then update on a 2 year cadence and then always have a bugfix version that you’re relying on (assuming the community updates fast enough for you to switch Python versions shortly after they are released). Otherwise we start looking at 2.5 years or longer to give space for the community to catch up.

That might slide into a discussion about doing LTS releases if we want to push out to something like 3 years and only do that for even-numbers minor versions. That would let people skip every other release and have a whole year to migrate to the new version they choose to target.