[OBSOLETE] PEP 596: Python 3.9 Release Schedule (doubling the release cadence)

NOTE: this discussion is a valuable historical artifact but it does not represent the end result. For the actual 3.9 release schedule, see: PEP 596 – Python 3.9 Release Schedule | peps.python.org

Abstract

This document describes the development and release schedule for Python 3.9. The schedule primarily concerns itself with PEP-sized items.

Release Manager and Crew

  • 3.9 Release Manager: Łukasz Langa
  • Windows installers: Steve Dower
  • Mac installers: Ned Deily
  • Documentation: Julien Palard

3.9 Lifespan

3.9 will receive bugfix updates approximately every month for approximately 12 months. After the release of 3.10.0 final, a final 3.9 bugfix update will be released. After that, it is expected that security updates (source only) will be released until 5 years after the release of 3.9 final, so until approximately June 2025.

Release Schedule

3.9.0 schedule

Note: the dates below are proposing a 9-month release cadence. Rationale is described in sections below. This is not set in stone yet. The PEP is in draft status. Suggestions and comments welcome.

Expected:

  • 3.9 development begins: Tuesday, 2019-06-04
  • 3.9.0 alpha 1: Friday, 2019-09-13
  • 3.9.0 alpha 2: Monday, 2019-10-14
  • 3.9.0 alpha 3: Monday, 2019-11-18
  • 3.9.0 alpha 4: Monday, 2019-12-16
  • 3.9.0 alpha 5: Tuesday, 2020-01-20
  • 3.9.0 beta 1: Monday, 2020-02-17 (No new features beyond this point.)
  • 3.9.0 beta 2: Monday, 2020-03-23
  • 3.9.0 beta 3: Monday, 2020-04-20
  • 3.9.0 candidate 1: Monday, 2020-05-18
  • 3.9.0 candidate 2: Monday, 2020-05-25 (if necessary)
  • 3.9.0 final: Monday, 2020-06-08

Subsequent bugfix releases at a monthly cadence.

Features for 3.9

Nothing yet, go write a PEP!

Moving to a nine month release calendar

Rationale and Goals

This PEP is proposing a release calendar that doubles the release cadence. Python 3.9 will be developed for 12 months, 7 of which are spent on feature development (pre-beta), while the remaining 5 are spent on hardening the release (beta and RC). Due to feature development starting as soon as Python 3.8 Beta 1 was released, this creates a nine month delta between major Python releases.

This change provides the following advantages:

  • makes releases smaller: doubling the cadence doesn’t double our available development resources, meaning that consecutive releases are going to be smaller in terms of features
  • puts features and bug fixes in hands of users sooner
  • creates a more gradual upgrade path for users, by decreasing the surface of change in any single release
  • decreases the urge to rush features shortly before “Beta 1” due to the risk of them “slipping for 18 months”
  • shortens the beta phase, which also shortens the calendar overlap of “hardening” the currently developed (still unreleased) release vs. “feature development” on the next release
  • increases the explicit alpha release phase, which provides meaningful snapshots of progress on new features
  • significantly cuts the implicit “alpha 0” release phase which provides limited use for new development anyway (it mostly overlaps with the beta of the currently developed, still unreleased, version)

Non-risks

This change does not shorten the security updates calendar for a Python release.

This change does not accelerate the velocity of development. Python is not going to become incompatible faster or accrue new features faster. It’s just that features are going to be released more gradually as they are developed.

Consequently, while this change introduces the ability for users to upgrade twice as fast, it does not require them to do so. Say, if they upgrade every second release, their experience with Python is going to be very similar to the current situation.

Risks

This requires changes to how integrators, like Linux distributions, release Python within their systems.

This eventually doubles the testing matrix for library and application maintainers that want to support all actively supported Python versions.

The following policies depend on the release cadence and will have to be updated:

  • the deprecation policy
  • the__future__ import becoming the default
  • the term of the Steering Council
  • the term of the Release Manager

Finally, this shortens the bugfix period to twelve months. This allows core developers to increase the cadence without increasing the biggest maintenance cost, however provides an inconvenience to the user. This particular risk might be solved by making every n-th release a LTS, in the vein of Python 2.7. This is open for debate.

Rejected Ideas

Keep the current 18 month release cadence. From the perspective of the core developer:

  • it creates a surge of rushed commits before (and even after!) Beta 1 due to the stress involved with “missing a release”;
  • ironically, after Beta 1 it creates a false sense of having “plenty of time” before the next release, time that passes quickly regardless;
  • it causes certain elements of the workflow to be executed so rarely that they are not explicitly documented, let alone automated.

More importantly, from the perspective of the user:

  • it creates releases with many new features, some being explicitly incompatible and some being accidentally incompatible, which makes the upgrade cost relatively high every time;
  • it sits on features and incompatible bug fixes for over a year before becoming available to the user; and more specifically
  • it causes every “point zero” release to be extra risky for users. While we provide and recommend testing with alphas and betas, “point zero” is the first release of a given Python version for many users. The bigger a release is feature-wise, the more potential problems are hiding in “point zero releases”.
7 Likes

Previous attempts:

Can you clarify how this will affect the maintenance period for older releases? Currently the rule is that 3.x switches from all-bugfixes-mode to security-fixes-only-mode when 3.(x+1) is released, so naively this will halve the period that each release receives non-security-fixes. Is that what you expect?

2 Likes

The PEP as it stands proposes to shorten the regular bugfix period to 12 months (the final bugfix happening ~3 months after the release of the next version). This is covered, likely not very well, as last of the risks listed.

I’d really like to drop some of the unreliable and unnecessary steps from the Windows release process if it’s going to happen this frequently. Specifically:

  • no more GPG sigs
  • no more MinGW import library
  • no mid-release Tcl/Tk and libffi updates (except for CVEs)
  • no release blocking OpenSSL updates (except for impactful CVEs)

These are all mild annoyances that cause me friction and provide practically no value. GPG sigs are only good for non-Windows users, as the embedded signatures work better there, and I’m pretty sure MinGW users need to regenerate the import library with the correct version of their tools anyway, so we can just document it.

Tcl/Tk updates almost always require patching upstream, and so far so has libffi. OpenSSL is also an annoying overhead, though somewhat more critical so I’m prepared to spend my time on getting it ready. (These all require access to the PSF’s code signing certificate, which is why they all have to come through me for now.)

Unless we’re increasing the release cadence specifically to take dependency updates more frequently, reducing these points of friction is a good way to keep the whole process running smoothly.

2 Likes

Gotcha. I was mostly confused by this bit:

Consequently, while this change introduces the ability for users to upgrade twice as fast, it does not require them to do so. Say, if they upgrade every second release, their experience with Python is going to be very similar to the current situation

I guess this is why you say “very similar” and not “identical”, but it still might be worth highlighting exactly what would be similar and what would be different.

Ironically, it sounds like you’ve spent significantly more time generating these than it would have taken for someone to fix MinGW so it can use the regular import libraries: 19540 – Format reader for ILF format (used by MSVC-generated import libraries) does not properly handle data imports

Can you elaborate on what this means, for those of us who aren’t as involved in the release process? What do you do with openssl now?

Mostly just import the sources into our cpython-source-deps repository, run one of my custom builds (on our Azure Pipelines account, but using a VM that is mine) and push the binaries up to the cpython-bin-deps repository. Then someone else can make the version number changes in the CPython repo and verify that it works (if it’s not me doing it), but the process definitely gets blocked on being able to build signed binaries.

I don’t want to be potentially responsible for relating releases any more than necessary, and with more releases going on I’d prefer to not be rushing to update OpenSSL the day before a CPython release just because they put out an update.

So the workflow problem is: right now, every time we make a CPython release (including point releases?), we always ship the latest openssl at that time. And sometimes this means you have to do an annoying fire-drill, when a new openssl version happens to come out the day before the CPython release?

What would be the guarantee instead? The openssl that comes with CPython won’t be more than a week out-of-date? The openssl that comes with a CPython release might be arbitrarily out-of-date, but it won’t have any public CVEs?

I like the idea, but would also like to add the comment that the current longer preparation cycle isn’t entirely free of advantages. A five months beta/RC period that overlaps with an effective 9 moths release cycle may leave as little as four months of concentrated development time for new features, for those who are involved in beta bug fixing. In a project where most developers are not working full time, this could become a real limit to what kind/size of projects they can incentivise themselves to invest their time into.

Five months of beta does seem longer than I’d naively expect. It looks like 3.8 is planned to have 4.5 months of beta to stabilize ~16 months of feature development, and 3.7 had ~5 months to stabilize ~16 months of features, so I guess this schedule cuts down the feature period while leaving the beta period the same. But wouldn’t we expect that if we have fewer features, then we won’t need as much beta to stabilize them? If we were keeping the ratio the same, I guess 3.9 would split the schedule into ~2-3 months beta versus ~9 months features. @ambv, can you say more about why you chose to go with a 5/7 split?

This also double the number of version where Python need to potentially backport patches, and thus increase potential conflicts. So it may not be cost-less for you.

It also may increase the number of release-specific codepath that are included in libraries – but that’s likely to be mitigated by less changes between releases.

Depending on the upgrade of depreciation policies (that you mention in risks), that may not be the case. If deprecation policies stays n+2 between deprecation and removal that will quite shorten the time to adapt to new versions. Things sometime get deprecated/removed on n+1 in which case leapfrogging will not give a change to integrator to see a deprecation warning. This will be made worse on linux distro with LTS which are even more likely be leapfrog Python versions – the Python version chosen by developers might not be their direct choice.

I’m generally happy to see a faster release schedule; I share the concern of that 9 month would be tough for devs spending time on beta bug fixes.

Does it also double the number of binaries such maintainers need to build, ship and support if they have C extensions? My impression is that it would. This may be something that maintainers of complex C-based libraries (I’m specifically thinking of the scientific community) should be consulted over.

1 Like

Java adopts 6 month release cadence

Ruby looks to have a similar cadence as proposed in this PEP.

1 Like

Can you also post this on python-dev? I’m sure not everyone is on Discourse.

then

seems inconsistent. Is it 12 months or 9 months? Did you change your mind just before posting? :slight_smile: Or am I misunderstanding something?

1 Like

Ruby releases new version around every Christmas, 12 month cycle.
I prefer 12 month cycle too, because it can sync with 6 month cycle of Ubuntu.

1 Like

That’s a mistake on my part. It’s actually more of a 8/4 cycle if you look at the actual 3.9 dates. I just failed to update this part when fiddling with the dates.

It’s nine months between every “point zero” release. But the last regular bugfix release for a given version would happen at 12 months after “point zero”.

@ambv To quote your text again:

Python 3.9 will be developed for 12 months, 7 of which are spent on feature development (pre-beta), while the remaining 5 are spent on hardening the release (beta and RC).

So it’s 12 months before the initial release (7 pre-beta, 5 between beta1 and release). It does not include the bugfix period.

If we’re going to go to the effort of making the cadence faster, standardising on 12 months would make more sense than standardising on 9 (that’s what gcc have been doing for the past few years, releasing in late April/early May).

Syncing on the calendar year makes for straightforward alignment with other 6, 12, and 24 month cadence projects (including annual development sprints), and makes time planning easier for both individuals and organisations.

8 Likes

Essentially yeah, though it’s always been up to the RM whether to hold the release or not. “Arbitrarily out of date but with no impactful CVEs” is fine by me (we don’t expose a lot of OpenSSL’s functionality, and it’s easier to figure out whether a CVE applies or not than to update).

Officially avoiding most firedrills is a good way to describe what I’m trying to achieve here :wink: