But we can not know about which Fedora version will be base of next RHEL…
I agree that Fedora is important distribution too. But since Fedora is about 6 months cycle, I don’t think we should schedule Python release based on it. It is same to Ubuntu non-LTS.
Correct. I just wanted to point out that the release cycle of RHEL is not important here.
I don’t think Fedora users care all that much about having the latest CPython powering their systems. (And developers, venv users, are fine – they can use new and old CPythons packaged without system integration.)
But, as part of releasing Fedora, all the packages are re-built with the new CPython version. This tends to uncover lots of issues both in CPython and projects that use it. We had 500+ bug reports for 3.8 – very many were not CPython issues, but we did get good overview of how AST changes, buildsystem changes, API removals, kw-only args, vectorcall, XML ordering, etc. affect the ecosystem.
Release alignment affects when we can do this (alpha/beta/rc), and how long we have to help finding and fixing the issues.
It’s certainly possible to do the testing in other places, or by other people. I don’t want Fedora to dictate CPython’s release cycle. On the other hand, there are benefits of aligning with a project focused on integration – hopefully enough to keep this in mind as one of the factors.
A small feedback from a user. I personally prefer long release cycle, so I would side with Steve Downer on two year cycle.
It is easier to communicate feature-wise on a big release and to motivate people to upgrade, because it creates a Momentum. More frequent releases create less momentum because of the repetition. That’s clearly a subjective opinion of course but I do stand by it.
Another feedback I would like to give is that in my company, people do not really have smooth upgrade path. When they started using Python, they settled on Python 3.1 . Part of the company has now upgraded to Python 3.5 but the rest is still stuck on 3.1 . I am trying to initiate a move to 3.7 but it will take probably one year to have it executed. This means two things :
My company does not plan release by relase upgrades. They jump from an antiquated version to whatever is the current one, usually with a kick in the butt (“What, windows XP is not maintained anymore ?”). So all the plan about feature deprecation for two releases, extra warnings and stuff does not really work. When they did the 3.5 transition, nobody checked what happened with 3.2, 3.3 and 3.4 . With a one year release cycle, the jump will be even bigger and I am concerned by this.
It takes more than one year to complete an internal migration. It is frustrating to work one’s ass to organise a big migration and having people come to you asking why we moved to an obsolete version since a new one is out (grrrr).
These two points seem to be contradictory. So is the former a personal feeling while the latter is how things are for you at work?
Assume everything will shift accordingly such that deprecations last about as long as they do now time-wise (this is not guaranteed, but I’m willing to bet this is how we will solidify our deprecation process). I.e. if we move to an annual release cadence we could quite possibly set a policy of deprecations must last 18 months in a final release which would mean two releases at an annual release cadence or one release under two years.
They are not as contradictory as you could think. Because bigger feature create a bigger momentum, it is much easier to advocate for them in my work environment. For example, I plan to use the progresses in type-annotations as a motivator to switch to something newer than 3.5 .
Less releases also simplifies the landscape of which version we should target.
This is akin to what used to happen for me at Boeing (in the 2.2 → 2.6 era), with two major things dictating the Python versions used:
It’s not a model I’d recommend if continuous incremental upgrades are an available alternative, but it’s also a model that’s likely to remain pretty common (especially in industries with strict certification requirements).
This alternative proposal has now been published as PEP 605, and has a separate discussion thread at [REJECTED] PEP 605: A rolling feature release stream for CPython
As discussion on this particular proposal died out and I’m happy with its current form, I’d like to file a motion for final comments on this PEP. Given no additional issues are identified, I’d like to put the PEP under the Steering Council’s consideration on October 17th (two weeks from now).
The Steering Council will likely want to wait with deciding on the PEP until the alternative PEP 605 takes final form and is sufficiently discussed. That’s expected, I don’t mean to rush things. However, release cadence PEPs are special in the sense that they are literally tied to the calendar. I would like to signal that the proposal described in PEP 602 is ready.
One consequence of this is that on Monday, October 14th I will be releasing the first alpha of Python 3.9.0 as described in the PEP above. I thought about this for a while and I hope you won’t be seeing this as forcing anybody’s hand. In fact, should PEP 602 be ultimately rejected, this is a harmless event, we will just have one additional alpha tagged unusually early. However, should PEP 602 be accepted in the future, the calendar described in the PEP will be upheld.
We actually discussed this at the last steering council meeting and we decided to let PEP 602 and PEP 605 try and finish their discussions naturally before we exerted any time pressure to try and resolve this before 3.9 scheduling would be impacted and before the election starts since that will basically freeze up PEP decision making until mid-December.
I hadn’t raised it before, as I thought you were already planning to address it, but the questions in PEP 602 – Annual Release Cycle for Python | peps.python.org need concrete proposed answers in the PEP itself.
It would be fine to make that a simple adjustment that assumes a non-trivial proportion of users will only be running every other release:
__future__ imports: 2 releases (up from 1)That way, these aspects would be essentially the same between PEP 602 and PEP 605, and the core process decision to be made would be between:
source-only releases never fit the brain (nor the need) for Windows users. Any maintenance release shall have installable binaries (and continous integration environements)
Matplotlib, scikit-learn, IPython, Jupyter, yt, SciPy, NumPy, and scikit-image have proposed:
Where:
This NEP recommends that all projects across the Scientific Python ecosystem adopt a common “time window-based” policy for support of Python and NumPy versions. Standardizing a recommendation for project support of minimum Python and NumPy versions will improve downstream project planning.
For example, they suggest:
This project supports:
- All minor versions of Python released 42 months prior to the project, and at minimum the two latest minor versions.
- All minor versions of
numpyreleased in the 24 months prior to the project, and at minimum the last thee minor versions.
And they also note:
Given the current release cadence of the Python, the proposed time (42 months) is roughly equivalent to “the last two” Python minor versions. However, if Python changes their release cadence substantially, any rule based solely on the number of minor releases may need to be changed to remain sensible.
It looks set to be accepted soon.
The intended audiences for source-only releases are redistributors and folks that were building their own Python from source anyway (and in many of those cases, it’s going to be the tag in the git repo that matters more than the published tarball).
Folks relying on the python.org binaries are going to want to move on to new releases at the end of the regular maintenance period, since they don’t have any way to effectively consume the source-only security releases. (Neither PEP 602 nor PEP 605 explicitly discuss this, since it isn’t a change from the status quo, but I assume it was one of the factors in PEP 602 reinstating 18 months of regular maintenance for each release).
The Steering Council asked me to address in PEP 602 how policies dependent on the release cadence should be updated. I did that and my answer to those that were identified in the PEP is pretty much: “the policies don’t need to be updated”.
The PEP body in this Discourse topic has been updated with this change. The diff on the peps repository is linked below.
I’d like to bump this, because I never got an answer. I personally like 602 way more than 605 (I consider 605 too complicated and not actually solving the problem). However, 605 has a section called Release cycle alignment for prominent Linux distributions. That topic is very important to me and we stopped discussing this entirely. Does it mean it is out of the question? If the release date is synchronized with Fedora and Ubuntu releases, it would really bring benefits, not just to Fedora and Ubuntu, but to the entire Python ecosystem.
I understand that there are PyCons and sprints. But how much are their dates non-negotiable? Can we plan for the release 1 day sooner than currently planned? Can it be 1 week sooner? Can it be 2 weeks sooner? Can it be in September? What are the implications? For example, what precondition other than the release date actually fixes the date of the core sprint to September?
Another argument to align the Python 3.9 schedule with Fedora schedule is that Red Hat has a team working on Fedora to ensure that all Python packages are working well with the next Python (Python 3.9 here). We do bug triage for each broken package and attempt to report issues to upstream (broken projects and Python upstream) as much as possible. We are also actively working on fixing these issues (with our limited resources).
It is related to the PEP 608 “Coordinated Python release” that I just published: PEP 608 – Coordinated Python release | peps.python.org
Core developers have no say in when PyCon is being held. They also have little say in when the annual sprint is happening because finding a date that is sufficiently far apart from PyCon and fits everybody turns out to be pretty impossible. Finding the “US Labor Week” was a fluke on my part and I doubt we will be moving that for future years.
The plan in PEP 602 talks specifically in months and not in concrete dates because all events: PyCon, the annual sprint, committers’ vacation and sick days, are different each year. We also sometimes identify issues that push us back a week or three. So we’re talking in months and not in concrete days. But we’re talking about leeway within the months specified. Sadly I don’t think there is any leeway forward.
Note: with the current status quo the dates are just as non-synchronized. The release schedule of 3.8.0 turned out to be misaligned for Fedora and if we keep the status quo, 3.9.0 will also be misaligned (because we only ever release later than planned, never sooner that planned).
That being said, it would of course be cool to have the release schedules aligned for mutual benefit. But I don’t understand what your needs are. I mean, can you specifically tell us which date you need to move and by how much for this to be helpful to you? For example, @brettcannon suggests that maybe we could call “Beta 4” our “Release Candidate 1”. No dates would change but the stability expectation would be different. You’d have your RC1 in August. Would that help?
Looking at current proposal in PEP 596 and the expected Fedora 33 schedule.
At 2020-08-25, Fedora 33 Beta Freeze starts. At that date, Python 3.9.0 is still at b4. Having it as rc1 would be much better. When Python slips, we would satisfy with b4. We want to avoid b3, because there are dangerous changes happening between betas, such as changing the bytecode magic number (and we need to rebuild the whole distro when this happens which is hard after beta freeze, but not impossible).
At 2020-10-06, Fedora 33 Final Freeze starts. Python 3.9.0 final is scheduled 1 day before that. It is not very uncommon that such a release slips by a day or two and what’s worse is that Fedora freezes at 00:00 UTC and Python schedules deadlines as AoE, adding additional 36h difference. That’s why I’d like to aim for at least 1 week earlier. 3.9.0rc2 is currently scheduled at 2020-09-14 which is quite good, but moving it sooner would make me sleep better.
So, technically, this boils down to:
It doesn’t really matter that much what is it called, but rather what are the guarantees behind that. (OTOH shipping a Fedora release with an RC version of Python definitively sounds better than shipping it with Beta.) Doing a real RC at the time of current Beta 4 would be indeed nice, however I’m not that sure that it would not require further adjustments of the rest of the milestones. IMHO shifting the schedule by 2-3 weeks instead would make it more likely that we don’t end up with major differences between rc1 and rc2.
The proposal I floated by @ambv was to drop b4 and spread out rc1 and rc2 so they were a month each. So it’s not just a name change but an actual shift in the length of the RCs.