[REJECTED] PEP 605: A rolling feature release stream for CPython

Unless you believe that 3rd party library developers will expand their matrices to support multiple active/bugfix versions of Python, and yet won’t expand it to include a single beta release, then that would mean neither current proposal is okay and you’re advocating for status quo? (Which is fine, I’m not against status quo here, just wanting to clarify whether this is a “-1 vs the other proposal” or “-1 vs any change”.)

That does seem possible as this proposal does seem to suggest that something could come in but then get removed between betas which would increase support costs (e.g. adding to an API in a beta that’s positional-or-keyword but then making it keyword-only in another beta when you realized that’s better).

That’s quite unlikely to happen for projects that try to stay compatible across several python releases, in that the codebase obviously cannot use features from python 3.(X+1).0bY (which as you say might be changed in 3.(X+1).0bZ) while being usable across several python minor versions. Usually, having the beta version in the CI matrix is only meant to catch deprecations and breakages in current APIs early on, and stay compatible with the newest (pre-)release.

One non-negligble cost I see is for the many projects that exclusively use conda to manage their dependencies in the CI (e.g. pandas). I don’t want to restart the pip-vs-conda debate here, but this is where things stand currently. For example, I’ve tried to package the 3.8.0 alpha/beta releases (1, 2, 3) for precisely this purpose, but couldn’t get things to work (3.8.0a3 was merged through my PR, but all the work was by @jjhelmus). The most recent PRs have been stalled for a while as well. And because everyone is maximally busy, pandas still has no 3.8 builds or CI.

If those packages were to get built in a timely manner after each beta-release (i.e. not taking 1-2 months out of the 2 months until the next beta), then it would be relatively easy for these projects (which are using conda for dependency management) to follow suit in their CI. This at least should be made easier by the smaller increments between betas.

Projects using a different setup for their CI might conversely benefit from an easily downloadable CPython interpreter builds for those betas, cf. this discussion thread. Certainly, few projects will be able to build CPython betas from source as part of their regular CI.

Hmm, it would be nice if you could make the PEP a bit terser. Right now it’s a wall of text, and I’ve only read parts of it.

Edit: actually, I was reading Nick’s “design discussion” post. But still :slight_smile:

2 Likes

That would be helpful. One way you can achieve that is by introducing pictures. Everybody likes pictures and so far PEP 602 is clearly superior in that regard :laughing:

Joking aside, I doubt we’d see much adoption of the beta stream as described. For users to be able to tap into it, libraries would need to be consistently available for it. For a library maintainer making binary wheels available for those betas is laborious enough but first they’d need to make sure their library works at all. If this is already a concern for an annual release cadence, I can only imagine it being a deal breaker in an every-other-month cadence. This is my -1 right there.

But there’s another thing that bugs me here. One of my goals with PEP 602 was to help the core developer, the volunteer. Make things more predictable, easier, smaller. PEP 605 on the other hand leaves the current status quo intact (stable releases made rarely and thus large) while adding additional complications to core developers. The expectation would be to keep the master branch suitable for production use. Last month and the many adventures we went through during that time shows the team is not quite ready for that.

Finally, despite what the PEP here suggests, I still believe the annual release cadence would get features faster to users. Both compared to the status quo and to what PEP 605 is proposing. Sure, maybe not every user would be able to use the latest version every time. However, every new version of Python would see adoption of a significant subset of those users. After all, it’s not like all users would orchestrate skipping the same versions, right? On the other hand, I feel that the beta stream wouldn’t see significant adoption. So, new features would be guaranteed to reach almost all users slower.

1 Like

Note that our available data indicates the most widely used Python 3 version is Python 3.6. There isn’t a strong reason to believe that would be any different if we were publishing 3.9 right now instead of 3.8. (It does lend weight to the argument that folks are OK with just skipping releases, though)

I do agree on the “wall of text” problem, but it isn’t clear where pictures would help in this case.

For wheel archive availability on beta releases, we probably need to better emphasise that for projects that also support the latest stable release:

  • compatible pure Python wheels can be published once and left alone
  • binary wheels can also be published once and left alone as long as they target the stable ABI

So pysip based projects, for example, would be usable, as long as pysip itself was able to keep up with the beta releases.

We also wouldn’t necessarily need to start with a 2-month cadence straight away. Instead, we might start out with a 6 month cadence (speeding up before a feature release)

The situation is actually even better than that: for projects that also support the latest stable release, the only cases that are likely to cause extension module breakage are those where public structs change size, and even then, extension modules are mostly accessing those through pointers provided by the runtime rather than allocating memory themselves.

The PEP currently isn’t focused on those cases - it’s focused on in-house modules that are actively using the provisional APIs, and hence need to be updated when they change.

Given the initial feedback, it sounds like we should flip that, and emphasize that extension module rebuilds are only likely to be outright required at the following points:

  • on the first alpha or beta release of the new series
  • on each subsequent alpha release (with “Do we want public extension modules rebuilt?” being one of the criteria for making an alpha release)
  • on the first release candidate (to remove the pre-release qualifier)

Some of the other text in the PEP would also need to change, as at that point we would be encouraging library authors to publish pre-built wheels for the beta series (we’d just be trying to minimise how often they needed to update them).

(Note: pyarrow is an especially complex case that’s already hard to handle reliably on stable CPython releases. We’re hoping conda-press may help on that front, by allowing conda-forge to be used as a wheel building environment)

1 Like

I have had data shared with me saying 3.7, so I don’t know if we can ever get a clear, definitive data point on this.

You focus a lot on users, and very little on the package maintainers responsible for your -1 vote.

As Nick pointed out, the churn between each beta release would likely not require changes each time for most 3rd party packages, and an actual guarantee on the stable ABI being stable even throughout beta should help move extensions onto it and ensure their stability as well.

In contrast, you’re offering breaking changes once a year, and despite users having the flexibility to skip versions, maintainers do not have the same flexibility. As far as “orchestrating skipping the same versions”, I would expect that any version skipped by numpy is going to be skipped by the entire data ecosystem out of necessity, and meanwhile users are faced with a stable release that doesn’t have what they need.

This is true, but I think more aspirational/hopeful than how you present it. The more rare stable releases will have had much more usage and be more ready than the status quo, and the “additional complications” are probably the best way to prevent the “many adventures” we have when changes aren’t getting real use. I don’t see a problem with injecting a need for more caution into our process - each Python release impacts literally millions of people, and we ought to take that pretty seriously.

Under PEP 602, we would already have shipped async streams (now being removed, last I heard) and a “stable” initialization API (now no longer guaranteed stable), which would mean those changes would have been permanent. Under PEP 605, the lead time on committing to those designs would have been longer but releases containing them would have come sooner, giving us a better chance of avoiding the issues entirely.

I suspect the biggest sticking point is going to be calling the fast-track releases “beta”, even though it is defined differently from how we’ve used it in the past.

Perhaps we should just choose a totally new name, but keep the versioning scheme as described here? That way it still works nicely on a technical level, but won’t scare people away solely on the name.

I think if binary wheels can be readily made available without placing a significant burden on publishers, and don’t need to be routinely rebuilt for every beta release, we should be able to work through those reservations.

The initial draft spends a lot of words on “How things can go wrong”, as I spent a bunch of time thinking about that.

Even in the full CPython API though, ABI breaks are relatively rare (mainly relating to public structs changing size), so it would be better to have a clear distinction between “ABI breaking” pre-releases and “binary compatible” pre-releases.

What is the actual motivation to change away from the status que. Unless I am missing something the arguments for are “because we can”, and the arguments against are “This will create too much extra work and confusion”.

Is a faster release cadence solving a real problem that people actually have? If it is, fine. I don’t like changing it because it creates more work for me, and slows down feature access because I will end up ignoring most python releases.

But if it is not solving someone’s real problems, whats the point? Just because Azure Pipelines makes it so we can, does not mean that we should.

1 Like

I see what you’re saying but please note that your choice of language is not helpful. Assume good faith. Saying that it seems the argument for the change is “because we can” is needlessly adding negative emotion to the equation.

As for reasons for the change, they are documented both in PEP 602 and in PEP 605.

This is fundamentally the problem with PEP 605, though. Nick’s “Design notes” seems to pretty clearly suggest that numpy won’t be publishing binaries for the beta stream.

IMO, it’s pretty much a showstopper for any proposal if we can’t be reasonably certain that the key binary extension providers (the scientific stack being the most obvious example) will provide binaries for all the releases that we expect to be generally usable.

Something I think both our PEPs are assuming without spelling out is that readers already understand the potential impact of faster turnaround times for all kinds of contributions, but especially commercial ones.

I believe you, Steve, and I agree on the basic shape of the problem to be addressed, so maybe this is one of those cases where we should split out a separate shared background PEP, so we can cut that aspect out of the individual proposal PEPs?

Are you sure? It’s true that in both cases the stakeholders changed their minds later but the moment they did was informed more by the release calendar than by the time it took to understand the flaws in the respective implementations. In fact, the entire point of reverting Streams was to avoid shipping a flawed design. A faster release cadence would inform this even better, pushing some of those work-in-progress to PyPI instead.

More importantly, I feel this argument is invalid in the sense that there’s always some likelihood that a decision was wrong and it will have to be reverted in the future. This is a problem, yes, but if you want to optimize against stable users ever seeing this kind of thing, we should maximize the number of years between releases. What you’d get in result is the Python 2 to Python 3 migration.

Maybe in the case of Streams and the initialization API an 18-month cycle was drawn out enough to catch those deficiencies. But we’ll see after the release if there were other decisions that should have been different. Heck, we’re still discovering deficiencies in decisions made around Python 3.5… or even 19 years ago.

2 Likes

There in lies some of the problem here. Both PEP rationals do a really poor job of defining the actual problem they are trying to solve, with one not even attempting to define the problem. They both really do read like Because CI tools exist, we can do it, so we should do it.

Maybe that’s the bias of someone who thinks the 18 month lead time between releases is a feature of the language rather than a flaw leaking through. I don’t see a problem in the rationals.

Right, which we don’t want to do.

Or you get provisional packages, which we no longer want to do.

So we can try “provisional” releases (PEP 605), or more frequent stable releases (PEP 602). But the latter doesn’t look at all like the other ideas we’ve had to solve this problem, so is it really even trying to?

They should, but that section is based on their current plan and they haven’t been consulted specifically on this proposal yet.

Note that unlike real releases, the beta stream are all fundamentally compatible (where the core team gets to determine how compatible, and we argue that we should aim to maintain compatibility better).

So while 3.8 requires new wheels (and as a result, doesn’t have them all yet), each beta release can use the same one. Bugs/changes found between releases would also impact the next release, just as today, but they would be found far sooner.

Right, and I think this is the aspect where the initially published version of PEP 605 painted a picture that’s far more dire than I believe would actually occur in reality. So Paul’s impression of the current text is correct, but the current text is excessively pessimistic regarding the viability of publishing rolling release wheels that target the full CPython ABI rather than the stable ABI.

I’ve put a PR up that I believe should fix that problem: https://github.com/python/peps/pull/1190/files

The major change is that it brings back routine alpha releases, but with a very specific purpose: marking ABI breaks in the full CPython ABI.

Thus, there would always be at least one alpha release for each release series: the very first pre-release, which defines the new ABI compatibility tags. Whether there are any more alpha releases between X.Y.0a1 and X.Y.0rc1 would then depend on whether or not we introduced any actual compatibility breaks in the full CPython ABI.

1 Like