Python LTS and maintenance cycles

Ideas
101

That’s the second time you’ve taken my words out of context. Think of a sentence as a statement within a closure; it is closely related to other sentences.

I want to emphasize that we developers need a point of reference for dropping support for Python versions. One significant reason is security issues; end-of-life (EOL) Python versions do not receive security fixes. This is a strong reason to drop support.

On the other hand, if a long-awaited feature becomes available, that’s another strong reason to drop support, as everyone would likely transition to the new Python version. Supporting older Python versions would become costly because users would abandon them. If LTS were in place, it could create another transition era similar to the one from Python 2 LTS to Python 3.

2 Likes
102

I’m not sure how else to interpret this. I took this to understand you will move your projects to only support 3.14 if free-threading is successful. Are you saying you meant it as an example? Either way, my example still stands, some people will determine what Python versions they support based on features and may only support the newest one.

1 Like
103

It’s been mentioned a number of times already, but there simply is no central group or “common voice” with the authority to dictate, or even encourage, projects to follow such policies.

Even if CPython did choose to implement a LTS style of support policy (and @ambv made it very clear that that’s never going to happen) the likelihood[1] is that most projects would simply ignore it, and continue as they are. So while maybe there is some form of benefit, but as the idea is almost certainly unimplementable, I don’t see how that matters.

  1. In my opinion. But feel free to disagree with me, or dismiss my opinion - all I can offer is my view based on over 20 years of experience working in Python packaging… ↩︎

5 Likes
104

This reads like social engineering to me - you want a cpython policy change to put social pressure on package developers to do support over a longer time period.

2 Likes
105

I think you’re both reading this the same as LTS, when this is something very different. This is essentially centralizing what already exists independent of the PSF maintenance cycle.

1 Like
106

I wasn’t making a broad generalization. My point was that if free-threading becomes available, I would personally drop support for other Python versions and move fully to Python 3.14. That would give me a very strong reason to justify dropping support or encouraging users to upgrade. And with no LTS in place, I have the freedom to request that they upgrade.

There have been multiple instances of Python 2 libraries reaching end-of-life (EOL) because their features became standard in Python 3. The same would happen with the GIL and free-threading.

Dropping support solely because of the structural pattern matching feature isn’t justifiable. It’s just syntactic sugar—I’ve been doing the same thing without it, and everything works fine.

107

I think you have the freedom either way. Free-threading isn’t as much of a win for me, but it will definitely help for some specific use cases.

I feel the same about structural pattern matching, though after playing around with Rust’s match statements, I’m curious how far it can be pushed.

1 Like
108

TL;DR; Interesting discussion, but I think this is a bad time to make decisions on future release schedules today, based on past decades experience, because we are getting into unknown territory of security regulations of the whole software development industry.

I think we should not see “past best practices” as a benchmark - even if we think we have them. It does not take into account what’s coming - i.e. industry-wide change in how commercial companies and open-source foundations and developers will have to change their approach to security - when the CRA (Cyber Resilience Act) in EU and similar regulations are live. The CRA effective date is 10th of October 2027 (it was just adopted by the EU council ~ two weeks ago).

For now, we do not know how this will impact industry, upgrade scenarios and patterns etc., we know however, that it will definitely change. My personal assesment is that it will be far cheaper for companies to upgrade software and keep it always upgraded (because of risk, financial and criminal penalties if they don’t) - which is completely opposite of today, when it’s cheaper to not upgrade - because there is virtually no quantifiable risk and cost connected with not upgrading. Currently most forces driving upgrades are only non-quantifiable beliefs of few enlightened engineers who beg their management to spend engineering time on upgrading the systems. I am generalising and exaggerating of course, but that’s what happens in many places.

But with CRA, it’s going to be set on its head - and I’ve heard (from people who are much smarter than me to understand the industry) that far faster upgrades of software will be driven by (and here surprise) insurance companies, that will refuse to insure your company if it does not follow the regulations.

I personally think that this will vastly speed up upgrade cycles of any software out there. But, we do not know yet, there are many factors - including the fact that standards that the regulations are going to regulate are not written yet (and open-source industry scrambles together to make them reasonable and good and bearable for the OSS community - and there are already some successes on that front).

So my point is - I think we need to basically wait (while monitoring and actively participating in shaping what’s going to happen) and discuss when we see the impact it is going to have. It’s very dangerous to draw conclusions on best practices for last decades, because it is the first time in software develiopment history that the whole industry is going to be regulated, and it means that any conclusions drawn on best practices from the past are very likely to be wrong.

For anyone who does not know what I am talking about - here is the great talk from EURO Python by Cheuk and Deb talking about CRA and it’s impact.

4 Likes
109

Should we wait or scramble together?

110

No, regulation in software is not new.
HIPAA, GDPR, and various government compliance regimes like FedRAMP, ITAR, etc[1] are all well established. In fact, some of those are discussed in this thread.

  1. Apologies to my EU based peers, I’m more familiar with the US gov space! ↩︎

111

Diferent subjects and objects.

  • The open-source industry (many foundations) are already scrambling to define standard.
  • We (Python communtiy) should wait with redefining release patterns until we see what happens

Of course regulation is not new. But as I wrote, this is the first time whole software development industry is going to be regulated. Everyone who releases software, provides services, exposes any kind of service to end users who are in EU will have to follow the regulations. Not companies that are building medical softwares, or cars. Everyone (except hobbyists who do not earn any money on their softwre - even for support - and do not “put software on the market”).

Everyone who publishses open-source project of theirs in Github, that is intended to be used “in any production”. Basically everyone (and every company) who develops software and makes any money on it.

So no, this is the first time when software development industry is going to be regulated as a whole.

112

So what you’re telling me is that EU regulation magically applies to everyone, no matter where in the world they are? Or do you have a different definition of the “whole” industry?

1 Like
113

Yes. If you want to sell software (or services) on EU market you will have to comply (so basically if any of your end users are in EU)… Some foundations in early stage of that regulation (when it was bad for OSS) considered blocking their websites and downloads in the EU.

And - same regulations are being developed and will be put in force in US and China and South America. Politicians from all those regions universally agreed on it already and EU is just first who passed the laws.

114

This seems relatively alarmist. Depending on what industry you’re in currently, the CRA might have zero changes to your security posture because you incidentally already comply, or that you have no customers in the EU. It’s been “on the way” since 2020, with only minor changes in language after 2022. The version that’s to go into effect is already known and not in full effect until 2027

115

I’m not in any of those regions either.

What exactly is the enforcement here? You’re implying that everyone will have to follow these rules “or else”, but what precisely are you saying and what precisely is the actual legislation? (Ignoring the whole “EU doesn’t make legislation, member states have to” part, since I think we can safely assume the majority of member states will indeed ratify this.) You said:

which implies that the moment I put a hobbyist project on GitHub, it has to comply with EU legislation. Even though I’m not in Europe. I really hope you’re just being alarmist.

116

We’re a bit off topic here but briefly: I’m American, Constitution. First admendment. Code is Protected speech.

117

Surely, it might sound alarmist, and a bit tangential, but I think it’s very relevant that we are all aware that “decades of experience” might and likely will soon become irrelevant.

My point I strongly believe (and a number of people I work with - I am a secuity commitee member in Apache Software Foundation and involved in CRA, and participated in several workshops in Brusssels where we helped to shape CRA) - that many patterns of releases/installation/update frequency we take for granted now, will just be heavily influenced and changed by those regulations which were not existing at all in the past.

Companies will be required to upgrade (without delay) to latest security fixes - this will be mandated by law (!) for commercial users (those who make money on software). Under the penalty (this is the “or else”) of huge financial penalties (significant percentage of revenue), and in some cases criminal penalties for the top management if they did not follow the regulations (if harm was done to “natural people” because they neglected security and there was breach of security).

Basically those commercial users will not be able to legaly take money from EU residents, if they do not have “EU compliant” label associated with the software and services they have - which will mean - for example - their whole software stack should not have “known, exploitable vulnerabilities”. The moment vulnerability is public and fix is available, everyone who takes a single EU from an EU resident, in watever form (might be serving ads to them), will have to apply this fix without delay (details of what delay it is etc. is exactly part of the standards the open-source foundations are scrambling now to define - together and in parallel to standard bodies). This is already happening (the standars being fleshed out).

Look at the finanancial penalties big US corporates have to pay for breaching EU privacy laws (and there are more penalties coming and AI regulation etc.). It’s definitely enforceable.

So in a very short time - 3 years - not the engineers, but top-management of every company who makes money of software will suddenly have an incentive to make sure their company upgrades to latest version of software they use - because that software will be the one that will have latest security fixes applied.

As “open-source stewards” (special light-weight new type of business entity introduced by the law) - foundations (and maintainers of the project within the foundation) will no longer beg our users to upgrade, our users will eagerly upgrade whatever we release, whenever there will be a security issue fixed (which in many case will be every release). Or so I think it might happen.

And please - don’t take my word for it - watch Cheuk and Deb talk i posted above. I am a bit of outsider here and I know I sound alarmist (though I see some very good outcome of this change for us - engineers), but they have much more merit here.

I think we are speaking in the same voice - see the talk. I am just attempting to anticipate that the whole thing will have huge and largely pivotal impact on how our users will install and upgrade our software - only we do not yet know how it will change exactly - we might just guess.

1 Like
118

As a Python user and a library maintainer:

10-year LTS by PSF would be effort mis-spent in my book.

There are companies that sell LTS or offer LTS subscriptions, let them do it, let them make money on folks who need this.

Now about libraries. Let’s say 3.10 is declared an LTS release, would you expect random pypi lib to eventually include special-cased support for LTS in their mainline versions and end up with, e.g.:

  • support 3.10
  • not support 3.11, 3.12, 3.13
  • support 3.14

Or would you expect community packages to backport bug fixes into older library versions?

That all being said, for some reason or other (Ubuntu maybe?) at least 3 teams I worked on had a soft rule to properly support even Python versions like 3.12 and pay much less attention to odd versions like 3.11 or 3.13.

I think that speaks to a mismatch between CPython release cadence for these specific teams.

1 Like
119

I believe the work I currently do is more strict than where the CRA will land when the things that have not been defined yet are defined. I understand there are plenty of companies where security is an afterthought. If anyone is alarmed at that having to change, I don’t have much sympathy for their alarm; Though I do have some sympathy for those who this will place in a position where they have to fix something that would have been easier if they were allowed to do it the right way the first time, but corners were cut for cost.

The idea that best practices will radically change because of a law doesn’t strike me as likely, especially when the process involves the industry in multiple capacities for determining definitions. Best practices aren’t just whatever someone writes down, even if reputable websites maintain such writings to help improve collective knowledge. They are rarely even “only one way”, except perhaps at the most abstract expression of them (ie. The principle of least privilege is possible to enforce in multiple ways when designing an application) They are the result of logically doing the best we can with the knowledge and tools at our disposal, and will always evolve faster than the law is equipped to handle enforcing them in fields that are still actively changing.

I cannot imagine a situation where the CRA penalizes for the existence of a zero-day vulnerability not caused by negligence, taking reasonable precautions to test patches (and therefore having a reasonable delay) before deploying them or being able to certify that the known vulnerability in a component is impossible to trigger in the specific use of that component by the product; So, I’m not at all alarmed, but I am willing to update my level of alarmism if any definitions, thresholds, or penalties that have yet to be hammered out end up unreasonable.

Which is a long-winded way to explain my calm. It’s not because I don’t understand, it’s because I don’t think it’s something to be alarmed about.

With the above context on my lack of alarm in general expressed with the hope that I can help prevent the spread of undue alarm out of the way; When focusing on the impacts in the context of this discussion, I think it will not impact how people update Python. Using an end-of-life version of something is going to be a problem; Not applying security updates would also be problematic, but both are already problems and would not change when the CRA goes into effect, nor would the process of updating Python change. The existence or non-existence of an LTS version does not change this either with or without the CRA, it would only change the versions considered still supported, and we’ve already been over the issues that could cause from technical and social angles in a way that rules out the perceived viability.

2 Likes
120

I think that … I do not know. But I know major factors that make companies decide when to upgraed will definitely change. And I do not know (and basically no-one knows) how.

During the next 3 year various actors involved will clarify all the “standards” practices - or even (as I do now with Alpha-Omega fund, @sethmlarson and Apache Software foundation with our “Airflow Beach Cleaning” project - when we want to analyse and help 700+ of dependencies from Supply Chain of Airfow to improve their security processes and approach - we try to work out and experiment with “new practices” - more suitable for the future as we see it.

In many ways it will turn on its head many things - who will be responsible for fixing issues - for example corporate users will largely be mandated (!) (under penalties) to attemp to contribute back security fixes they applied to their forked version of software and to be able to prove they tried. It will change who will pay for what - we can likely expect a lot more money to trickle down from commercial users to projects they use because that will be cheapest way for them to maintain security. This might change some basica assumptions in this discussion - how busy maintainers are, are there more people who will be actually paid to do release and security management? Finally maybe LTS’s will be more - or less - needed by the companies who will have to not only upgrade OS-s but also all the sofware that runs on it to say - latest versions (might be that this software will not run on LTS versions) …

A lot, a lot of unknowns. That’s all I wanted to say.