It’s an over-simplification (and as @njs pointed out, is also wrong 
). My main point was just that there are fewer variables on Windows when it comes to questions of compatibility. But “fewer” != “none”…
Anyway, it was something of a distraction from the main point here, which is PEP 600.
The main answer to this question is the same (IMO) as with a lot of decisions in open source - resources and (relatedly) people’s attention. All of this work is being done on a volunteer basis, and simply having people interested enough to contribute is a significant factor. Right now, we have people engaged and willing to develop, promote and discuss PEP 600. If we leave this for whatever period your suggestion translates to (months, maybe even years?) then we risk losing that interest, and have PEP 600 get abandoned through lack of interest. And then, when the existing standards do start reaching EOL, we have another rush to make a decision.
I’m not saying this is the only (or even the major) factor in the process here, but it is a factor.
As a second point, @njs has claimed a number of times that work is being done under the existing PEPs that could be avoided if PEP 600 gets accepted - so that’s extra resource and effort freed up by a quick decision. I’ve not seen any concrete details on how much effort would be saved, and in all honesty, I wonder if @njs might not be optimistic about the savings here, but I’ve very little idea of the details of what work is involved, so I’m fine with taking his word that there are savings to be gained from a reasonably quick decision. Again, I’m treating this as a minor, but not irrelevant, factor in not delaying too long.
First of all, by its very nature this is entirely a matter of opinion. None of us can know what might happen. But secondly, and more importantly, why is it so disastrous if we do have to revise PEP 600? PEPs (or rather the features they define) get revised and updated all the time. And furthermore, PEP 600 says nothing about transition processes. Nor did the previous manylinux PEPs. Any problems we find in the transition from manylinux1 → 2010 → 2014 are likely (again, just IMO) to be transition problems, and so not in the scope of PEP 600 anyway.
To summarise, I hear your points, and acknowledge them. But as the one making the decision on the PEP I don’t plan on delaying a decision for so long that the momentum is lost. I’m not deciding in haste (ask @njs what he thinks, if you don’t believe me 
!) but we’ve had a decent amount of time for discussion now, and I think we need to move past vague “we mightn’t have spotted everything” concerns. We’re not aiming for perfection here, just for agreement on a workable way ahead.
If @njs wants to add some words to the PEP summarising the concern here and adding a response, that would be great, but I’m not insisting on it. Equally, if he has anything further to add in response, that would be good too.
OK, and once again, things have gone quiet.
I don’t think there’s going to be much more of a substantive nature added to the discussion here. I’m not sure I’d describe what we have as “consensus”, sadly, but I do think that “no significant objections remain that haven’t been addressed or responded to” applies. So I’m inclined to approve PEP 600.
One thing I would like to see discussed, though, is how the acceptance of PEP 600 would affect the work going on right now to deliver manylinux 2014. It’s been frustratingly difficult for me to get a real sense of what work is (or will be) going on “behind the scenes” to actually implement the manylinux specs, and there is definitely part of me that simply wants to ignore the problem, and say that if no-one else is interested in exploring the question, then I’ll just assume all is fine and not worry. But I feel that one last attempt to get input is in order.
@dustin, as the author of the manylinux 2014 PEP, do you have any reservations regarding the approval of PEP 600? Are you comfortable that timescales for implementation of perennial manylinux can be managed so as not to derail the work going on currently with 2014? Note that I’m perfectly OK with approving PEP 600 but for its actual implementation to be delayed - approval does not imply “you must implement this right now”.
@njs, I’m sure you’ll have comments on how PEP 600 won’t cause problems for manylinux 2014. Please don’t address them to me - I don’t have the knowledge to evaluate them. Rather, please address any comments or clarifications to @dustin, but do so in this thread so that the information is available to everyone, and not just covered by “we had a chat and it’s OK”.
Anyone else who feels they have anything new to add, please also feel free to chip in. But if it’s something that’s already been discussed or pointed out, please don’t bother. I have re-read all the discussions and frankly have a low tolerance for going over the same ground yet again.
(Caveat: I haven’t read most of this thread or the new PEP 600 draft.)
I think that PEP 600 is probably moving us in the right direction, however I don’t quite see what the rush is.
While I think waiting for some adoption level of manylinux2014 to be achieved is probably unnecessary, I do think delaying any implementation until the manylinux2014 rollout (which is moving along just fine) is more or less finished might help us focus on one thing at a time, and give everyone involved (including myself) a little more perspective on what actually needs to change before we start changing things.
FWIW, the idea of accepting PEP 600, and then following on with actually implementing it after https://github.com/pypa/manylinux/issues/338 (the manylinux2014 rollout) has been resolved makes sense.
As long as the implementation happens some time in the next 12-18 months, that should be soon enough to address any strong demand to target the CentOS 8+ era of distros.
So, where are we now with the manylinux2014 rollout? The only non-optional action item on the referenced issue (that’s not related to transition from earlier manylinux versions) seems to be the final release of auditwheel 3.0.0.
As far as I am concerned, there is nothing remaining to be discussed on PEP 600, in terms of what the proposal states. No-one has asked for further changes to the PEP, and leaving it sitting here “in limbo” is doing no-one any favours.
So therefore, in the next few days, I plan to accept PEP 600, leaving it to the manylinux developers to plan the timescale for implementation.
OK. I’ve thought long and hard about the decision here, and I am going to accept PEP 600. Congratulations to @njs for getting the proposal through to acceptance.
One reason I found this a difficult decision to make was that I remain concerned that the way the PEP specifies compatibility, in terms of a manylinux_tag_is_compatible_with_this_system function, does not match well in practice with how existing installers (i.e., pip and packaging.tags) currently handle compatibility. So I’d strongly recommend that implementing PEP 600 in packaging.tags be a high priority, in case it exposes any issues with the PEP.
Ultimately, though, the implementation plans are down to the manylinux developers, and I’m sure they will do a great job.
Is there a tracking issue for the PEP 600 rollout so we can see which tools already support it and which tools need to add support? I see that people have opened https://github.com/pypa/manylinux/issues/501 and https://github.com/pypa/packaging/issues/280 , but neither of those feels like a systematic tracking issue similar to the one for manylinux2010.
Nope, there isn’t.
@mattip thank you for creating a tracking issue! And thanks @mayeut and @pradyunsg for editing it.
@takluyver @njs I suggest you take a look, improve it, etc.
With almost all check boxes ticked in the tracking issue, what would be the process to mark the PEP Final (or Active ?) ?
I Also opened a PR to reflect the following statement from PEP 600:
When this PEP is accepted, the previous manylinux PEPs will receive a final update noting that they are no longer maintained and referring to this PEP.
On reflection, is it right to mark the older PEPs as superseded? A tag of manylinux2014 is still valid, and what it means isn’t stated in PEP 600. So “Superseded” seems incorrect.
A tag of
manylinux2014is still valid, and what it means isn’t stated in PEP 600.
You’re right, IMHO, PEP 600 should be amended to also mention manylinux2014 as it does very clearly for manylinux1/manylinux2010.
If that was you’re only concern, you can skip the following that emphasize why those older PEP should, IMHO, be superseded.
PEP 513 is “broken” as it refers to “libncursesw.so.5” which is in contradiction with PEP 600 and will install a package that can’t be loaded on not so recent systems if linked against.
Auditwheel now breaks rules from PEP 571 / PEP 599 because it allows a new whitelisted library which is legit per PEP 600 and requested by users of manylinux2010/manylinux2014. This might extend to other libraries (zlib comes to mind immediately) in the near future.
The full intent of PEP 600 regarding older PEPs:
Previously, we had an open-ended and growing commitment to keep updating every manylinux PEP whenever a new Linux distro was released, for the rest of time. By making this PEP normative for the older tags, that obligation goes away.
When this PEP is accepted, the previous manylinux PEPs will receive a final update noting that they are no longer maintained and referring to this PEP.
I hadn’t noticed that the other manylinux tags were (re-)defined in PEP 600 (it’s been a while since I reviewed it ) so thanks for the reminder. I’d just picked manylinux2014 to check at random, and luckily spotted the omission.
But yes, given those definitions, PEP 600 needs an update to redefine manylinux2014 in terms of PEP 600 tags (or PEP 599 needs to remain active, but that seems to undermine the intent of PEP 600).
I feel like that’s a significant enough clarification that I’d like confirmation from the PEP authors, @njs and @takluyver that they approve it. It should probably also be confirmed by the manylinux2014 author, @dustin.
For the purposes of that review, could you summarise the proposed change here, so people don’t have to get it from the PR?
Proposed changes:
manylinux2014 aliases in the Legacy manylinux tags section. There are a few more since manylinux2014 also defines platform tags for aarch64, armv7l, ppc64, ppc64le, s390x.manylinux2014_compatible module boolean attribute for compatibility with previous specifications in the Package installer section.manylinux2014 aliases to the manylinux_tag_is_compatible_with_this_system example.manylinux2014_(x86_64|i686|aarch64|armv7l|ppc64|ppc64le|s390x) to the list of regexes package indexes are recommended to accept.