To follow up some of the discussion here (and to jump around in order a bit to speak to the non technical stuff first):
While I’m one of the people in the other thread advocating to not use the PEP process, it’s got nothing to do with not interacting with the community. It was mostly my fault, I’m in the middle of a job search (well tail end of it now hopefully) and I simply got lazy (or more charitably, I was distracted) and answered inline in Github rather than asking to have it published here and answered here.
Most of the other discussion around this PEP did occur in the open, it was just quite some time ago and on the disutils-sig mailing list. I do not believe that we had an in person discussion about it at PyCon (other than people saying they wanted packaging signing, and pointing them to PEP 458) but I could be misremembering that.
No. This PEP completely transparent to package authors and to package consumers (except if it detects something going wrong, as it does add additional failure modes that may be exposed to end users).
In the general case, no (for pip yes but for different reasons).
No.
Fundamentally the impact to various groups (at least in this PEP) is roughly the same as going from HTTP to HTTPS was. For your average publisher/consumer of Python packages, it’s just something that the tooling handles for them, for PyPI it requires changes to the infrastructure and for a tool like pip it requires code to handle the new protocol.
It sort of isn’t I think? Random consumers of PyPI basically shouldn’t care about the details of this PEP anymore than they care about the details of how TLS works.
I don’t think a PEP like this is actually super useful, this feels to me more like something that should be documented either on PyPI or as part of packaging.python.org. I mean it could be documented as a PEP as well, but ultimately to me it’s not really asking for a proposal, it’s just documenting how to-s.
Particularly (3) isn’t related to PEP 458 at all, since it doesn’t seek to solve that problem at all (I could think of some ways to make it part of the solution, but it would b e apretty minor part overall). (1) and (2) the answer as far as this PEP is concerned is “you don’t have to do anything, it gets handled the same way TLS ensures that an attacker with a privileged network position can’t change the content of responses”.
While we could discuss the relevant merits of different problems to focus on solving, we don’t really get to dictate what exactly we have people willing to work on. This particular PEP was written by volunteers (If memory serves me correct they were grad students at the time) and wasn’t directed work from the PSF or the community. We don’t get to tell volunteers what to work on (unless they ask us), so we have to judge contributions as they come in, not what we’d rather they work on. So in terms of whether we ultimately decide to accept this PEP we don’t really get to decide the effort spent writing/discussing it would be better spent on something else.
We do have some funds that we plan on using to implement this PEP if it is accepted, and perhaps @sumanah or @EWDurbin could better answer this question, those funds were given to us with the understanding we’d use them to implement, among other things, “cryptographic signing” (though it was left open ended what exactly that entailed) so even in that case we have limitations on how we’re able to direct work to be done since part of it needs to be implementing a cryptographic signature scheme for PyPI (part of it is also implementing malicious package detection, but that doesn’t have a PEP because it’s just a new feature of the PyPI code base and doesn’t have ramifications for projects beyond PyPI really).
So AIUI, the current state of things is we’re funding both things, but we need to do both things, or we need to give back (or something? I don’t really know how it works) a portion of the money since we’d ultimately be deciding not to move forward with implementing a feature that satisfies the conditions it was given under.
So in terms of package signing we have 3 options:
- Discuss/refine this PEP it until it’s ready to be accepted and then move forward with it.
- Someone else volunteers to write a competing PEP and we discuss the relative merits of them both and choose between them.
- We do nothing and we let the PSF figure out what the fallout of not implementing package signing would be.
Personally I think we should do either 1 or 2, but unless someone steps forward to do (2) our only real options are 1 and 3 and since I both agree with the idea of getting packaging signing onto PyPI and that TUF is, to me, the best option i’ve seen I’d love for that answer to be (1) (barring someone coming forward to do 2 and coming up with something even better)
PyPI? Not yet, other similar repositories? Yes they have (see for example Rubygems).
There’s also a wider impact to this as well. We currently rely on TLS to ensure that the bits that a user gets is what PyPI is trying to serve them. That works reasonably OK if a user is talking to PyPI itself, but what about mirrors of PyPI? Right now there’s no mechanism in place to ensure that if you ask for “foo-1.0.tar.gz” from a mirror of PyPI, that you’re actually getting what PyPI thinks foo-1.0.tar.gz is. This is a big problem in places like China where access to PyPI is incredibly slow (I’ve had private reports of pip install taking days to complete if you’re not using a mirror there, just due to the bandwidth limitations of trying to go through the great firewall). Currently the only real solution for those people is to either suck it up, or install from a semi random mirror inside of the great firewall. We could try to have the PSF put a presence inside of China, but that only solves the problem for China not for other locations, and specific to China I believe the legalities of doing that are kind of tricky (but that’s a better question for the PSF itself, not me).
Like I said above, this isn’t really related to this particular PEP, unless your argument is this idea isn’t useful and we should only focus on that.