Pre-Pep: Staged Releases separated from PEP-694

[cjames23 (Cary Hawkins)]

I do not think you missed anything.

@barry

• Where there are overlaps on the “control plane” API, such as publishing, discarding a stage, getting its status, I think we should just use 694’s protocol. Whether that’s exactly and specifically the same protocol or state machine is TBD, but at least the shape and transitions should be identical. That way, clients only need to know one dance.

I am fully onboard with just using 694s protocols here in terms of the control plane.

@barry

• Publishing permissions. Right now, 694 says that if you have upload permissions, you can both upload artifacts and issue a publish request. We could potentially separate those two roles. It’s possible we could say that the “publish” role must be completed via the webUI, for the first release, or every release, although I’d argue that the latter at least would be pretty inconvenient for CI publishing. Either way, this fits in with 694 nicely.

I do not think this needs to be completed through the webUI for every release. I think the primary separation I want to see here is that separating out those two roles to provide some security in layers such that if one or the other role has compromised credentials then it still makes it much harder to get malware through the process.

@barry

• Auto-publish after timeout. IIUC, This is one place where your proposal diverges from 694, since 694’s publish event is always client initiated. Perhaps an index can auto-transition from the open state to the processing or published state when the legacy upload + staged=true + fail-open timeout trips. That would only be allowed for publishing a single artifact using the legacy upload protocol, but it could use the 694 machinery under the hood (and perhaps still expose the same status end points).

This really only differs because I was trying to find a way that says we wont block forever if a security scanner does not come back. So I would read the timeout here as only applying to automated scanners. I was treating no signal after a given window as a positive signal instead of a negative signal.

@barry

• Public visibility of stages. This is a big divergence from 694. If we want to expose the stage base URL in the 691 JSON metadata, to enable scanners (third party, automated, manual) I think that would require an amendment to 694. Perhaps a key saying something like “public=true” so that the unguessable stage URL (and session token) are easily guessed, say from the project+version tuple. PyPI could even enforce this setting for first-upload. Alternatively, PyPI could share these stage tokens only with trusted security partners. Either way, I think it’s important to to preserve 694’s embargo functionality and balance that against some level of security scanning by third parties.

I was more thinking more that PyPi would share stage tokens with trusted security partners. I like the embargo functionality of 694. It fits in with what I am trying to achieve here which is providing a bit more proactive security in layers.

I am more than happy to have this end up as mostly amendments to 694 if we think that is the right path forward. Or still happy to write a separate PEP here for the legacy API portions.