TL;DR: I’ve made https://pypitoken.readthedocs.io/en/latest and I think it would fit nicely under PyPA and use it in Warehouse. What do you think ?
For a little time now, I’ve had the opportunity to look at the PyPI API token (a.k.a macaroons) implementation from up close, on multiple aspects:
- Finding security issues
- Working on a PR on warehouse to automatically disable tokens when seen in a public GitHub commit
- Creating (& subsequently fixing) problems in Warehouse around the implementation of those
- Suggesting improvements (1, 2) to those tokens implementations
- and seeing other people’s work go unmerged too
Macaroons were chosen as PyPI tokens for multiple reasons, including that they have a very interesting property: any bearer can add restrictions on a token to limit its scope. This is something that can be done locally by deriving a new macaroon with added Caveats.
This is partly implemented in Warehouse, which can generate user-wide or project-scoped macaroons, but it has never really been communicated that you, as a user, can generate your own project-scoped macaroon from a user-wide macaroon. Also, Warehouse only allows creating single-project macaroons, but the implementation actually makes it possible for a macaroon to be valid for a list of projects.
Finally, a lot of new restrictions were suggested initially and haven’t been implemented yet, I believe due to the complexity of adding them into warehouse as an already big codebase with limited maintainer time.
I realized recently that we would gain a lot to have the macaroon computations be delegated out of Warehouse for the following gains:
- This would help document the different restrictions (caveats) that one can add to a macaroon, and making it much much easier to restrict one’s own macaroon
- We could even implement this in tools such as twine
- This would make the implementation in Warehouse much simpler and less error-prone
- This would make implementing new kinds of restrictions much easier, at least for quite a few of them
- This would ensure that the code for generating, adding caveats and checking macaroons is all consistent: it’s in the same codebase.
So I’ve gone ahead and created pypitoken:
- GitHub - ewjoachim/pypitoken: Creation & manipulation of PyPI tokens
- pypitoken · PyPI
It’s a small & well documented library, that I’ve written with the goal of it being the missing link described above. If you want more details, I encourage you to
crashtest try out the documentation.
I’ll start the work of proposing an integration into Warehouse, but in parallel, I wanted to get opinions, and, if you folks are interested, start discussing about putting this library under PyPA. As a PyPA member, according to PEP 609, I can ask a PyPA committer to put this to vote. It might be interesting to make a first round of feedback first before jumping to that, but if any of the PyPA committers reading this wants to go ahead and start this process, it can work too.
We could wait for this package to be used in Warehouse before deciding to have the PyPA adopt it (or not), or we could do it the other way around. I believe that both actions would help the other one make more sense. If this package isn’t used by Warehouse, it still makes sense to develop it as the client for adding restrictions to tokens and thus it makes sense (to me) to have it under PyPA. And if Warehouse were to add this, while it would make me more comfortable that PyPA controls a package linked to PyPI security, it’s not an obligation (for example, PyMacaroon). So all in all, the 2 choices are not necessarily linked.
Lastly, whether this project is adopted by PyPA or not, if anyone is interested to contribute or help maintain it, please get in touch