2FA usability on PyPI and with packaging tools

This is already resolved by using Trusted Publishing, see https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/.

In data lunedì 13 novembre 2023 01:43:34 CET, hai scritto:

This is already resolved by using Trusted Publishing, see
https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/

The link seems specific to projects on github.

Does the method work in general?

I agree, this seems to be an issue. While it doesn’t matter to me personally right now - all my projects are on github - I am concerned if we are tying the most secure publishing methods for PyPI to a single source code hosting provider[1]. Apart from giving github a privileged position, we risk a situation where a major issue with github leaves project maintainers with no fallback (we had a similar situation with travis CI in the past, and that was simply the community favourite approach, not something explicitly supported by our infrastructure).


  1. Whether that “tying” is a result of only supporting that provider, or making support for other providers a “build it yourself” approach, isn’t the point - “build it yourself” means people can make mistakes and introduce vulnerabilities. ↩︎

4 Likes

Also, this assumes that people want to automate project creation, which seems a bit weird and overkill.

4 Likes

I am slowly moving from github to codeberg these days. I moved all the small
projects and I am kind of stuck with the bigger ones. But certainly new
projects I’d start on codeberg and not on github.

PyPI currently only supports GitHub since they were one of the first providers that had the necessary functionality to make this possible, however it is based on open standards and we have plans to add additional providers in the future (Google Cloud, ActiveState, GitLab, CircleCI and Cirrus CI are in the works.)

The goal is to make “project creation” (or publishing the first release) happen exactly the same way as publishing additional releases, and not require special steps, workflows, tokens, or interfaces just because it’s the first release.

2 Likes

This seems to be entirely because PyPI does not allow creating a project without pushing a first release (which IMO is a wart, not a feature). Otherwise, people would just push the “Create project” button somewhere on the PyPI Web UI :slight_smile:

4 Likes

This seems to be entirely because PyPI does not allow creating a
project without pushing a first release (which IMO is a wart, not
a feature). Otherwise, people would just push the “Create project”
button somewhere on the PyPI Web UI :slight_smile:

My understanding is that the “feature” of being able to explicitly
create empty projects in Cheeseshop was intentionally omitted in
Warehouse’s design because people were using it to create projects
to which they never pushed any packages, and this new workflow sends
a clear signal that such package name “squatting” is discouraged.

3 Likes

If I could create packages the way I create GitHub repos that I never follow up on, PyPI would be in trouble… :sweat_smile:

2 Likes

Or there could just be a delay after which a project without releases is deleted.

I also don’t understand how this relates to namesquatting. People who want to namesquat popular names can upload empty releases anyway…

1 Like

The extra burden I would assume discourages 99.9% of cases, similar to CAPTCHA.

Well there was never any support for creating empty projects really. The only way to create a project was always to create a release.

Previously you had two options, you could use register to create a metadata only release, or upload to upload an artifact and create a release. We dropped the register command, requiring an artifact to be uploaded to create a new release.

Many people used register to create projects with a zero version, and then would delete the version after, but it wasn’t really fundamentally different than what we have now, except you also need to upload an empty artifact.

7 posts were split to a new topic: How should I GPG sign packages I build?

I don’t think there’s any reason we can’t create a “Create Project” button in the UI.

We basically already have this, but it’s scoped to just trusted publishing, we call it “Pending Publishers”.

With a Pending Publisher, you’re basically registering the intent to publish something using trusted publisher, but you’re not actually registering or claiming the name. Rather the name remains unclaimed until you actually release something with that Pending Publisher, at which point it tries to create the project (and fails if someone else already registered the project in the interim).

So you could image something like that, where you can create a Project scoped API token that is scoped to a pending project, that converts to a real project on first use, and if someone else uses the name in the interim then it fails.

5 Likes

Perhaps I’m missing something. What problem does it solve to acquire an API token in advance, if there’s a risk that it becomes invalid before first use? Where is the intent being “registered”? If that registration doesn’t actually include a name registration or claim, then who is expected to care about the intent, and why?

2 Likes

The point is to limit damage if a token is compromised. If you have to create a global token to upload a new package, then someone who acquires that token can upload unauthorized packages to any project you have permissions on. Creating tokens for as-yet-unpublished projects eliminates this use case that still requires global tokens.

2 Likes

Currently you can either register a “user scoped” token, which can upload to any project that the user has access to or upload new projects OR a “project scoped” token, which can upload to the projects that the user has elected.

Because project scoped tokens encode the actual primary key of the project within the token (to prevent rebinding attacks where someone creates a token, deletes the project, then the project is recreated), you can’t currently create a project scoped token for a token that doesn’t yet exist.

This creates a gap where if you want to create a new project, you can’t use a project scoped token, and are forced to use a user scoped token.

There are of course options for how this could work. We could just create a form that lets you create a fully fledged project with just a web form, which would then allow the user to create an upload token. This has the problem (or well, trade off) that we make name squatting a little bit easier.

Maybe that’s ultimately OK, but it’s a different choice then we made for Trusted Publisher, where instead of a Create Project button, we let people create a Pending Publisher, that registered your intent to create a project with a given trusted publisher, but didn’t actually do it until the first upload. The rough idea I outlined creates a mechanism similar to what we have for Pending Publisher, but for project scoped API tokens.

It would be easier to just make a Create Project form, so there is that benefit.

1 Like

Under this proposal, would they encode something else, then? (I’m assuming you refer to a database table in the Warehouse implementation, and that it’s primary-keyed on something other than simply the project name… ?)

Could it be possible to have a kind of token whose only scope is to allow creating a new project and pushing files for its first release?

3 Likes

There’s no technical reason we couldn’t do that.

Basically our API tokens work by allowing us (or anyone who posses the API token actually!) to append any infinite number of restrictions onto the tokens, and all of the restrictions must be satisfied in order for the token to be valid. The main constraint about these rules that can be appended is that they must only ever restrict what the token can do, they can’t expand it.

So anything you can write in a language of “restrict this token to only be valid when …” you can pretty much do.

In this case, you’d add a restriction that it can only be used for project creation, so if the project already exists, it can’t work.

That would require a little refactoring work in our actual permission checking code because we currently don’t check permissions until we’ve either retrieved the existing project or created a new project, because our permissions are currently of the form of “has permission to do X on Y”, but I don’t think it would be terribly hard.

I’m not sure if that would be better or worse than the idea of being able to make a project scoped token for a project that hasn’t been registered yet. The main practical difference I guess would be the one time nature, whether it could only be used for a new project, or whether it ends up functioning like a regular project scoped token after.

Well I guess the other practical difference is that you could use the “only allow new projects” token as your default token in twine, which then gets overriden for existing projects with project scoped tokens.

2 Likes