2FA usability on PyPI and with packaging tools

Packaging User Feedback
1

Hello,

I have recently tried to upload a package to PyPI with 2FA enabled and would like to share my experience.

More or less at random, I elected to use hatch for this package. However, using hatch publish gave mystifying errors (it doesn’t help that hatch seems to use home-grown mechanisms to store credentials?).

I then switched to twine, which gave me a slightly more helpful error message…

User XXX has two factor auth enabled, an API Token or Trusted Publisher must be used to upload in place of password.

… but only slightly, since it didn’t tell me how to actually use an API token (it also didn’t tell me how to create a token, but I figured that must be done from the PyPI account settings).

I then created an API token, and thought that I could try hatch publish again, just by pasting the API token in the “credentials” prompt instead of my PyPI password (by the way why does it say “credentials” instead of a more explicit “password” or “token”?). Which didn’t work.

Switching to twine again, I copied the snippet from the API token creation results into my .pypirc and manually changed the required information such as the project name (a tedious operation, I have to say). I also noticed that the special username __token__ was used for that purpose instead of my regular username.

Now I can upload using twine upload -r my_project_name ..., which works, but I have to remember to pass -r my_project_name. The project name actually refers to an entry in .pypirc that contains the project-specific API token, and pretends to be a separate repository entry…

However, I still don’t know how I could do all the above using hatch instead (should I have gone with something else from the start?).

I would add that neither twine nor hatch publish ever mention 2FA in their respective command-line helps. The hatch online docs revolve around classic password auth (I admit I haven’t checked the twine online docs). This is unfortunate since 2FA is strongly recommended these days.

Side note: at first I had to create an account-wide API token, since the project I wanted to upload didn’t exist yet. Once I managed to create the project by uploading, I deleted the account-wide API token and created project-specific one. I then had to reconfigure .pypirc a second time…

Side note #2: I just wanted to actually register the project so as to reserve the name. It is obviously not possible (why?), so I had to upload a version and immediately delete it…

4 Likes
2

Just enter __token__ for the user name and the token as password. You don’t need .pypirc, you don’t need to leave hatch.

I just tested this with hatch publish on test.pypi.org and it just worked.

IMO, instead of “Enter your username:”, Hatch should write “Enter your username, or __token__ to use an API token:”. @ofek WDYT?

1 Like
3

I also find it interesting that this is mentioned in the PyPI documentation here, but you were looking for it in the Hatch or twine documentation instead.

1 Like
4

Thanks. Will hatch also remember the per-project token (meaning if I set up another project, it won’t confuse the two)?

5

It doesn’t look like it does, unfortunately.

6

Can you please open a feature request?

7

For the record, it also seems unfortunate that, when 2FA is enabled, you are required to use token authentication for package uploads. Token authentication effectively bypasses 2FA…

2 Likes
8

It turns out an issue was already opened for the 2FA discoverability and usability issues, so I just expanded in a comment there:

As for the ability to remember per-project API tokens, I only use hatch on a single project for now so I’d rather let @jeanas open an issue themselves.

1 Like
9

You’re correct that using API-token based authentication bypasses 2FA, but this is because we need a solution that works in commonly used non-interactive environments (like GitHub Actions) where waiting for user input is not possible. 2FA also primarily helps protect against credential stuffing (reusing leaked username/passwords from other websites) and phishing attacks (WebAuthn only) which are less of a concern for API tokens.

That said, using API tokens without 2FA is still a significant security improvement over username/passwords:

  • API tokens can’t be used to log into PyPI and change settings, only to upload packages;
  • API tokens can be fine-grained and scoped to individual packages, reducing the ‘blast radius’ if they were compromised;
  • API tokens can be automatically detected and revoked if/when they leak.

Finally, what we (PyPI) would really like you to do rather than upload with username/password or API tokens is use Trusted Publishing as alluded to in the original error message, because his removes the need to store long-lived credentials entirely. Currently we only support GitHub Actions, but we plan to add more providers in the future – this is relatively new, so I’d be interested to hear if you think this would work for you!

4 Likes
10

Hmm, I am not planning to add any automation on this package, so it probably wouldn’t help in this case.

11

Not trying to change your decision, but just want to make sure you’re aware: if you’re already using GitHub, it’s fairly easy to do with https://github.com/marketplace/actions/pypi-publish, which will use Trusted Publishing automatically if configured for the project.

1 Like
12

I could see a future where PyPI and twine support interactive authentication, eg Microsoft’s auth code flow

In fact, it seems that’s most of the way there according to PyPI’s docs

13

This is basically what @dustin pointed to above, right?

But it would be nice if there was also a solution for manual uploads, such as:

$ hatch publish --token-auth
API token for project 'project': ...

Your package has been uploaded successfully.
However, it is now pending validation before being published.
Please use the following link for two-factor authentication:
https://pypi.org/2fa-validate?some_magic_args....

or:

$ hatch publish --token-auth
API token for project 'project': ...

Your package has been uploaded successfully.
However, it is now pending validation before being published.
Registered project maintainers will receive a validation link by e-mail,
which they can click to undergo two-factor authentication.
14

This feature is specifically not an interactive flow. It’s also complete. :slight_smile:

What your describing sounds like https://github.com/pypi/warehouse/issues/726.

15

I do not see any mention of two-factor auth there?

16

Is the issue of “needing to create a global token to create a new project” getting forgotten here?

17

This is already resolved by using Trusted Publishing, see https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/.

18

In data lunedì 13 novembre 2023 01:43:34 CET, hai scritto:

This is already resolved by using Trusted Publishing, see
https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/

The link seems specific to projects on github.

Does the method work in general?

19

I agree, this seems to be an issue. While it doesn’t matter to me personally right now - all my projects are on github - I am concerned if we are tying the most secure publishing methods for PyPI to a single source code hosting provider[1]. Apart from giving github a privileged position, we risk a situation where a major issue with github leaves project maintainers with no fallback (we had a similar situation with travis CI in the past, and that was simply the community favourite approach, not something explicitly supported by our infrastructure).

  1. Whether that “tying” is a result of only supporting that provider, or making support for other providers a “build it yourself” approach, isn’t the point - “build it yourself” means people can make mistakes and introduce vulnerabilities. ↩︎

4 Likes
20

Also, this assumes that people want to automate project creation, which seems a bit weird and overkill.

4 Likes