Pip download from password protected pypi index - With use of Custom keyring for password Keeps failing with 403 error

Hi there,
I’ve been struggling to resolve this issue. Any help or troubleshooting tip would be great.
Plz feel free to ask any question if clarification is needed.

Following pip download command runs fine - Successfully download “foo” binary distribution.
pip download -d ./ --no-cache-dir --only-binary=:all: --no-deps --index-url https://user:password@NETLOC:443/artifactory/api/pypi/pypi-virtual/simple foo==1.0.66

Problem is when I try to resolve password through my custom keyring implementation -
That means in this case If I run following - Ends up failing with 403 errors.

Failing Command line with 403 response code.
pip download -d ./ --no-cache-dir --only-binary=:all: --no-deps --index-url https://user@NETLOC:443/artifactory/api/pypi/pypi-virtual/simple foo==1.0.66

keyring-provider is set to “import” in pip.conf, which looks like this:
[global]
keyring-provider = import

Also setting values for default-keyring and keyring-path in ~/.config/python_keyring/keyringrc.cfg like this:
[backend]
default-keyring=customkeyring.CustomKeyRing
keyring-path=

Anyone - Any tips / pointers :slight_smile:

Thanks in advance.
AGee

Just to elaborate - Sorry As new user in my original message I could not provide more than two links. Just to elaborate:

With my 2nd pip download command - since I do not use password embedded in the url - I see pip first constructs Basic Authorization header - First without password and upon submitting request - first 401 response is returned - As part of 401 response handler - CustomKeyRing is used to get_password and new request with new Basic Authorization header (with username and password) is constructed and request is resent. And that is where I have problem, this send call [pip/src/pip/_internal/network/auth.py at fe0925b3c00bf8956a0d33408df692ac364217d4 · pypa/pip · GitHub] returns with 403

In the first case when password is embedded in index-url.
I see no problem - I get 200 from this send() call
[pip/src/pip/_vendor/requests/sessions.py at fe0925b3c00bf8956a0d33408df692ac364217d4 · pypa/pip · GitHub]
And prior to this send - obviously I see Authorization header properly constructed properly.

I am still scratching my head - What went wrong with use of keyring and when 401-handler path is invoked. I see new Authorization header is constructed with both username and password (i.e. fetched from my CustomeKeyRing object).

Apparently the PreparedRequest object that is constructed in first good case is same as that updated request object that we update with new Basic Auth header in 401-handler path - but cannot understand why I would end up with 403 after resolving 401 issue in handle_401.

Anyone - Any tips / pointers :slight_smile:

@chrahunt ?

I had moved this thread away from Packaging because typically that is what is recommended for such posts, but after seeing how this post is not getting any attention and seeing how pip’s own documentation page recommends the Packaging Discourse channel for help after all, I move it back.

I would recommend to maybe rephrase your question and getting straight to the point, maybe show the code of your “custom keyring provider”. Also I very much recommend giving a lot of care into formatting your messages the best you can, use all the formatting tools at your disposal, in particular the “code blocks”.

I’m gonna go ahead and update pip’s documentation to instead point to the help section, since that is definitely a better place for such requests. I’d suggest keeping this thread in the help section with the packaging help tag (as we typically do for posts).

Just circling back after more than a week I see - some view to this thread but no response.

In the mean time I discovered some root cause. Just sharing - in case someone bumps on to this thread. One piece of information I missed and did not realize could be related was. My python registry, I was having problem was on JFrog.

I found these two references related and helpful

  • Jira - In this case JFrog Client is zypper - but I noticed same pattern of failure with pip - As pip would first get 401 because pip installer/downloader would send Basic Auth Header without password - Then in handler_401 upon fetching password from keyring, although correct Authorization header is sent - but JFrog would immediately return 403-Forbidden
  • JFrog Help Center - See section “Suggested actions to remediate the issue”.

There’s a lot of discussion around this on the pip tracker - for a starting point, you can look at Do not pre authenticate requests by odormond · Pull Request #12496 · pypa/pip · GitHub, but the discussion is spread over many issues and PRs at this point.

I’m not clear whether this is a quirk of how Artifactory works at their end (we haven’t had reports of this happening for any other index implementation) or if it’s something where there’s an accepted standard that pip can follow. I’m not a HTTP authentication expert, so I can’t say what’s “right” here.

I think the answer is that currently this doesn’t work, but it’s not clear whether pip or Artifactory needs a fix.

1 Like