I received an email from PyPI which read, in part:
Congratulations! A project you (‘skip’) maintain has been designated as a critical project on PyPI. You can see which project(s) has been designated at Log in · PyPI. .
I logged in and checked. It turns out my old, defunct lockfile project has been deemed critical. It’s not at all clear why. Right at the top of the README I wrote:
Note: This package is deprecated . It is highly preferred that instead of using this code base that instead fasteners or oslo.concurrency is used instead. For any questions or comments or further help needed please email openstack-dev and prefix your email subject with [oslo][pylockfile] (for a faster response).
PyPI determines project eligibility based on download counts derived from PyPI’s public dataset of download statistics. Any project in the top 1% of downloads over the prior 6 months is designated as critical.
Can a project opt-out or become non-critical in any way?
No, once the project has been designated as critical it retains that designation indefinitely.
Thanks. I can’t believe there are actually 10M valid downloads. Does that eliminate web crawlers?
A bunch of Open Stack folks were added as owners several years ago, with the intent that they take it over. That’s the source of the bit at the start of the README. All of them have
in their 2FA column. I have 2FA enabled. I assume they don’t. Can I contact them somehow (I don’t think I have email addresses anymore).
I pulled that number from https://pypistats.org/packages/lockfile, which uses PyPI’s BigQuery stats, which itself doesn’t eliminate much of anything, but makes the user agent data available. I’m not sure what filtering pypistats.org does, but the fact it’s breaking it down by Python version suggests it’s at least filtering to just installers (which are the only user agents we can parse that information out of).
I believe that they should have gotten their own emails, so you should be fine to ignore it. I also think the plan is that uploads to that project will just be rejected if the uploader doesn’t have 2FA, so if the project is no longer getting releases, just ignoring it is probably fine too?
I’m one of the sysadmins who controls the automation utilizing the
“openstackci” account listed there. The individuals listed as
maintainers on lockfile are not current OpenStack contributors
anyway.
No OpenStack projects declare direct dependencies on lockfile since
May 2015, though it appears at least one project has a transitive
dependency on it through some other dependencies (I’m digging now to
determine which one exactly since it would be good to eliminate
that as well).
Well, that’s just it. Other than our CI account and the one human in
that maintainers list who I know was involved in fixing whatever bug
it was that caused an upload to be needed in 2015, I’m not sure
which other maintainers were added by Skip “for OpenStack” so I’m
hesitant to remove any accounts other than the one I control (once
I’m done figuring out and eliminating whatever is still indirectly
depending on lockfile anyway).
I recognized one username and sent him an email. Haven’t heard back yet. @fungi once you figure out the openstackci stuff, feel free to disconnect it. I don’t really monitor discuss.python.org. If you want to chat, skip@python.org works for me.
I’ve found that some OpenStack projects depend on ansible-runner,
which in turn depends on python-daemon, which itself declares a
dependency on lockfile. I’ll need to confer with other contributors
on a way forward, but probably it’s to either help python-daemon
maintainers replace their use of lockfile, or help ansible-runner
maintainers replace their use of python-daemon.
PyPI Stats filters out downloads from mirrors (such as bandersnatch) in an attempt to better reflect end-user downloads:
Looking at lockfile’s relative position, those 10m downloads put it in the top 300, so it’s clearly important to the community:
One option to encourage people stop using it: if you’re still in a position to release, emit a DeprecationWarning on import suggesting the replacements. Or something noisier like a UserWarning.
lockfile has a note in its readme saying it’s deprecated and hasn’t had a release since 2015. I’m not at all sure “tell people not to use it” is a viable strategy for getting marked as “not critical”.
(To be clear, as someone with genuinely critical projects on my list, I’m fine with having 2FA. But I am concerned about the classic xkcd guy who maintains all by himself a single project that loads of people depend on, deciding that the implications of being “critical” are just too much, and abandoning his project altogether…)
Is there a pip flag for users to request failure / warnings e.g. when installing files uploaded without 2FA, or when installing files of non-critical projects, or something like that?
No. Metadata stating when a project is “critical” or when a file has been uploaded using 2FA is not available, so we can’t do that (even if we wanted to, which is definitely not a foregone conclusion).
Though the notification sent to critical project maintainers isn’t
clear on that point. In fact it says this…
“in the coming months maintainers of projects designated as
critical, like yourself, will be required to enable two-factor
authentication on their account in order to add new releases or
otherwise modify a critical project.”
…which very much implies the opposite. Our automation adds new
releases by uploading distributions (via twine). I agree that the
current documentation about PyPI’s 2FA implementation says it’s only
for WebUI interactions, and that’s what we discussed in the previous
threads on the topic, but the mass communication to maintainers
could be construed as stating otherwise.
It will be required for uploads in the future, currently basic auth with password still works even with 2FA enabled. There’s a ticket somewhere on warehouse for it, that bit just hasn’t been done yet because it breaks compatibility so we have to figure out how to manage that.
