PyPI as a Project repository vs. Name registry (a.k.a. PyPI namesquatting, e.g. for Fedora packages)

Hello,
As PyPI has become the de-facto canonical namespace for freely redistributable Python packages, I would like to encourage maintainers of packages in the Fedora Linux distro to synchronize their Distribution Package names with PyPI – effectively, to merge Fedora’s namespace into PyPI’s.

This has not yet been discussed in Fedora; I’m kicking off the discussion there as well. Here, I’d like to discuss the PyPI side of things.

While this discussion is ongoing, the names that Fedora uses but are not yet on PyPI have been kindly blocked by the admins, so that:

  • good faith actors are advised that the name is taken in some sense (though not in the PEP 541 sense), so using the name for a different purpose might confuse users, and
  • bad-faith actors can’t snatch the names that easily.

This will need to be reverted and replaced with a better solution, once we know what that is.
But let me frame the issue a bit more generally.

PyPI as a namespace vs. project repository

I feel that currently, PyPI has a bit of a split identity. Per PEP 541, it tries to be purely a repository of useful projects. However, PEP 518 uses names on PyPI as “markers” in pyproject.toml.
This puts us in a tough spot when we want to use tools that aren’t packaged on PyPI.
(Note: I’m not necessarily arguing that pyproject.toml tools are a good idea, but that reasonable people like PEP 517 authors view PyPI as a namespace.)

Why would a tool not be packaged on PyPI? For example, there bindings to software written in other languages, which use non-Python build systems (e.g. make; make install). One could write a PEP 517 build backend for such software, but in some cases it wouldn’t be very useful: for reasons like non-Python build-time dependencies, building wheels just sometimes doesn’t make sense. In Fedora, pyproject.toml handling could reasonably be implemented with as RPM macros, not a Python library.

It does make sense to have well-known names for projects that aren’t installable from PyPI. For example, If I’m querying system-wide installed from Python, I want to include my system package manager in install_requires. But for that package manager, “installable from PyPI” is more of a box to tick than a useful feature.

Blocking and squatting

There are several legitimate reasons to block certain names on PyPI.

The proper way to do that is to request PyPI admins to mark packages as unregistrable (which they will generally be happy to do, given good reason – but it takes their time). The “guerilla way” is name-squatting: registering an empty or uninstallable package. Note that current regulations prohibit name-squatting.

Some examples:

  • Parts of the Python standard library, e.g. math, are not registrable.
  • The trademark microsoft is currently name-squatted
  • The name ldap is currently name-squatted (by me) for reasons described on the PyPI page
  • The package manager dnf is currently name-squatted; AFAIK the buildsystem it uses doesn’t produce sdists/wheels.

My proposal

Personally, I actually find name squatting more user-friendly than blocking:

  • The explanation can be much more verbose than with marking the package non-registrable
  • It’s self-service: anyone can name-squat (though it’s discouraged), and conflict resolution only needs admin intervention if it escalates (as it does now)

Do you think it would be good to update PEP 541 to allow name-squatting, under some rules, for:

  • Packages installable from elsewhere (like from a Linux distro) that are Python-related (other Python projects might want to depend on them; they are build tools pyproject.toml), and
  • New, non-abandoned packages with a repo before their first release (see my rationale)?

Perhaps a register command could be brought back, so you could upload or edit the description without wasting a version number – but that’s a detail :‍)

Of course, another solution could work as well.

3 Likes

As the owner of the various Microsoft trademark [squatting packages], I’m obviously in favour of squatting for those purposes (we had to use lawyers to get a couple of the names). Being able to explain why a name is unavailable is very useful.

There are quite a few packages that have typosquatted themselves this way, e.g. piptools (which I just happened to run into last night).

It would be nice to be able to register the name and description without a real package, and possibly even add a friendly error to send back to installers requesting the package, but otherwise I’m quite happy with this approach.

2 Likes

I think adding some kind of dedicated support for “owning” a name without actually uploading a release for that would be a reasonable addition. There would be some open questions about how that got represented in our APIs like /simple/ but those are all solvable problems.

I don’t think we would allow you to claim a name for a reason that goes against the spirit of PEP 541 but I also think we can amend PEP 541 if needed, and that the things mentioned that are technically against PEP 541 would be fine to add to it. Roughly we just don’t want people locked out of names that don’t serve some kind of real purpose. Coordination of namespaces with downstream and upstream tooling or for legal reasons all seem like they are real purposes that can be served,

2 Likes