PEP 755: Implicit namespace policy for PyPI

[ofek (Ofek Lev)]

Since last time, I landed a PR which did the following:

1. Improved wording in the rationale to make it clear that community organizations can reserve namespaces.
2. Used “paid” organization over “corporate” organization to ease understanding for non-native speakers and reduce concern among those who consider the word corporate to have a negative connotation.
3. Removed expectation that grants for communities should be open.
4. Improved language around why reviews for paid organizations are prioritized.
5. Included PEP 541 in the expected teaching process.

@takluyver

the ‘grant removal’ section currently has no mention of clawing back a prefix without the agreement of the owner, even if that’s a company that no longer exists.

I missed this and will include directives to make that easy in the next PR, thanks!

@takluyver

There is no page to view all active namespace grants because this has the potential to leak private information such as upcoming products.

It’s taking some effort not to descend into snark. Companies can block off a prefix and the rest of us don’t even get to see that, because it might spoil their reveal if someone can see the name? What happened to transparency & openness?

Paid organizations I think should have that ability. If you want I can make it optional and opt-in by default but there are operational challenges to exposing such a page. I’ll ask someone to comment on this (if they are able) who works on a similar project to this proposal.

@sinoroc

Feels like the ecosystem is fine to me. It seems to me like it is mostly companies that are uneasy with the current state of things. Maybe I am wrong. I think this proposal could use more references to actual complaints from the community (corporate or not) that this change is necessary.

Done! PEP 752: Implicit namespaces for package repositories - #46 by ofek

@pf_moore

Major cloud providers like Amazon, Google and Microsoft have a common prefix for each feature’s corresponding package

Have they indicated that the current situation is a problem for them? Can representatives participate in this thread to make their case?

I added a buy-in section and am trying to have more people comment.

@pf_moore

For example, say a new product is released for which monitoring would be valuable. It would be reasonable to assume that Datadog would eventually support it as an official integration.

It’s also entirely reasonable to assume that 3rd parties could produce safe, useful integrations. By blocking of the “official” namespace, those 3rd party integrations would have to use unrelated names (unless Datadog creates some form of “contrib” area, which is then just as vulnerable to malicious projects as the current open namespace). So users looking for integrations will get used to the idea of useful integrations having names outside of the Datadog namespace, negating the benefit of having an “official” namespace.

It’s a trade-off. Official packages have (unless mistakes happen) a 0% chance of distributing malicious code whereas unofficial packages do not have the same guarantee. I think the benefit outweighs the minor inconvenience of having to be cautious even for popular unofficial packages.

@pf_moore

In contrast, the PEP says basically nothing about the potential risks of the proposal. It assumes “corporate organisations” are stable entities, and can be trusted to make reasonable use of the feature. While this may be true for the given examples, it ignores (for example) the possibility of ill-conceived startups grabbing a namespace based on speculative plans and VC funding, then burning out or changing direction and leaving a bunch of reservations that (as stated) the PEP has no means of dealing with.

This will be in the next PR, thanks!

@BrenBarn

The gaps in the namespace guarantees are so huge that it seems to me they won’t solve enough of the problem to be useful. One such gap is the grandfathering-in of existing packages, and the other is the “open” namespaces, which will still allow unrestricted uploads. With these gaps, the amount of information that a user can get just by looking at the name of a package foo-bar seems likely to be barely more than it is today.

1. I don’t think the grandfathering-in will be very impactful because the PEP 541 process may be judiciously used in applicable scenarios such as bad actors and unmaintained packages with no users. What remains would dwindle over time.
2. The open namespace concept doesn’t detract from the guarantee because the most common scenario I envision would be a private root grant with an open child such as namespace-contrib.

@BrenBarn

Even if the system was totally useless for small-time package maintainers, as long as it didn’t actively harm them, it might still be a net win if we could just get one or two companies to cough up, because that money could be used to improve the PyPI experience for everyone else in various ways. […] I feel it’s difficult to evaluate a proposal like this without having some kind of concrete estimate of how much money it will bring in relative to how much work it will create.

This is my thinking as well although it will be significantly more than just two companies. I’m trying to have people repeat publicly in this discussion what was spoken about in private but it’s tricky

It’s possible that this gives our community enough funding to attempt work on the explicit namespaces feature in future.