Packaging Project Manager Monthly update- January & February

Here are the updates for the months of January and February-

  • Finalizied PyPI Org. Account project details. Created RfP and performed community outreach to advertise the contractor roles.

  • Created initial drafts of implementation plans for PyPI Org. Account project

  • Further discussion with PEP authors to identify roadblocks in implementation

  • Initial discussions around creating a packaging newsletter

Which PEPs are you referring to?

2 Likes

Thanks for the update. In general, it would be really helpful if we could have a little more detail about the points you mentioned, so the information is more useful and actionable. In particular:

Are these drafts available for the community to view? If not, is there a timeline for that? I recall that the previous update two months ago mentioned

  • Shared development roadmap with the packaging working group. This document is under review and will be shared with the community once the plan is approved.

I’m a little confused—Is this the same document, or a different one? If the latter, has this roadmap been shared with the community? If so, where, and if not, when will that be?

Could you be a little more specific here? Which PEPs and roadblocks?

I understand this is still at a preliminary stage, but could you summarize the outcome of the discussions thus far? Would this be a newsletter just for the package working group, or the packaging community as a whole—and if the latter, how can the community be involved in this process? Is there any consensus would collate and publish the articles, around how often, and in what format?

3 Likes

I’d second this - it would be really useful to have more of the process available publicly. The summaries are useful, but they give no links for people to check if they want additional detail.

When we did the resolver upgrade for pip, we had a number of internal meetings about progress and plans, but these meetings were always minuted and the minutes published on a public wiki, for people to read if they wanted. The minutes probably weren’t very interesting to anyone outside the team, but that’s not the point, it’s the transparency that’s important.

This is a great example of how I’d prefer more of the process to be publicly available. I’m really unclear on what’s going on with this work, and I know of no way to find out:

  • I don’t have a good feel for why it’s being prioritised, I gather that it was the biggest vote on that survey that happened, but there’s a lot I don’t understand about the survey (who was surveyed, why were the specific options included on the survey, etc).
  • I don’t know what projects are expected to be impacted. Is there going to be any work for pip here? Will we be involved in any discussions about the design of the feature? What happens if we don’t want to implement it at all?
  • How much effort is being diverted from other work to address this project? Even if development is going to be done by externally recruited people, there’s still a need for PR reviews, mentoring, etc. Has anyone discussed whether we want to divert work from other tasks?
  • As a paid feature, I’ve seen nothing public about whether Warehouse are happy getting into billing management, and similar. Will we be funding an ongoing “accounts department” for this feature? I wouldn’t feel comfortable if something like that was managed on a volunteer basis :frowning: Also, where is the revenue from organisation account fees going to go? I don’t recall seeing any discussion on that.

Like others, I don’t know what PEPs you’re talking about here, or what roadblocks. Why not have such discussions in public? Often it’s not PEP authors who will be doing implementations, so by talking to PEP authors, you’re not getting at the root of the problem. For example, as author of PEP 643, I’m unaware of any discussions on this beyond an initial comment I made to you in email:

PEP 643 (Metadata for Package Source Distributions) - getting that into various build backends would be really useful, once that’s in progress, pip can start implement the “consumer” end of the interface, but there’s not much point doing that until the data is getting written. So that would initially involve working with backends (in particular flit and setuptools, and maybe poetry).

There’s a couple of other PEPs I mentioned in the same email, and I’ve never heard anything further:

PEP 621 support in setuptools would also be nice - but that’s a purely setuptools issue, other backends like flit already have this.

There’s work going on around this, but I’m not sure what your involvement (if any) is.

PEP 660 - The only outstanding action here that I know of is to get setuptools to implement this.

I don’t know what’s happening with this.

PEP 658 - I’d love to see this implemented. As far as the Warehouse (PyPI) change is concerned, I assume they have it in hand - I’m not a Warehouse developer, so I don’t know. What would be useful here is co-ordinating expectations around how other implementers of the simple index API (devpi, Artifactory, etc) will implement this, so we don’t end up in a situation where users get a degraded experience unless they use PyPI.

My comments remain the same here. I’ve no awareness of what might be happening here. Is this part of the discussions you mentioned?

On vaguely packaging-related, but actually a core PEP, getting some resolution on what’s happening with PEP 582 would be something I’d personally be interested in, but I don’t know if it comes under the “packaging” remit.

I didn’t get any feedback from you on this one. I feel as though this has the potential to make packaging easier for newcomers, at the very least, so I think it’s worth at least having a discussion with the community on whether it’s a reasonable use of your time to look into it.

3 Likes

Just to echo what others have said, regarding our common desire for more specific detail, actionable links/mechanisms for followup and greater transparency and community involvement in these (bi?) monthly updates and well beyond, this is a sentiment I’ve heard elsewhere from other maintainers of critical packaging infrastructure as well. It would seem to me that a modest investment in sharing more with the community would likely greatly benefit all involved.

For what its worth, AFAIK I’ve read everything that @smm has publicly posted over the past few month related to this feature and the lead-up to it, and I was not aware until you (@pf_moore) mentioned it that this feature would be paid-only—did I miss something somewhere (I’m assuming I did)?

I don’t begrudge PyPI from seeking to raise revenue, if it is needed to justify its implementation. However, as a member and maintainer of a volunteer community organization (the Spyder project) which maintains a fair amount of core critical infrastructure in the scientific and broader Python community, but only has a shoestring budget (all of which is devoted to more critical priorities), this is also something I really would have appreciated being made clear and transparent up front, as it dramatically affects my calculus for spending volunteer time supporting this feature.

Also, just to clarify, with regard to

which seems to be one of the areas of greatest interest to us, and indeed has been one of the most critical issues facing the packaging community, this is also something that was requested on the previous update,

to which the response was

Given two months has passed since then, and presumably more since you first reached out to them, are you now able to share at least a summary of your findings? If not, when do you expect to be able to, and what are the main blockers on that?

It’s mentioned in the linked RFC, but I think I’d heard it elsewhere as well. As I understand it, the idea is that it will be available for organisations to pay for, but will also be offered free to open source organisations. But I’m not clear on the details.

1 Like

Ah, thanks @pf_moore . @smm , any clarification you can provide at this point on what is currently envisioned for those aspects (What is the overall pricing model? What are the qualifications for getting it free, and how will they be validated? Etc.) would be much appreciated.

To cross reference things, @smm posted a response to some of the questions here in the most recent update: PyPI Organization Account Project Update

1 Like

Additionally, some of the other questions were at least partially addressed in other related threads posted around the same time, namely

and