Agreed that this needs more details. I’ll think on it, consult with my co-authors, and we’ll propose some more detail. In the meantime, I’m sure we’d welcome concrete proposals.
+1 - it seems like consensus is converging on that.
I remember some discussions among the co-authors about this, but unfortunately, I don’t remember the details. Maybe @Deb or @pradyunsg do. (PEP 13 also doesn’t specify a quorum.) I will put this and your other related suggestions on the list.
Both true, but I’m not sure about the need. The PEP does describe how to change itself, which largely mirrors PEP 13. Obviously we don’t want to allow those whom the document governs to be able to make changes unilaterally.
The SC I think generally operates effectively even without this possibility, although I’m biased and others may disagree! Generally speaking, people can and do put any topic in front of the SC by way of its public tracker, can and do use SC Office Hours for F2F conversations (although not as often as we’d like!), and do conduct polls on DPO both at the request of the SC and on their own. If folks disagree with the SC decisions or priorities, they can (and I think do) make their disagreements known through the voting process, which seems like the right balance. I don’t think the analogy to the EuroPython Society or PSF Bylaws is a convincing one, for me at least. Ultimately the PC (and SC) are technical steering bodies, not directly in charge of legal or financial decisions. Therefore all the regular and established open source communication and influence channels are the way for the membership to help advise the PC are IMO most effective. For really egregious violations of the community’s trust, there are recalls and annual voting to take care of that.
For the SC analogy, it’s the election administrator who compiles the voter rolls, checks for inactivity, sends out the notifications, etc. I expect the same process for the PC elections – after the initial one.
For the initial list, I think it can be done either before or after. I don’t think the list materially affects whether the PEP should be accepted or not, but I don’t feel strongly one way or the other.
I’ll be explicit about one more point, and I think it’s currently covered in the PEP.
I think having several community members (pyOpenSci, Scientific Python, Django, Pallets) as part of the voting entity is essential. These are community experts who see the issues in usability that developers don’t see (since they are close to the project).
I want to make sure that all stakeholders: maintainers, package repositories, package producers (people making packages regularly), and package consumers (people who work with end users who are installing packages and understand current usability issues).
This is an interesting idea, and I’ll make note of it for future thought and discussion.
Something that I think is also really crucial is finding the balance between interoperability standardization, but still encouraging a healthy dose of innovation. If we’ve learned nothing over the decades, it’s that the packaging landscape will continue to evolve in ways we can’t yet even see. We need to solve for the problems we know about and struggle with today, without painting ourselves into too much of a corner for the future, which we probably will anyway because of the unknown unknowns.
+1 for the Returning Officer to manage the initial membership nominations.
PEP 772 says:
The PyPA is expected to work with the Packaging Council to establish a decision making process that governs the technical projects under the PyPA umbrella.
This and other language in PEP 772 regarding the relationship between the PC and the PyPA is left as flexible as possible.
As noted in the abstract, the focus of this PEP is on providing a minimal-but-solid foundation for further governance decisions. The specifics of this relationship would be figured out by the inaugural council.
What none of the PEP authors want to do is hamstring either organization by locking things down too strictly in the governing document. Instead, we fully expect that once the PC is established, one of its first tasks will be to figure out the exact right working model between the bodies.
Since the initial members who are voting in the PC will include these folks it means the PC can’t do the choosing. That either leaves the folks the PEP already outlines as definitely getting to vote as also voting in the additional people before the PC vote or the SC chooses these extra folks. Personally, I say let the folks who definitely get a vote have an initial chance to vote in additional people before the first PC vote (but disclaimer, I fall into that category of people who get to vote upfront thanks to being a PyPA member).
Another way to slice this is let the SC choose the groups that can nominate up to 7 people (i.e. what the PEP currently says are non-profits), and then let the folks who can vote then vote in the people that the groups nominate.
Thanks Brett. Clearly, the initial seeding of membership from the “wider community” is a tricky point. I’m intrigued by the idea of of two rounds of seeding: first, the PyPA members, PWG members, and Interested core team members, then second nominations wider community members.
Something that I think is also really crucial is finding the balance between interoperability standardization, but still encouraging a healthy dose of innovation.
Yes, but sometimes interoperability standardization opens the door for a healthy dose of innovation, like PEP 518 did by decoupling pip and setuptools. In any case, I agree that “commit themselves to follow the interoperability specs” should be understood as a general attitude, not as blindly following rules.
In any case, I agree that “commit themselves to follow the interoperability specs” should be understood as a general attitude, not as blindly following rules
I agree with this but I’m worried that any language at all describing this in a standards document will be misinterpreted as absolutely required.
I think of both of PEP 8, a document that’s very careful to say you shouldn’t be blindly following it, and PEP 20, a text that makes fun of its own “rules”, and how people have misinterpreted them as absolute rules.
This makes me worry that 1) undue burden will be put on maintainers and support staff from misquoting that because they have members on this organization they MUST follow all standards, and 2) companies may be hesitant to interact with this packaging process if they think the text implies they have to commit to specific work at the discretion of this organization.
I personally think the implication of having members join a standards organization is that you’re invested in those standards.
With my Fedora hat on, what’s the status of the downstream Python packagers in this proposal? Are we a part of the “wider community members”? Or are we perceived as being too far from the topics decided by the Council? If the former, would it be possible to include a mention about us in the PEP?
I’m glad to see this proposal, thanks for putting it together. I’m especially happy that “improve Python packaging’s user experience” is front and center as part of the mandate, as I think that’s the bottom line here.
My main reservation is that I’m still unsure how much this will accomplish if the core team does not explicitly buy into the idea that packaging is part of Python. However, I’m also unsure that there’s any way to bring that about, so I think something like this proposal may be the best we can do, and has the potential to build momentum in that direction.
One thing that might help in this direction would be to include something the PEP to the effect that the council would have jurisdiction over packaging documentation in the sense of giving actual recommendations or how-tos about Python packaging. This is perhaps in the penumbra of responsibilities like “improve Python packaging’s user experience” and “coordinate Python packaging efforts”, but I think it’s an important enough dimension of that to be mentioned explicitly. It would be nice to see the document say that the PPC’s mission includes not just helping coordinate efforts among developers of packaging tools, but also helping users actually navigate that space of tools. As I’ve said before, I believe what is ultimately necessary is for the PC to be able to put packaging-related documentation on docs.python.org (not just packaging.python.org), although I suspect that is even more contentious for most people.
Aside from that, I share the concerns mentioned by others about the initial seeding of the voting membership. I think the details of this should be more explicit. For instance, aside from the (important) issues about who gets to nominate members, it’s unclear where the number seven came from in that section or why it’s the right number.
I don’t think that’s a good idea. It would for instance preclude involvement from conda, which I think is the project from which “standard” Python packaging has the most to learn. The notion of “community” should be wide enough to incorporate projects that are operating outside the standards, as this is a useful source of new ideas that can improve packaging.
I agree with that sentiment, but I would like to see actual users added to that list. Not just people who work with users or think about users or care about users (we certainly have many of those ), but actual users, people who use Python packaging tools but do not develop them. That is the perspective that I perceive as underrepresented in current discussions about packaging.
I hope this proposal can move forward and bring about some positive change for Python packaging!
I believe @willingc is mentioning something very important here - maybe even critical (thanks Carol). Many communities end up having very “specific” requirements, having someone within the PC keenly aware of how these communities function and what they need is absolutely fundamental.
Let me give a specific example - I have spent the last 10 years contributing to all sort of machine and deep learning libraries, frameworks and research tools. This community today is facing today some fairly specific challenges from a bunch of different horizons. I would love to contribute that specific experience to the PC.
At the same time, I have no experience understanding what the “Web Oriented” community needs (Django, Flask, FastAPI, etc.) and can’t be expected to provide an informed opinion beside listening to a knowledgeable person able to fairly represent this community and do my best to fairly judge what they need, what they would want, and balance that with the interest of the entire python community at large.
To be more specific, I think it would be detrimental to have too many people from the same viewpoint, horizon, or to have nobody representing a very key aspect of the Python Ecosystem. There’s a delicate balance to strike between the different horizons and interests of the PC members.
How do you envision the core mission of the PC ? What is your representation equilibrium point between package “installers/managers/builders” (pip, uv, pdm, pixi, hatch, etc.), “end users” (people doing tool install <package>) and “library maintainer” (people building the wheels).
I have a feeling that there might be a need of clarification in the mission statement
There’s an inherent bias in our DPO community, the people interacting are mostly people who build the packaging and installer ecosystem at large. I believe it’s important we try to remember why are we doing all of this “packaging/installer stuff”. It’s to serve the needs of all our (often disjointed) communities and their specific needs (at least that’s how I ~ personally ~ see it).
In that sense - at least to me - I would believe someone representing django, flask, fastapi and friends to be equally (if not more) important than a core maintainer of any package installer (trying to avoid to name drop anyone here). Both are important and critical, though to me at the very least - the mission shall be “serving the package builders and ecosystem tools’s users” not the other way around.
From what I read above - I believe the conversation is too focused on packaging/installer tools, and not enough on people actually using these tools.
Unfortunately - to counter point what I just said above - most people of these “communities” are foreign to discuss.python.org and the PEP process at large. They don’t pay attention and don’t care (much?). So there might be an education part “why should you care ?”, engaging these communities to participate and help define the future of how Python package, distribute and manage its softwares.
Maybe one actionable statement: I would like to see the PEP clearly stating what key communities are “critical” to the Python Ecosystem and that we should avoid more than X people from the same “community” (one being “installers”) - how many is too many ? And that a “complete board” should have at minimum 1 member of community A, B, C, etc.
In the same way that I believe it would be wrong to not have anyone representing package installers I believe it would be wrong to have 50% of its members from that “perspective”.
Maybe we should introduce that when “Person X” run for a PC position, they should state which community they intend to represent. That will help to formalize who is associated to what domain and ensure we can’t elect a full board of people representing Django (that would be funny though).
I think the PyCharm Community Survey is a great way to indicate where our users are what communities are the most important for the Python language at large.
I think your use of the abbreviation PC is a problem here - do you mean the packaging council or the packaging community (the people who vote for the council). Because I agree we should have a broad base of expertise in the community, but it’s unrealistic to expect all specialisations to be represented on the council.
I think it’s prefectly reasonable for the community to start as essentially the existing PyPA (plus some obvious additions, as noted in the PEP) and then to grow to include other viewpoints by the process of self-nomination. I would certainly welcome someone coming along and saying “I want to be in the packaging community, I have experience in deep learning and how it is affected by the packaging ecosystem”.
That’s almost certainly one of the first things the packaging council will be working on, I imagine. But I do think it should be handled by the council, and not argued out up front as part of the PEP process. We tried that and got multiple hundreds of posts and no conclusion. Let’s not do that again…
I don’t disagree, but those package maintainers need to be willing to put in the time and effort to be involved. Everyone’s a volunteer here, and we can’t force people to contribute. I do think you’re right that outreach, and attempts to find volunteers from user communities, would be a good use of our resources. The council can provide some direction here, but it will be down to the community (both the formal “packaging community” as defined in the PEP and the wider DPO community) to do the work.
This suggests that by “PC” you do actually mean the council. Frankly, I think there are far too many relevant “communities” to be equally represented by five council members. I can come up with 10 categories without even trying. And the PyCharm survey you quote lists 20. Will we require nominees for the board to be explicitly involved in a minimum of 2 each (probably more, to account for overlaps)? I don’t think that’s practical.
Thanks @barry (and to all those that made supportive comments) – I appreciate it. I don’t know that I have all the right answers, but based on the intent I’m seeing in this thread, it seems fine to me to omit “Non-profit” from that sentence and (optionally) include “open source”, if that’s helpful for the initial seeding (e.g., “Organisations that participate in packaging or working with new packagers in an open source capacity”).
Yeah, this was my other concern: if it was decided that the “wider community members” for the initial seeding should be limited to non-profits, and folks from for-profit organizations were to be voted in after the initial seeding, the latter group might miss out on having a voice in some of the early decision-making. If it ends up being easier to do some kind of more restrictive initial seeding followed by a voting-in of broader members, it’d be nice if we could avoid an ordering that prevents those members from participating from the start.
But such a list is near impossible to create due to Python’s broad appeal and usage. It will also inevitably lead to someone being upset that they were left out, while another group will be upset that their perceived importance or size wasn’t scaled appropriately such that they have more of a say than a smaller one.
I don’t think we can quantify things so much. We can make sure to remind people who choose to nominate folks to think beyond the borders of DPO, but I don’t think we can do more than that without risking getting something fundamentally wrong. One of the two key reasons the last attempt at setting up a packaging council failed was we argued about who would get to vote so much (the other reason was defining the powers of the packaging council, but we basically agreed on that in the end).
My assumption is the initial group is seeded and the next, and only next thing, is voting in more people. After that is voting in the council and starting to do stuff.
(idk why I’m writing this with typo happy thumbs on my phone, but I’m in too deep now)
Same.
I share this sentiment. This is a big part of why this PEP is not trying to fully pin down specifics of how the council should operate.
Instead, it’s setting up the council and making it their responsibility to figure out a bunch of the details that we can defer to them (with relevant checks on their actions, by the voting group). This is something that we have seen working out for Python core developers and it’s well established that we want to have something shaped like that for packaging IMO.
There’s one aspect that I think is important to note here… all package tooling authors are also going to be package maintainers and all package maintainers are going to be package users. It’s a venn diagram of concetric circles with the circles getting many orders of magnitude bigger outward. Like, pulling some numbers out of thin air… There’s 100s of packaging tooling maintainers, 100,000s of package maintainers, and many millions of package users.
I’m expecting that we’d probably end up with a few hundred voters here. Make of that what you will.
As an aside: While the individual subcommunities have specific needs, I’d expect that the ecosystem’s generic tools to not over-specialise for a specific/small subset of the community at the detriment of all others merely because that subset is over-represented in the council.
I disagree that this is something that the PEP should be doing.
This isn’t something that exists for the PSF Board or the Python Steering Council and those groups almost definitionally need to cater to a larger audience than what the Python’s packaging would need to cater to.
(+1 to everything that @brettcannon said above, who said things way better than I could)
I agree with Barry! I’d like to start with a small group, and what level of engagement the council will have with downstream packagers is something I’d like to defer for the council to determine.
+1
I also share this opinion. IIRC, our intent around this was to have an initial proposal to iterate upon, evolving it based on feedback like this.
I don’t remember all the details either.
We do have the automated removal of folks who are not actively voting in elections, something that should keep the quorum in check for the most part.
I’m not sure that removing the quorum requirement is better than, say, a quorum of 25% – the one thing it could avoid is someone starting a vote and getting no interest but the vote still passing. OTOH, I’d be fine with any option we pick here with remove quorum seeming like the approach most of the folks here like.
IIRC, my pedantic concern around letting the council control who votes for them was collusion within the existing council to do something malicious that’d render the voting system ineffective to keep them in check (eg: bias all future votes in their favour, by adding their “friends” as members).
I’m realising that we should probably also have limits on how many folks a council can remove via their emergency-ish powers (which are needed for CoC enforcement and such).
As far as I understand, the place we’re at today is that packaging stuff is not a concern for the core team, but rather of a group that is a peer to them – with the future Packaging Council collaborating with the Steering Council.
I don’t imagine things changing much in the near/medium term future (last I checked, there was no interest in the SC toward having any oversight related to packaging stuff, and very limited interest in the core team for making any major packaging focused changes).
I think we need to rework/change how the initial voting members are picked, and your assumption aligns with how I’d expect it to work (if we go down the multiple rounds of adding people route).
Yea, but I don’t expect that to change because they have the ability to vote. I expect that we’d continue to not have much interest from end users. As I see it, in general, interested users with ability/time to dedicate toward such stuff would end up becoming package maintainers/tooling maintainers.
The work that needs to happen for productively engaging with end users is… Well, someone needs to actively each out and engage with users. We did that as part of UX studies for pip and it was certainly useful. It involved hiring a bunch of relevant experts and also took paid + volunteer time to make it happen.
That’s something I think we’d be better at organising resources/effort for with a council.
For the record, conda is and has been for a while an open source project with an own multi-stakeholder governance organization (full disclosure: I’m one of them) and is a fiscally sponsored project by the NumFOCUS non-profit. More info about the conda ecosystem can be found in our recent blog post if you want to know more.
This is how the inaugural steering council for Python bootstrapped processes. @barry was part of that effort and understands how to implement processes for a new council.
I’m encouraged by this PEP and the intent to make it inclusive of many stakeholders. Let’s take advantage of the interest and momentum to keep moving forward.
Thanks to everyone supporting this effort in the past and now.
), but actual users, people who use Python packaging tools but do not develop them. That is the perspective that I perceive as underrepresented in current discussions about packaging.
This is how the inaugural steering council for Python bootstrapped processes. 