RFC: Survey to help define a Python Packaging Vision and Strategy

smm · May 10, 2022, 5:47pm

As discussed at the PyCon Packaging Summit, we intend to launch a survey to define the Packaging Vision and Strategy. The aim of this exercise is to get an idea of the future direction of the Packaging ecosystem and high level goals.

The proposed plan and objectives are available here- Python Packaging Vision and Strategy- Copy - Google Docs

The first draft of survey questions is available here- Python Packaging Vision and Strategy Survey - Google Docs

Please share your thoughts/feedback about questions that should be added to the survey, new stakeholders that should be consulted and other general feedback.

pf_moore · May 10, 2022, 6:21pm

Who will the survey go to? I’ll be blunt here, the majority of questions on the survey as it stands either make no sense to me, I don’t have an opinion, or they ask about my views on other languages’ packaging systems, which I don’t really have enough experience to comment on.

I’m fine if the survey isn’t intended for me, but on the other hand, we still don’t seem to have had the “Packaging maintainers” consultation, so I’m wondering when I do get to have my say

smm · May 10, 2022, 6:40pm

The survey will go to all Packaging users, steering council and PSF Board. While certain/most questions may make no sense, we still have to consult all possible stakeholders.

I have asked for feedback on pypa-committers which is what I meant by Packaging maintainers initial consultation. We will be scheduling workshops depending on how many maintainers are interested in participating in it. If there isn’t sufficient interest, we will be sending out a survey. This survey will be similar to the one listed above but will include more in-depth questions. Again, we will ask for feedback about the questions before sending out the survey. You will have sufficient opportunities to have your say.

domdfcoding · May 11, 2022, 7:26am

In the plan where it says “Packaging maintainers”, does it mean the maintainers of the packaging project, or maintainers of packaging tools in general (pip, setuptools, flit etc.)?

gunungpw · May 11, 2022, 8:56am

First of all, sorry for my ignorance or for my English I am just a person who follows Python packaging landscape. From my observation, conversation in Python Discourse, between Python Packaging Manager and maintainer of Python packaging tools in general (PyPA and other), seems a bit unclear to me not just in this thread but other. In this example, feedback for survey questions from what I know just have asked in pypa-committer mailing list, here, and emerging from PyCon Packaging Summit who not everyone has the opportunity to attend.

From outsider news about Python Packaging is very hard to find user must see Python Discourse or mailing list. Maybe survey questions can have been asked first to other maintainers of packaging tools like:

Poetry, the discord is active maintainer sometimes help user there.
Hatch, author I think active in Python discourse and GitHub (see thread Hatch 1.0.0 available
PDM, active in GitHub and Support PEP 582 (draft) “good study case”
Setuptools, experimental support for pyproject.toml
Pyflow, flit, Pipenv
Or maybe feedback can be get from Python Discord not the core dev but the community.

Also, many Packaging-related PEP has long drawn conversations maybe Python Packaging Manager can help to address some viewpoint.

smm · May 11, 2022, 8:36pm

The latter.

smm · May 11, 2022, 8:43pm

Thank you for this feedback. At the moment, I am still getting feedback on the survey questions to make sure I have the right set of questions. I will approach the maintainers of the tools you have mentioned to ensure they are included in this exercise.

I will engage with Python Discord to advertise the survey.

Regarding Packaging PEPs, please use the office hour to raise specific issues. More details here- Packaging Project Manager Office Hours

pradyunsg · May 12, 2022, 6:36am

Now and here seems to be a good time/place to provide your inputs.

There was a round of asking for inputs on this on pypa-committers: Mailman 3 Python Packaging Vision and Strategy - PyPA-Committers - python.org – Since not everyone is at PyCon US or in that mailing list, this is intended to get inputs from a broader audience.

pf_moore · May 12, 2022, 10:15am

OK, here goes. My apologies in advance if this seems negative. I feel that this has appeared without much context, so there’s a large sense of “why am I being asked this” about my position here. And maybe my frustration about that lack of context is showing through. But that’s a separate discussion, that has been raised elsewhere, so I’ll say no more here.

Let me start with a very broad question. Why do we^[1] even feel we need a “Vision and Strategy” here? I appreciate that in some sense it’s a good thing to have in the abstract, but we’re (nearly) all volunteers and have limited time to work on packaging, so why is this being prioritised over more specific issues? And if we do decide on an overall strategy, how does that fit with the idea that people should work on what they are interested in? This isn’t a business, so even if we decide on a strategy of (say) “make packaging work better for people who aren’t programmers” then how does that work for contributors who have no interest in educating non-programmers or UX issues?

Looking at the survey itself:

Vision My big problem with this section of the survey is that it is written from the point of view of expecting contributors to write their own suggestions. I think many people, when faced with this, will simply respond “I don’t know” - I know that was my response. I think that someone should come up with a proposal, and offer that for discussion. You’d get much more useful feedback and engagement if you give people something to engage with, rather than just saying “so what do you think?” The “mission statement” does this to an extent, but by putting it in a survey there’s no opportunity for discussion, just a simple “say why you do or don’t like it”.

Contribution isn’t just about making suggestions, it’s also about expressing and discussing a view on proposals, and this feels like it ignores that sort of contribution. Maybe the intention is for that type of discussion to come later. But if so, there’s no context^[2] to this survey to reassure people that they can wait and will get their chance later.

Influences I really have no idea what the heck this is meant to be about. Political factors? This reads like some sort of exam paper on “the social context of open source” rather than a survey The whole tone turned me off here, and I feel completely unmotivated to respond. With the exception that I really want to make the point that “we need more competition in the installer frontend space, having pip be the only option for install/build functionality is unhealthy”, so maybe I have an answer for the 3rd question, at least - but it doesn’t fit well with the “write an essay on the impact over the next 3-5 years” feel of the section.

Comparison with other ecosystems Meh, I see the point, but I have very little familiarity with other languages’ packaging ecosystems, so I can’t really say much. But I do have a problem with the tone here - the repeated implication that Python wants to become “the world’s best Packaging service”. Why? It’s not a competition, I want Python’s packaging to be good, yes, but not if in doing so we make trade-offs that make it “less like Python”. For example, I value Python’s “not corporate” feel^[3], and as a result, I don’t want Python packaging to lose that “feel” in the process of trying to work better for corporate users. But I’ve no idea how to express that in the context of the questions offered here.

Learning from other open-source communities This is fine. I don’t have enough involvement in other open source communities to have any significant input, though, so this is another section I end up just not responding to.

I hope this is useful. As I said, I’m very conscious that it’s mostly very negative - which was why I was reluctant to say anything. That’s why I wish the thought processes that led to this survey had happened in public - it’s much easier to question some of these things before someone’s put a bunch of work into developing a questionnaire, etc. I know that forums like discourse are rubbish for that sort of discussion - but IMO, small groups having private conversations, or individuals working in isolation^[4], are far worse, unless someone makes a significant effort to publicise what’s happening offline (and speaking as someone who’s almost never involved in these types of offline discussion, we don’t have a good track record for doing enough of that, in all honesty).

Whoever “we” are. There’s an even broader question here of who are the consumers of the responses from this survey. There’s no central packaging governance, so who’s going to take action based on the results here? ↩︎
We’re back to this point again. No context is a common theme here… ↩︎
Please don’t read any agenda into this. But Python is fairly unique in having a history of almost entirely volunteer/hobbyist development, and I think that’s important. ↩︎
I have no idea which of the two applies here. ↩︎

smm · May 12, 2022, 8:01pm

Thank you. While I do disagree with parts of it, I do appreciate the level of detail and the useful feedback.

The vision/strategy exercise is meant to facilitate a conversation between various stakeholders to get a general understanding of the direction that Python Packaging is headed towards and as a community do we agree with that direction. It is not meant to impose any restrictions, expectations or requirements on any Project. More importantly, we will not ask any contributor to do something that they do not want to do. The autonomy of a contributor and Project will always be respected. After consulting with the stakeholders, we will collate the data and submit a few recommendations. Whether/how the Projects choose to respond to the recommendations will be left to individual Projects.

Survey questions

To be clear the current survey questions are aimed at end users. I would like to reuse some of the questions for the Steering Council. But I don’t know how they will respond to these questions. If someone has guidance regarding this, please let me know.

Vision

I agree with you that there is more value in a discussion rather than asking people to make suggestions. And I will try to come up with a few suggestions for a vision statement and get respondents to vote on it. But given this is supposed to be a global survey, I cannot get everyone in a room to discuss it. The next best thing is to engage with maintainers with a discussion. Given that maintainers are volunteers, I cannot place too much expectations on their time either. So, I cannot assume that a workshop/discussion will take place. A survey (with questions that maintainers would find useful) may be the only possible solution.

Influences

This is where I politely disagree with you. I can think of one political/legal issue that I expect to have some impact on Packaging. Can I safely assume there are no other factors that will ever affect Packaging? No.

The purpose of this question is to get different perspectives on issues that we don’t have much grasp over or don’t think about that often. I agree the question looks weird and I expect most people will skip that question which is a valid response in itself. We may not find anything useful/actionable in the responses. But that question does not take away anything from the survey.

Comparison with other ecosystems

There is no intention to imply a competition or expectation to cater to corporate users. The questions are aimed at understanding why users would prefer one tool over another. If users find something lacking, it would be good to know what that is. We may collectively decide not to address that issue but I don’t see any harm in knowing about it. I have taken on board the issue about the tone. I have to try and see how to better phrase it. I welcome any suggestions regarding this.

As such exercises do not happen that often, I see it as a way for maintainers to engage with end users and each other to define the future of a space that most people are passionate about deeply.

As there are changes to be made and context to be added, I will delay sending the first survey until the survey is in a better place.

For anyone reading this, if you have any questions/suggestions/reservations about this piece of work and would like to discuss it with me privately for any reason (eg: because you’re uncomfortable sharing your opinions publicly), please feel free to email me- sm [at] pyfound [dot] org. I am happy to schedule a meeting to discuss this too.

brettcannon · May 12, 2022, 8:23pm

As in you want to send the survey to the SC, or you want to do a survey about the SC?

smm · May 12, 2022, 8:26pm

To the council.

brettcannon · May 12, 2022, 8:34pm

Don’t worry about it then. We are just other Python users in this instance, having devolved power around packaging.

pf_moore · May 12, 2022, 9:51pm

Thanks for your reply. It does help me understand the intention here better.

I remain baffled as to why having a discussion on Discourse isn’t an option. Sure, it’s not as good as everyone sitting down together and having a conversation, but as you said that’s not possible, so why not do something suboptimal but at least possible?

That’s a long document! Can you explain what the issue for packaging is, here? I’m curious, not least because I don’t really understand how development of an open source project can be affected. Or are you thinking in terms like “if pip doesn’t do X, then companies subject to the law won’t be able to use pip”?

Are you also open to discussions here^[1]? I really want to have discussions in a public forum, and not via private email/conversations. I would encourage everyone else to have discussions in public as well - with the obvious exception of anything that’s genuinely not for public consumption, of course.

To be clear, I don’t have anything pressing to ask at the moment, so right now this is just so I know for future. ↩︎

pradyunsg · May 12, 2022, 9:59pm

I’m speaking from a position of not-complete information – as I understand it, broadly, the US Government is going to increase their investment into cyber security, digital supply chain security and related things. This could turn into dedicated grants or something of that sort, which would have effects on Python Packaging, since there might be new efforts to increase the security posture and possibly additional funding for such work or generally improving open source software.

brettcannon · May 13, 2022, 7:15pm

I concrete example are SBOMs (Software Bill of Materials), which are a list of software you ship (both directly and indirectly via dependencies). Want to know if you use log4j? SBOMs would make that possible.

smm · May 13, 2022, 10:18pm

You’re right. It did not strike me to use Discourse. We will consider that as an option. Regarding the discussions, there was an error in phrasing at my end. I have edited my original post. Discourse is preferred but if someone wanted to reach out to me directly, they have the option to do so.

My understanding of the executive order and the impact it will have is quite high-level and limited. I will defer to @sumanah’s expert opinion on this.

pradyunsg · May 14, 2022, 1:23pm

Overall, while I think it is valuable to broadly survey our users with the intent of developing a better collective sense of “what stuff should we focus on” (which is what “vision and strategy” mean, AFAICT)… in its current draft, the survey would not help much toward that.

Noting down a few suggestions that I have (every bullet starts with “I think”), some of which I’ve shared with @smm out-of-band already:

Currently, the survey requires too many primarily long-form text responses.

There’s a limit/budget of how many open-text boxes a single survey can reasonably contain; even if we put them at the end and explicitly state that they’re optional (which is a way to make it more likely that folks fill them – since they’ve already “invested” into the survey through the mutliple-choice questions).

Anyone filling the survey will have to spend a decent amount of time providing a response (which means that there will be significantly fewer completed responses, and we’d implicitly select for people with more available “free” time to fill this). Plus, the individuals reviewing/condensing the survey will need read through every single one of them to identify actionable things which is a non-trivial investment of time. It’s very common for such broad-audience surveys to have yes/no, pick one-of-these or rate-out-of-X questions with, optionally, user-provided inputs (think: Python Developer survey or Stack Overflow’s Developer survey). It’s nicer to provide a text box to let the user elaborate on their response to these one-click responses, than to give users an empty box with a question.
It might make sense to use a platform that records partially-filled survey responses, especially if this survey will have multiple long-form text boxes.
The nuance around “here’s a mission statement” followed by “write a vision statement for us” is… not-great. I don’t know what the difference between them is and, honestly, don’t care? If I were filling this survey, this would be the point where I’d drop off.
Asking users to write a vision statement for us, is not a useful exercise. Rather, let’s use this to get a sense of “what do users want” to help guide tooling authors/maintainers toward that demand. That can be something along the lines of “Hey, what broad directions you would like Python packaging to evolve/focus in?”, with a list of options (like stability, documentation improvements, focus on security, aesthetic improvements, usability, debuggability etc) where the user can provide in one-or-more “other” options and a text box after that to allow users to elaborate on their response if they want^[1]. This is similar model to PyPI’s “buy a feature” surveys: PyPI User Research | Nicole Harris
While the influence-related questions are useful, they’ll benefit from a rephrasing to feel less like a question from a social studies paper. See also note about too many long form text boxes.
I think all the ecosystem comparision/learning questions would benefit from a tone-down from superlatives. Those sections seem to be getting at “What are nice things from other packaging ecosystems that Python could borrow?” – and maybe just ask that directly in a single question? We don’t need to have a formal tone and using words like “nice” is fine.
The question of encouraging long-term contributions is… a weird one. I guess asking a large audience about this could result in some bright ideas, but it’s also likely that many ideas suggested there are completely infeasible. Plus, this eats into the budget of long-form text questions.
I don’t think we should use the terms “prioritize” or “service” anywhere – that’s inappropriate phrasing IMO for a volunteer-driven ecosystem. The software that the users get is, borrowing language from MIT license, “PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND”. I think the questions that use those terms should either be rephrased or dropped. See suggestion above about asking for a “broad direction” instead.
for the best-practices question, I’d be surprised if we get an actionable response to that.

I’m imagining a Google form where these two things, the question w/ options and a text-box after are on their own separate page. ↩︎

smm · May 17, 2022, 6:10pm

Duly noted. Thank you for the excellent feedback.

To summarize all the feedback that has been shared so far-

Provide choices instead of asking for long answers. Avoid too many text questions.
Use friendly/informal tone. Avoid using superlatives.
Ask questions that users will feel engaged to answer
Record partial responses
Avoid using the term ‘service’

Loren · May 27, 2022, 5:20pm

Hello! My apologies in advance if this response is unhelpful or misunderstands any of your comment; I’m still very new to the PSF and Python and still getting my bearings–in some areas more than others–so I do not have all the context and nuance of these discussions down.

With that disclaimer , I wanted to share one way a “Vision & Strategy” for packaging (built on stakeholder input) is valuable, from my perspective as the lead on resource development for the PSF about:
What I hope will come out of the survey that will be useful value to me/my department is just that there exist stakeholders with interest in Python packaging who would potentially want to invest resources in packaging work, through the PSF (I am not thinking of any actors specifically here; just generalizing based on what I’ve seen/heard so far). It should make it easier for me to know what kind of funding/projects to seek out when I have better information about what the broader packaging community believes to be valuable / what the community believes the vision and strategy for Python packaging is or should be (i.e. the direction it is going and whether the community agrees with it, per @smm, or “a better collective sense of “what stuff should we focus on”” per @pradyunsg ).

[As an aside, it seems to me that in the best-case scenario, seeking funding for community priorities would help bridge the potential gap you mention between people working on what they are interested in and things people think are important but don’t want to/can’t work on. ]

TL;DR: Having more info on what the community wants and thinks is important would be helpful in seeking out resources for packaging.