Howto engage Python contributors in the long term?

(Peter Wang) #41

Although this wasn’t formally written as a “question for Steering Council candidates”, I kinda sorta take it as such, since it’s an important topic.

The highest-order bit, for me, is creating a sustainable cadence of development in Python (which is mostly CPython right now, but perhaps won’t always be). This should respect the tradition of quality work done by our community’s volunteer development model. But it should also recognize that we are failing to engage commercial users of the language, and that gap could threaten the long-term growth and health of our language.

I have been somewhat outspoken in my view that we need to form an auxiliary capability to engage commercial development dollars, to fund both the upkeep & maintenance of core Python dev infrastructure as well as to put real velocity behind fixing outstanding bugs, reviewing PRs, and tackling larger “thorny” work that would be hard to drive based solely on relying on volunteers.

Oftentimes it gets repeated that “just give devs money” solves the OSS sustainability problem. I agree somewhat, but I hold that that only works in the small scale. Once the dollars and the level of effort gets to something the size of Python, we need to treat the project and product management of Python-the-language as an intentional, explicitly coordinated group effort. I believe this should and can be done in concert with organic coordination of the volunteer group, and under the charter of the PSF.

The PyData community has suffered “user hypergrowth” harder and faster than most other tribes of Py, as the last three years have seen a global spike in machine learning and AI. Watching the strain put on my friends and colleagues in that community - and watching various different sub-communities try to respond to it - has really changed how I view the relationship between truly sustainable OSS and corporate usage and corporate dollars.

I was never a die-hard FSF-style open source purist, but my main point of learning is that when the world decides to really start using your free beer/free speech/free code, you had better be ready to innovate as hard as your users demand, or else they will embrace, fork, and leave you behind. We’re seeing this now with the fragmentation in core array libraries that big, commercial, open-source deep learning frameworks are imposing on the user community. We’re starting to see it with “proprietarization” of Jupyter by each major cloud vendor. And, I see similar dynamics with big commercial users of Python making private forks of CPython (or making plans to maintain internal forks of 2.x at the end of the year).

These latter examples demonstrate the failure of our community to scale product management, and to apply meaningful dollars/resources towards coherent software development. I don’t mean that statement in a harsh or critical tone, but rather one of self-awareness. Just asking everyone to row harder, when commercial ships pull up alongside with steam boilers, is not a sufficient answer.

I hope that the new Steering Council - whether I’m on it or not - will meaningfully engage on this topic, because I think it’s an existential meta-capability for our language and our community.

(Antoine Pitrou) #42

Richard Stallman would probably shudder at the thought of a “FSF-style open source purist” :wink:

(Peter Wang) #43

I would love to claim that that joke was intentional… :slight_smile:

It does speak to the importance of language. We really can’t use the term “free software” in a vernacular context, because, hey, Facebook is free! Gmail is free! Probably not what the FSF would approve of, though…

Likewise, I would actually argue that “open source” is a misnomer at this point, and we need to quickly bring a new concept or term to life that deeply integrates the spirit of “community-based innovation” which I think people intuit as being part of the soul of “real” OSS. Sole-vendor “open source”, which don’t really accept contributions from community, or would enforce trademark against crowd-based forks, I think really are a green-washing of the original concepts.

(Senthil) #44

Hi @pzwang - I appreciate the various points that you brought up.

Along the lines that you mention, let’s look at Linux Foundation. There are like 4 fellows who are paid full time to develop Linux, but the project management that Linux Foundation brings in seems to help foundation, and the multiple project managers, not the Linux kernel core. The work on Linux core is still being done by corporations and individuals who will benefit by enhancing the kernel to sell their products (cameras / chips etc).

If we ever start thinking about the next wave of commercialized open source for Python, I prefer, it should hopefully be better than Linux Foundation. Not that Linux Foundation is bad, it seems like it’s own entity now with it’s own goal for sustenance and influence.

(Peter Wang) #45

I have chatted briefly with one of the folks from Linux Foundation to see what their general thinking is about this kind of thing. The main thing they bring to the table is the ability to fund-raise from a large number of sources who are already used to cutting checks out to LF. In business parlance, this “access to channel” could be a very good accelerant. But I agree with you, that we should proceed carefully and not be afraid to make our own way if we feel that something does not line up with community principles.

IMO the key things that this auxiliary “commercial dev org” needs to do are:

  • bring in funds that can pay for dev time and project management
  • bring in requirements and desired feature lists from commercial Python users, keeping them engaged and providing them visibility into roadmap
  • fund one or more product managers that can help facilitate design conversations around trade-offs and delivery of various artifacts
  • provide secondary source of funding to the PSF, to diversify conference funds and ensure long-term consistent funding for PyPI

The one thing this dev org should not do is: hijack the process by which the Python language evolves and grows.

And to keep this comment on-topic: I think that having a certain critical mass of paid devs will allow us to address many “source code care & feeding” tasks which in turn greases the wheels for new users to get involved. I feel like many discussions around core Python are, sadly, made under a Stockholm syndrome of assuming that we must always be starved for developer time. So, over the last 6 months as I’ve been contemplating the future of Python, my focus has really gravitated towards this topic of sustainable funding.

(Carol Willing) #46

Let’s tread carefully with anything related to the Linux Foundation. I’m all for finding a sustainable funding vehicle for Python. Yet, in many ways, the LF creates a “pay for play” environment which does not necessarily serve Python’s best interests.

While I agree with you, Peter, in spirit, I’m not convinced that the LF is the vehicle for getting to the desired result.

(Senthil) #47

Just to clarify, Peter did not mention LF. He only mentioned about need for a viable model for real “capital” to create a sustaining organization. He focused on product and project management, twice in his argument that I brought up Linux Foundation, which has many project managers and few fellows, something we do not want to emulate.

Looks like we are all on the same page.

(Peter Wang) #48

Yep, I can definitely understand the concern.

I’m also aware of the broader impact of any sustainability measures we engage in for Python. The success or failure will have a ripple effect into the scipy and pydata communities, as well into other OSS communities that are not firmly rooted in a single commercial entity.

This is one of the reasons I do think that we should be pragmatic in thinking about the fund-raise aspect. If the new vehicle can raise a meaningful amount of funds early, and show some development velocity, it’s a completely different conversation than if it languishes for the first couple of years, limping along.

(Tim Richardson) #49

You appear to making a claim that attempts have been made to remove Linux developers from the project due to the new Linux CoC. Evidence please.

(Giampaolo Rodola) #50

Didn’t claim that (and yeah, I probably wasn’t clear). I said that Linux adopted the same CoC which by author’s own admission is designed to do that.


Improving diversity is a process, not instantaneous. Part of the process involves gaining trust from the underrepresented minority members, making them feel welcomed, included, heard, and accepted.

Simply changing terminologies in codebase will not make the community suddenly more diverse, but it is a good first step. Just because you or the majority of you don’t find the terminology offensive, it does not mean nobody does. A small group of us do find it offensive and off putting. But the minorities might not be able to voice this themselves, and they might not feel like they will have support if they were to bring it up. They’re outnumbered. As a member of minority group myself, it is important for me to see that Python is taking action, making changes, and that Python community do care and want to include and welcome the minorities.

Just because you don’t see immediate change in diversity, it does not mean the effort was not valuable.

(Antoine Pitrou) #52

And why not? Python is a public project, mailing-lists and the bug tracker are open to everyone.

Statistically, roughly everyone is a member of a “minority”, one way or the other. There are so many possible characteristics to choose from…

(Victor Stinner) #53

For example, when a woman speaks out, trolls attack her just because she is a woman. We are not all equal to express our self on Internet. See the recent discussion around Emily nomination (I gave a reference in a previous message).

(Antoine Pitrou) #54

Well, in this case (the “master/slave” discussion), you were attacked even though you’re a man… So, no, I don’t think that really follows.

That was a rather benign “attack” actually. The one being violently attacked was @pablogsal (again, a man).

(Nathaniel J. Smith) #55

Antoine, you know how you sometimes see know-it-alls on python users or whatever, aggressively telling people off based on zero knowledge? That’s what you’re doing here. If you don’t care to learn basic facts like what a minority group is, or how women get different treatment than men (look I googled for you), then that’s your privilege I guess, but if so you need to step out of the conversation, not aggressively question and contradict people with more expertise than you.

This is an ongoing problem and you haven’t responded to gentler nudges, so consider this an official mod warning. Start listening to and respecting your fellow devs or go find a technical topic to talk about instead.

(Victor Stinner) #56

Women in tech are harassed and their skills are challenged every single day of their career. The troll on her nomination is just one example at the top of the iceberg. I only used it because this case is public and happens in a communication channel used by other core developers.

I talked with other women developing in Python who have the imposter syndrome and are told that they should abandon development and choose a work which is more for women… I will not give their name nor their employer because they told me that in private. What would be the benefit for them to put the spotlight on their name as “hello, this woman is harassed”?

I don’t want to talk for Pablo, but I don’t think that he can say the same. When I talked with him, I don’t recall that he was harassed just because he was a man nor that his skills are challenged. Their peers and “people on the Internet” respect him and his skills.

It’s true that Pablo likes to moderate forums like and so is attacked just because of that. Again, it’s very different than being attacker on your gender or your origin.

Your comment is very disrespectful for women and really shows a lack of empathy. Please don’t argue that you are not aware of diversity issues, this thread is full of examples. It’s the purpose of the thread, as my talk…

Note: Women are just one example of minority group. They are others.

(Carol Willing) #57

And why not? Python is a public project, mailing-lists and the bug tracker are open to everyone.

Public, open, and transparent does not equate to being safe, professional, and respectful.

Speaking up is a risk both personally and professionally. Is it worth doing? Sometimes yes and sometimes no.

Much like in the Geosciences, erosion occurs and things wear down. The daily dismissive comments accumulate and have a cumulative impact. Something to consider…

(Steve Dower) #58

You would not believe some of the “traditionally powerful” people I’ve met who won’t speak up in certain forums because of this. And for the most part they’ve reached this point as observers, not even participants, so it shows that any and all interactions can discourage people whether they were directly involved or not.

(Edit: I mean the general “you” wouldn’t believe. Carol personally probably would :slight_smile: )

(Thomas Wouters) #59

It me. I have too much other stuff going on, things I enjoy and things that are more rewarding in other ways, to spend time and effort dealing with things that aggravate or upset me, especially when they’re subtle or possibly not intended that way. Dealing with outright bad things is easy, it’s the little things that do me in. And I’m not even in any underrepresented groups.

(Eddie Elizondo) #60

Given that most of the comments here are from other core-devs, I’d like to share my experience as a recent/new contributor:

The large number of mediums to communicate an idea is a bit daunting. For instance there’s GitHub PR, python-dev, discourse, the bug tracker, and 1:1 emails. Sometimes I get the sense that parallel conversations of the same topic happen in two or three of these different mediums.

Furthermore, having a PR out for weeks at a time without it getting reviewed is sometimes a bit discouraging. That being said, I’m really grateful for all the comments that I’ve received so far in all my PRs and I’m looking forward to be more active in contributions!

I know that solving the issues mentioned here is not trivial. However, we can start with small actions - for instance, what is the biggest source of pain for core-devs that willing contributors can help with? :slight_smile: