Concerns about how we vote on new core developers

Committers
1

Hi all,

Over the last few times we have voted on giving someone commit rights, I have been developing an uneasiness that I mostly do not know how to resolve. So I wanted to bring it up here.

tl;dr Our current approach to “voting” on new committers is a potential deterrent to contribution. We should find a better way.

The Problem

My fundamental concern is that our current process is adding to the barrier-to-entry for aspiring core developers. They may see the description of the candidate’s strengths and weaknesses and feel uncomfortable in continuing any effort to become a core developer. Specifically:

  • they may think “what do I have to go through to become a core developer?”
    • every negative vote in such a public forum is a potential source of embarrassment to the candidate
    • the vote makes subjective evaluations (negative or positive) of the candidate very public (another source of embarrassment)
  • people measure themselves against the descriptions of candidates and may erroneously conclude they aren’t good enough

All those things can be a strong deterrent to an aspiring core developer. The process almost feels like a right-of-passage sort of thing. For whatever reason it feels more public and adversarial than it used to. I realize we want more visibility for folks into the process of becoming a core developer, but I’m concerned that our current process for the final step (voting) is a potential deterrent.

Criteria to Get Commit Rights

What we really care about are the following:

  • do we trust the candidate core developer?
    • have they demonstrated that they really care about Python and its community?
    • do they take the responsibility seriously?
  • do they work well with others?
  • are they willing to seek help when needed?
  • are they willing to be accountable and fix their mistakes?
  • do they respect the time of other volunteers?
  • will giving them commit rights benefit Python?

Things we shouldn’t require, IMHO (expressed as extremes):

  • mastery of our processes
  • technical expertise on anything
  • unlimited time to work on Python
  • an expansive portfolio of past contributions

If we aren’t already, we should be extremely clear (in the devguide) about the criteria.

Favor the Mentor

One problem with our current process is that it operates on the premise that existing committers have informed opinions on viability of the candidate. I believe that most committers actually do not have enough context in order to make a definitive vote. Instead, we should rely more on the position of the person recommending the candidate (typically their mentor). We already trust them as a committer and we can count on the steering council to safeguard against a mentor’s judgement being clouded by over-exuberance. On top of that, any strong objections can be handled privately.

Possible Improvements

Ultimately we want to be inviting to new contributors, so we need to consider that relative to the process. I have some rough ideas on what we can do better, but have no illusions that they are the best approach nor a complete one. :slight_smile:

Here are things I believe we could do to improve the process:

  • drop the voting or make it permanently private (e.g. visible only to core developers)
  • introduce an arbiter (e.g. steering council) for the promotion process, to manage feedback privately and announce result
  • require a post-promotion mentor for every new core developer
  • introduce a fixed probation period for new contributors
  • adjust the promotion process to find a middle-ground between how we used to do it and how we do it now

A possible new process:

  1. existing committer publicly proposes giving commit rights to an active contributor
    • provides brief summary
    • solicits feedback on “voting” thread (see below)
    • strong objections to be sent to steering council privately
    • recognizes probation period
    • accepts responsibility to mentor and monitor during probation period
  2. (maybe) voting thread like now, but non-binding and never made public
    • strong objections sent to steering council privately
    • allows steering council to measure negative response against positive one
  3. steering council communicates with mentor as necessary
  4. after set period of time steering council announces result
    • decline only if any significant negative feedback
    • encourage mentor to communicate feedback with candidate and work with them to address any concerns
  5. mentor announces publicly at end of promotion period

Regardless of the practical outcome of this discussion, the above captures my point of view. Thanks!

8 Likes
2

Those are basically the things that would make a contributor a core developer/maintainer in an open source project. If we shouldn’t require those things, we should give pretty much anyone who managed to get one or two PRs merged into Python commit rights (I’m not really suggesting that - it’s a horrible idea…)

3 Likes
3

Sorry I wasn’t more clear. I did not mean to suggest that the bar is low nor that there be any automatic qualification. Rather, I’m suggesting that we not expect a new contributor to be perfect. :slight_smile: It is more important that a committer considers someone to be a good candidate, enough to propose that they become a core developer.

Regardless, the most important point that I don’t want to get lost is that our current approach to promoting new core developers may be a deterrent to aspiring core contributors.

1 Like
4

There are a lot of different ways to run an open source project, but FWIW, those have never been the things I use to judge whether someone should be promoted to maintainer on the projects I’ve worked on.

The main thing I look for is: can I trust this person to know their own limits? Can they tell the difference between a decision that they’re qualified to make on their own, versus one where they need to pull in someone else to help? If so, then great, that’s all I need!

As a silly hypothetical, if someone is only qualified to review typo fixes, and I trust them to know that, then I’m happy to give them commit rights, because hey, I don’t want to spend my time reviewing typo fixes, and if they stick around then they’ll probably level up and start being able to contribute in other ways too.

And at the other extreme, if someone is brilliant and knowledgeable and has a long history of incredible contributions but has a habit of making snap judgements and failing to ask for help when they need it, then I can’t trust them to work autonomously and I don’t want them to have commit rights.

It’s funny you say that, because the main project I work on these days actually has a bot that grants commit rights to everyone who has a PR merged :-). It works because the vast majority of potential contributors struggle with lack of confidence, not over-confidence, so as long as you’re prepared to notice and respond when someone does overstep, you can get away with assuming most people are trustworthy.

I wouldn’t suggest this for CPython either. I’m just saying that there’s a wide range of workable solutions here. And I do think we currently put too much emphasis on expertise, mastery, and past contributions.

6 Likes
5

(My message below is not specific to Paul nor Joannah, I just took this thread as an opportunity to share my point of view ;-))

Thanks Paul for being honest and sharing your opinion :wink:

Two years ago, I considered that contributors must be very productive in term of merged changes and on the tracker, and maybe also on the mailing list, to be promoted.

I changed my mind. In short, I consider that as soon as someone does anything useful to contribute to Python, this person deserves to become a core developer (in the long term, “when” depends on the commitment and my estimated value of these contributions). It doesn’t have to be code. It can be review, bug triage, useful comments on the bug tracker, mailing list, etc. It can mean helping other contributors to contribute (explain the workflow or whatever). People contribute in very different ways.

For example, Joannah wrote articles to give her feedback on contributing to Python. How to find things to do. How her work is reviewed. Other core devs rarely write such articles, whereas these articles are very valuable (IMHO) to help and motivate other contributors to contribute to CPython. The fact that these articles are not part of the devguide or related to a PSF blog doesn’t matter to me. Publishing articles elsewhere gives more freedom how to write the article, and include a more personal feedback.

People who are contributing for a long time to Python don’t write such documentation anymore, so we need more diversity in term of experience in Python: new contributors are more able to detect which parts of Python workflow are not well documented, and they like to share their experience.

Obviously, my intent is to get a snowball effect: more documentation and more mentoring lead to more contributors who write more documentation and help more people :slight_smile: Contributors naturally help each others.

I consider that we need more people and we cannot afford to be picky on the minimum set of skills.

Well, I know that some other core devs disagree with me on that. And that’s fine. The vote is here to record disagreement and discuss the criterias.

It seems that each core dev has different expectation, and the candidate must have all these combined expectations, which will never happen.

Contributors are usually motivated for 1 year and then move on to another shiny project (or just stop contributing to Python). We have to keep them motivated by giving them more responsibilities which is a way to say them: your contributions are valuable.

If we miss that, we loose contributors and we will only get simple changes from newcomers. Reducing Python technical debt requires more experienced developers.

Technical skills are less important to me than the ability to collaborate, be trusted by others, taking reviews in account (don’t ignore other core devs on purpose), etc.

Reminder: we don’t pay developer, so we have to be extra kind with people giving their time to Python!

In my experience, most contributors don’t do review. Core developers feel more responsible for reviews and so are more active in that area. My theory is that contributors consider that their opinion doesn’t matter. And some core dev confirmed that recently. In my case, I like to see reviews from contributors, it makes me more confident to merge a change. I don’t feel anymore as the only one to be responsible for any future regression :slight_smile:

Contributors tend to think that the only metric to become a core dev is to get merged commits and so they only focus on that. Well, most core dev think like that, so it’s a good stategy to become a core dev :slight_smile:

8 Likes
6

Can we move this note to its own topic? I would like to share some thoughts but do not wish to take focus away from the original purpose of this thread.

7

I have moved @vstinner message and your answer to this topic as it was (in my opinion) very close to the subject of the discussion. Tell me if you would prefer a fresh new topic for the discussion :slight_smile:

2 Likes
8

Thanks @pablogsal.

Two and a half years ago, @vstinner took the time to understand my feelings on core development. I am very grateful that he (and others) did so. Without their support, I would have left the Python community with the perspective that my involvement was unneeded and unvalued.

Core development can’t be easily reduced to one dimension. Reflecting on last week’s core sprint, we bring many people together to create something where the sum is greater than any one person’s contribution. Each of us are human, and we have our own strengths and weaknesses.

As we recommend and discuss the merits of promotion, I urge you to consider that you are human and the person being proposed is human. As humans, we may be biased or inaccurate in our assessment of someone’s readiness (or even our own readiness).

IMHO, metrics should be guidelines not the minimum requirement. Very few of us, if any, would meet the metrics and qualities of an “ideal” core developer. Though, all of us can and do make Python a better language with every contribution whether process-centric, person-centric, or code-centric.

4 Likes
9

Replying to @njs in this topic because I think this is more the locus of discussion about when we should and should not vote +1 / -1 for promotions:

I think this actually makes a case for only giving commit bits to people who need them to do reviews, because if the issue is “we are drowning in unreviewed PRs”, then it makes sense to prioritize giving commit bits to people doing reviews so that they can clear out the backlog.

If we were running this like a company with a fixed “commit bit” budget, I would probably focus on giving the commit bit to people who can help clear out the review backlog, which is people who have domain expertise in “understaffed” domains and experienced maintainers who we can count on to either merge or close PRs. We don’t have any shortage of people who want to participate in the conversation and generally where we do, we fill that gap with domain experts and experienced maintainers who are not core devs (e.g. this year’s language summit included many non-core devs). Generally speaking, you do not need a commit bit in order to gain that sort of experience.

That said, we are not running a company and there is not really a fixed budget for “commit bits”, which is why many of my objections are in the subjunctive. I believe the tension I’m feeling with these core dev votes is that there isn’t any guidance as to how we’re intended to vote. I tend to agree with you about this:

Under that rubric, I would promote Joannah and Karthikeyan and several other triagers today, because I consider commit bits essentially free. Right now what happens is that every vote is on the joint question “What does it take to be a core dev, and does this person have those qualities?” rather than “Here’s what it takes to be a core dev: does this person have those qualifications?” Some people sail through because they already meet nearly everyone’s definition of what a core dev should be and there’s no controversy. Other people qualify under one rubric but not under another, which leads to somewhat uncomfortable discussions and often no clear feedback or guidance for the candidates.

10

Contributors tend to think that way because we keep promoting people who aren’t interested in reviewing PRs. Personally, now I regret I wasted too much time reviewing patches on Rietveld. If I were a contributor in 2019, I’d just try to land a shiny feature and then promote it on Twitter.

11

Honestly, I rarely review pull requests. I only look a changes on few specific areas of Python and changes from some specific contributors. Some core devs rarely do bug triage. IMHO it’s unfair to require from contributors to review a lot of pull requests, and triage a lot of bugs, and produce tons on changes, and etc.

In my experience, contributors becoming core developers naturally review more and more changes as they build their “social network”: meet other (core) developers working on similar areas. Naturally, newly promoted core developers get more and more active, but also feel more confident to dare doing things that they didn’t do previously. Discuss maybe with recently promoted core devs to check how the way they contribute evolved after they have been promoted.

Personally, now I regret I wasted too much time reviewing patches on Rietveld.

You have been promoted in 2014. The promotion process evolved since that time. I don’t think that we should use your own promotion experience to decide for new rules.

When I started to contribute to Python, Brett Cannon contacted one to start discussion promotion. I misunderstood him, I understood that the promotion would be effective quickly. In short, it took me like 2 years to get promoted. I had to fix dozens and dozens of bugs to convince enough core developers to be promoted. Honestly, my own experience was not good, but it doesn’t mean that I would like to ensure that other all candidates have to suffer as I did to deserve to become a core dev :slight_smile:

We are all volunteers, so we cannot force anyone to do something. We can promote good practices. And maybe try to understand why even core devs do so little reviews. Or try to make some tasks more attractive.

There were discussions to try to involve contributors more in the review, advice them to review each others for example.

1 Like
12

As a core developer, I can merge my own PRs without a review by anyone else. I generally don’t like to do this, except for really trivial changes; but as has been pointed out, getting PRs reviewed quickly doesn’t happen very often, because there are a lot of PRs and relatively few reviewers.

If a PR of mine has a requested review and no review has materialized after a while, I sometimes merge the PR anyway, as not doing so would hold up progress in an area of work. While I generally do this when working in my “comfort zone” (logging, venv, py.exe, and documentation), I sometimes work in areas with which I am less familiar (e.g. ctypes, shlex). Even in such cases, a core developer is trusted to commit their PRs without oversight.

Does that mean that after submitting e.g. only documentation improvements and getting them merged (which is a useful contribution) a person can become a core developer and therefore get the right to merge their own PRs, relating to e.g. the CPython core C code or the stdlib, without review by others? Couldn’t that in theory lead to a loss of quality in the things we’re producing, or an increase in technical debt which would have to be paid in the future, even with the best will in the world for all concerned?

4 Likes
13

Hi Vinay,

You raise some good questions.

Does that mean that after submitting e.g. only documentation improvements and getting them merged (which is a useful contribution) a person can become a core developer and therefore get the right to merge their own PRs, relating to e.g. the CPython core C code or the stdlib, without review by others?

I know you value documentation (thank you).

As a project, I would hope that we can get to the point in our culture where it is recognized that many contributors to documentation understand the source code as well as those who prefer writing code.

Couldn’t that in theory lead to a loss of quality in the things we’re producing, or an increase in technical debt which would have to be paid in the future, even with the best will in the world for all concerned?

We have tests, CI, and buildbots to help maintain quality. We have the ability to ask for reviews and notifications when code is merged. Yes, it is possible that a mistake can happen. If it does, the project and its workflow need to be reviewed, and the blame does not rest on one individual.

Overall, I’m not a huge fan of self-merged code. Even a seasoned developer’s code benefits from review by a more junior developer.

2 Likes
14

Carol, thanks for your comments.

Sure, with some caveats. For example, understanding the C core is quite different from understanding the Python code which ships as part of CPython: IMO most Python developers can have a good understanding of the Python code in Python, but fewer would have a good understanding of the C core (I count myself in the latter category, though I would consider myself an experienced C developer - it’s a question of familiarity with the code base, as well as a skill set. Many core developers wouldn’t need to even look at the C code in order to contribute productively to Python).

Tests, CI, buildbots and the review process can indeed help with many things. But they won’t always catch design mistakes - they’re more effective at catching implementation mistakes.

I don’t believe blaming people helps a project. The focus has to be on the deliverables. But one also shouldn’t underestimate the inertia which makes reviewing workflow and processes difficult. Python is now easily big enough to qualify as a sort of oil tanker, with quite a large turning circle!

Agreed - any one can find things to improve in code. But it’s sometimes “unavoidable” to self-merge if one wants to make timely progress in one’s area of interest. Possibly, some tweaks need to be made to processes and different types of developer roles in order to minimise self-merges and better achieve continuous improvement.

1 Like
15

I think there’s some dramatization here. We do not require all those things at once. We set up votes and people give their opinions about candidates. Different core developers have different criteria, so someone looking at it distractingly may get the impression that the entire conjunction of criteria has to be satisfied. But that’s not how it works in practice. People can (and do) get promoted even though they don’t satisfy every core developer’s criteria.

3 Likes
16

Quote of the day :smiley:

Agree with your points too. It’s definitely about tradeoffs.

2 Likes
17

In a sense, yes, that’s always a risk when you trust people. But, this is actually how things work today :slight_smile: I don’t think there’s a single core dev who’s so familiar with CPython that they can safely make unreviewed changes to every single part of it. But it’s ok, because we trust core devs to know their limits. And in practice, those limits are usually fluid – like you said, there are some areas where you’re more confident and some where you’re less confident, but sometimes you might be highly confident in a patch even though it’s in an area you’re generally less familiar with, or want extra eyes on a patch even though it’s in one of the areas where you’re normally the expert. And if someone does make a mistake, well, that’s why we have version control :slight_smile: That’s why we don’t try to make super fine-grained permissions, where different core devs only get commit rights on specific files – trusting people just works better.

So yeah, I’d be totally fine with giving someone commit rights based on solely on their documentation work. (Besides, anyone who’s making meaningful contributions to the Python docs must have some substantial knowledge of programming, or how else could they do it?)

2 Likes
18

By the way, we actually do have some official criteria written down for core developers, in PEP 13:

Python core team members demonstrate:

  • a good grasp of the philosophy of the Python Project
  • a solid track record of being constructive and helpful
  • significant contributions to the project’s goals, in any form
  • willingness to dedicate some time to improving Python

As the project matures, contributions go beyond code. Here’s an incomplete list of areas where contributions may be considered for joining the core team, in no particular order:

  • Working on community management and outreach
  • Providing support on the mailing lists and on IRC
  • Triaging tickets
  • Writing patches (code, docs, or tests)
  • Reviewing patches (code, docs, or tests)
  • Participating in design decisions
  • Providing expertise in a particular domain (security, i18n, etc.)
  • Managing the continuous integration infrastructure
  • Managing the servers (website, tracker, documentation, etc.)
  • Maintaining related projects (alternative interpreters, core infrastructure like packaging, etc.)
  • Creating visual designs

Core team membership acknowledges sustained and valuable efforts that align well with the philosophy and the goals of the Python project.

This is one of the sections that came from Django without many changes, and I don’t know if people really read this part carefully when we were voting on the governance PEPs, so it’s possible we might want to tweak it. But at least for now these are the official guidelines on who deserves to be a core dev.

1 Like
19

Hi everyone. I’m writing this post on behalf of the Steering Council.

The Council’s weekly meeting was today and we briefly discussed the good points raised by many of you. If folks have an interest in starting a working group about core developer promotion, that would be a great next step.

To start a Working Group, we will use a process similar to the PSF. A group of individuals self-organize, create a proposed charter, and submit it to the Steering Council for approval.

Here are examples of a PSF Working Group charter:

3 Likes
20

FYI we have exactly the same discussion at each promotion vote…

Some people are concern that promotion candidates who are not fully ready can reduce the quality of Python, introduce regressions, etc. The candidate is promoted and… nothing. No increase of regression. Python quality doesn’t decrease. No announced “chaos”. IMHO Python is a good as it was before latest promotions, or maybe better, but not worse.

It’s maybe time to look back at previous promotions and realize that we were right to promote them, that they are doing great in Python and making Python better, etc.

Please don’t take my words as “we must promote everyone”. I don’t advice to remove the vote to promote canditates. The vote is the best heuristic to measure if a candidate is ready and to balance different opinions.

2 Likes