Community Stewardship of Faster CPython

Core Development
1

Microsoft’s support for Faster CPython is cancelled It's been a tough couple of days. Microsoft's support for the Faster… | Mike Droettboom | 45 comments .

I think it was a nice thing while it lasted. Just a recap: Python 3.14 is roughly 20-40% faster than 3.10 (when this project first started). I want to thank everyone who contributed to the project in some form or another — not just those at MS. The impact on the world of computing cannot be overstated.

We have a few options now (non-exhaustive):

  1. Finding another sponsor (unlikely it seems, unless we get an altruistic billionaire).
  2. Shutting down the ongoing experimental strands of work.
  3. Continuing with community stewardship.

As the title suggests, I’m in favour of option 3. If somehow option 1 happens in the next few months, then we can just ignore this post then.

Community Ownership of Faster CPython

While waiting for some other sponsorship to happen, or if that never happens. I propose we transfer Faster CPython to a community-owned model.

The steps are the following:

  1. Form a performance-wg, to replace the weekly Faster CPython syncs with MS and Meta. Meta could also host these if they wish to. Weekly/bi-weely meetings across people working on performance have proven to be extremely useful to resolve conflicts and gather consensus on performance design decisions. This WG has no decision-making power.
  2. Set up PSF-owned benchmarking infra. ARM has already graciously loaned/donated a benchmarking machine (thanks to Diego). We can use that to continue the existing benchmarking work. Thanks also to Mike, Thomas, and Matt the bench_runner infra is open source and has good documentation to set it up.
  3. Explore the next design choices for the JIT. The design choices have to be radically shifted to assuming no corporate sponsorship. This means we prioritise maintainability of the JIT, over peak performance. This takes a backseat. Community is more important right now.

For the performance-wg. I propose we have a community council, responsible for conflict resolution mediation and also the technical direction of the community Faster CPython project. I am modelling this after the typing council (PEP 729). We should have 5 members from varying backgrounds to decentralize power. These backgrounds should ideally represent free threading, the JIT, and various other stakeholders. Note that this WG has no decision-making power, it will simply be a conflict mediation and discussion gathering point. I nominate myself to the initial community wg in the transition period, as I am the longest non-corporate community volunteer in the Faster CPython project (~3 years volunteer, ~6 months or so working on free-threading by Quansight), and I have contributed to the specializing interpreter, the JIT’s optimizer, the tail-calling interpreter, and deferred reference counting (free-threading). I withdraw my self-nomination to the informal WG, and instead nominate Mark Shannon.

I believe it’s possible for this project to be community-led. We have excellent community JIT contributors in the form of Tomas Roun, Savannah, Pieter, and many more. I often look to PyPy for a project that has performance wins, but is led by a smaller (though still very dedicated) community team.

If this proposal receives strong opposition, I will discard the idea. If this proposal garners enough support, I will ask for nominations one three week from now, and we can draft a perf-wg PEP set up informal wg meetings.

TLDR

I propose for the formation of an informal WG that has no decision making power, that is merely just to facilitate ongoing collaboration and discussion among various parties concerned with CPython performance. This WG will succeed the role of Mike Droetboom, who has been an awesome mediator these past few years in Faster CPython.

58 Likes
2

It’s a bit late here, so I’ll keep it short-ish:

While I think community governance of and resources for performance work is valuable, I’m not sure I feel the same urgency to start making these big decisions right now. A lot of us are currently traveling and busy with conference activities, so I think waiting at least a couple of weeks for the dust to settle makes sense. For example, I expect to have a lot more clarity around my own role at Microsoft after the conference is over.

Regarding design decisions, community maintainability (and the active solicitation of community involvement) has always been a primary goal of our work. I’ve made this point repeatedly, from the original Brno discussion, to the Christmas Eve PR poem, to PEP 744, and every time I’ve spoken about the JIT since. This is something I think our project did really well, for precisely the reasons that we’re experiencing now.

So I’m not sure that “radical shifts” in our approach are necessary. I still feel personal ownership over much of the work that’s in-flight, and will continue to support it in my personal role as a core dev. I want people to try setting PYTHON_JIT=1 in 3.14, providing feedback, reporting bugs, and making it possible for others to try the JIT downstream, rather than worrying that things are going to start getting ripped out this week or whatever. Progress will be slower, but in my mind there’s really no need for the roadmap to change too much.

Besides, several people I’ve spoken with at the conference plan on sprinting on the JIT with me next week. The community is alive and well. :slight_smile:

31 Likes
3

Thanks Brandt.

So I’m not sure that “radical shifts” in our approach are necessary.

I wasn’t referring to the JIT as in the machine code generator. I don’t think anything needs to change there. I was referring to the optimizer and tracer. Sorry that it wasn’t clear. The JIT term is now quite overloaded.

I’m not sure I feel the same urgency to start making these big decisions right now . A lot of us are currently traveling and busy with conference activities, so I think waiting at least a couple of weeks for the dust to settle makes sense.

Makes sense. I will defer the decision to 3 weeks from now then.

5 Likes
4

Honestly, even if we don’t change our tracing approach, a lot of that stuff should just be cleaned up anyways (it’s grown quite organically). I don’t think it was ever designed to be that complex, it just sort of turned out that way.

But I suspect you’re hinting at a method JIT. I think either approach probably holds promise, but I don’t think a tracing JIT is inherently unable to be maintained by volunteers or whatever (but I’m aware of recent Ruby changes suggesting the contrary).

Besides, @markshannon is still very much a part of this community. :wink:

5 Likes
5

I was very disappointed to hear that Microsoft has ended its support for the Faster CPython project. That team has done a lot of great work and could have done a lot more.

It makes sense to switch to a more community-driven model for driving performance improvements. I’m not sure I see the argument for a formal “Performance Council”, though. Such a body would add a lot of procedural overhead: we’d have to decide its membership, decide on a procedure for making decisions, etc.

You mention the Typing Council as an example. For typing, there’s a need to create an interoperability spec and to decide on a lot of PEPs with very technical subject matter. I don’t think that holds for the area of performance. A closer analog might be the C API Working Group, but even there, I think there are more contentious decisions to make.

It might be better to stick with an informal group of interested core devs and community members. The SC can make hard decisions if necessary.

15 Likes
6

It might be better to stick with an informal group of interested core devs and community members. The SC can make hard decisions if necessary.

Agreed. I think I phrased it wrongly when I say modelled after the typing council. The perf WG would be conflict mediators, and if needed, it would go to the SC. By conflict mediators, I mean what Mike has been doing all this while for us in the faster cpython syncs (which I’m very grateful for). I have found it extremely helpful to keep communication channels open to discuss issues that some core devs might disagree with. You might already know this, but there has been some contention between the single threaded work and the free-threaded work. Having conflict mediators, even if not deciders, are valuable. I think I need to update the original post to stress the proposed WG would not be a decision-making body, unlike the typing council or C API WG.

I think there is still value in having a group nudge the overall direction of CPython’s perf story though. Before Faster CPython, cpy’s perf story was dependent on specific core devs championing a specific issue. The efforts of Victor Stinner, Inada Naoki-san, Mark Shannon, Tim Peters, Yury Selivanov, Pablo Galindo, Raymond Hettinger, Alyssa Coghlan, Antoine Pitrou (there’s too many to list), are extremely invaluable, and I do not intend to devalue them. There is however, value in coordinating some of the bigger projects among core devs.

But I suspect you’re hinting at a method JIT. I think either approach probably holds promise.

I don’t have super strong opinions leaning either way. However, I think we probably need to spend some time cleaning up our code as you said. I have edited my post to strike out the design changes to the JIT, as I feel getting a community group set up is more important right now.

4 Likes
7

Thanks for bringing up these issues, Ken Jin.

I agree with Brandt that it’s too early to tell what the best way forward is, but I also understand the desire for more clarity—especially for a strong member of the team like yourself and while many are at PyCon and less “online” than usual.

For the sync meeting, next week’s is canceled (to make room for the sprints), but May 28 is still on. That will be a good opportunity to discuss what’s next. I’ve had a few conversations at PyCon about this, and I think it’s possible the focus may change slightly from “performance” to “interpreter core” perhaps. But I think everyone wants to stay engaged with other contributors in the space and have found it useful to have regular synchronous updates on what everyone is doing to see around corners, etc., so I’m confident it will continue in some form. I think about the only change that is certain right now is that we will move off of Teams (but not for the 28th).

As for the benchmarking infrastructure, I coincidentally already set up some time with the SC to discuss ways to make it more of a community resource – not because I saw this coming, but because it felt like its use outside of MS and Meta was reaching a critical mass, and we should make that sustainable. I’m not sure of exactly the best way to fund / sustain that, and there are some security things to work out, but I’m open to suggestions on all those things.

So “watch this space”, I guess? We’ll figure it out.

18 Likes
8

The import SIG is another potential point of comparison - quite reference implementation focused (with the level of activity varying over time based on what proposals were in flight), and mostly tackling arcane technical details so everyone can benefit from the improvements without having to worry about those details themselves.

1 Like
9

I have no objection to a “performance WG” as long as I am the majority of it :wink:

As far as I am concerned, it is business as usual for the next few months, and I don’t see much change beyond that.
Things might slow down a bit, but that’s all. As Brandt says, this project always was driven by the community, not by Microsoft.

30 Likes
10

Hah. Very well, I take it that you’re nominating yourself to it then :). Good to hear that. Community management isn’t my strong suite. I’d like to encourage more contributors to work on the JIT, but beyond the outreach done by Brandt, I’m not sure what more we could emulate.

2 Likes
11

Thanks Ken Jin for thinking about the future (as well as all of your performance work so far).

I’m sorry that Microsoft made the business decision to end the program internally and let go of so many talented individuals. Similar to Google’s earlier decision, Microsoft’s decision is poorly thought out and executed. As companies who are betting their businesses on AI, it’s irrational (and incredibly short sighted) to cut talent driving the evolution of the foundational language for AI.

I agree with others giving it a month or two to let things settle a bit makes good sense. In the meantime, as a community, we should keep focusing on moving forward thoughtfully and respectfully, and good decisions will continue to be made. Microsoft, itself, is not what made the perfomance work successful, it’s the people who have made it a success.

Team CPython can and will continue to do great things together.

36 Likes
12

I will bring up the topic of the possibility of community governance with Mike at the next sync. Then I’ll wait for some time and we can see what the next steps are.

I still stand by the idea of setting up an informal perf wg. I believe there’s some benefit in coordinating the ongoing efforts around CPython’s performance. That can happen with or without corporate sponsorship.

As I’m confident that the project won’t die out now, I withdraw my nomination to the informal perf wg. Instead, I nominate Mark Shannon. If the community chooses to pursue the idea of a perf wg, I will help set it up in the transition period.

19 Likes
13

We had the first meeting 2 weeks ago where we discussed some of these issues.

  1. The meetings are going to continue, but fortnightly, as we don’t have that many interpreter changes as we did about a year ago.
  2. We’re moving to a different meeting hosting provider, one by the PSF.
  3. Mike and Ee are investigating options to move benchmarking infra to the PSF.
13 Likes