Warrick packaging review update

Posted without personal comment: just in case people here haven’t seen this one, occasional contributor here @Kwpolska has refreshed his review of the Python packaging world: Python Packaging, One Year Later: A Look Back at 2023 in Python Packaging | Chris Warrick

This has gotten some airplay through some of the services that post on recent activities around Python.

Pretty sure everybody’s seen it.

@pradyunsg posted some criticism on Thoughts on the Python packaging ecosystem | Pradyun Gedam (search for “LWN” in the text).

Edit: Ah, no, sorry, it’s a new post from the same author. My bad.

FWIW, @Kwpolska , while much of the first few sections reads as a opinionated but not unfair summary of your perspective of view on the year’s events and many of the dynamics that played out, for good or ill, at one point it takes a sharp turn into borderline conspiratorial territory, which I wanted to address:

Some people tried finding solutions, some people shared their opinions… and then the Discourse moderator decided to protect his PyPA friends from having to read user feedback and locked the thread.

In the interests of full disclosure, I’m a Python (and Spyder) core dev, PEP editor, Discourse mod, docs team member, and volunteer packaging standards and docs contributor, but not a PyPA member, so I cannot speak with absolute authority on this. However, to my knowledge, the particular volunteer moderator involved has no particular connections to the PyPA, PyPUG or any PyPA project (nor, in fact, to the Python team); in fact, while they maintain a number of open source projects, the only packaging-related involvement I see listed on their GitHub profile as of the past year are with PDM.

For the record, I agree with most of the points raised in the final message and found them well-stated (as did numerous others in the community, at least judging from the high number of “likes” on the post). On he other hand, as far as locking the thread is concerned I personally didn’t agree that was either necessary or beneficial, unless there was disruptive or CoC-violating behavior, and in several recent threads a fair few prominent PyPI members have in expressed the viewpoint that threads shouldn’t be locked in cases such as this.

Therefore, I see no basis in fact for the allegation that the thread lock was due to improper collusion, nepotism or conflict of interest on the part of the moderator involved, as opposed to simply being consistent with a third party’s well-established personal moderation style.

3 Likes

The first thought that occurs to me while reading this: why is Rust specifically so popular among people making Python packaging tools? (I mean, above and beyond the fact of it being another language besides Python - I don’t understand why it would have any particular advantages for this application.)

Have you used Rust?

1 Like

I think Rust is popular for these tools because it’s a mainstream programming language with a philosophy around performance, type-safety and concurrency.

It produces static executables easily for multiple platforms, so it solves the bootstrapping problem that interpreted languages like Python, Ruby, Java etc have.

  1. For Python bootstrapping purposes, it seems ideal to not have to first install Python
  2. For resolving and downloading packages you probably want something very fast and that easily enables concurrent downloads
  3. To avoid crashes and vulnerabilities you probably want to avoid C or CPP where possible

So Rust seems ideal there. Other mainstream languages that could fit might be golang or Zig. Both excellent languages too. I think Rust probably benefited from timing and community and perhaps a more sophisticated type system.

In terms of tooling, Rust, Golang and Zig all have many years of wisdom to benefit from since they can design things from scratch. rustup and cargo are certainly a North Star for what modern approaches can look like if you start with an opinionated and complete set of tools for a language that considers the full experience to include packaging systems, not only the language itself.

4 Likes

Is that “personal moderation style” compliant with the rules of this forum, i.e. the PSF Code of Conduct? Is locking a thread due to “personal moderation style” compliant with rules such as Being respectful of differing viewpoints and experiences or Gracefully accepting constructive criticism?

While it might not be caused by a PyPA request specifically, it certainly is an example of bad and overzealous forum moderation, that does not look good considering the important discussion topic.

I have, and while my experiences are quite limited, I found it complicated, requiring the developer to think too hard about the memory model. If the goal is to have static executables, why did so many projects pick Rust, as opposed to e.g. Go, which is more approachable?

Sorry, I wasn’t clear – my question was not directed to you, it was a reply to @kknechtel:

(and I don’t want to debate here over memory models or the merits of any language if they are not related to packaging).

2 Likes

As mentioned, I personally advocate and practice a much more conservative standard for when to lock threads, and believe we as a community should come to consensus on a set of consistent overall guidelines for moderation actions like this (which we currently don’t have, and so it is de-facto at the interpretation of each individual moderator). To that end, I encourage you to share your thoughts on that in the ongoing discussions in Discourse Feedback , such as those I pointed out previously.

Again, I agree it wasn’t the best look in that situation, and I personally advocate and practice a much more conservative standard for when to lock threads. What I was addressing, though, was the claim that it had anything to do with any collusion or nepotism between the particular moderator and PyPA members, or was otherwise improper in an ethical regard.

And to be fair to the moderator involved, as I think their comment as well as the background makes clear, I don’t believe they were acting in bad faith to silence the conversation (that contains 130+ posts detailing and discussing the user feedback) as was alleged, but rather to attempt to move things forward from back and forth discussion to concrete action to actually address the user’s concerns, by way of PR contributions as well as new, more focused threads on the specific issues.

2 Likes