# 2024 Python package manager overview **URL:** \ **Category:** Packaging\ **Created:** [July 9, 2024, 8:41am UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711 "2024-07-09T08:41:46Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![h-vetinari](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/h-vetinari/32/1220_2.png) [@h-vetinari](https://discuss.python.org/u/h-vetinari)\ **Post date:** [July 9, 2024, 8:41am UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/1 "2024-07-09T08:41:46Z")
A colleague of mine (does not seem to have a handle here) pointed out the following article, which I think is a pretty nice – even if opinionated – high-level overview over the current state of affairs in python packaging land (at least from the POV of user-facing tools): > **[Python has too many package managers](https://dublog.net/blog/so-many-python-package-managers/)** > > Overview of Python's Package management ecosystem in 2024 The one nitpick I have is that I’d wish the article underscored a bit more is how `pixi` is explicitly trying to bridge the gap between the world of `PyPI`-based packages and `conda` packages, which is (in my biased opinion) one of the coolest things to happen in python packaging in a long time. PS. I’ve found the author on [github](https://github.com/LarsDu), but doesn’t seem to be on DPO either – in any case, thanks! ---
**Author:** ![kknechtel](https://avatars.discourse-cdn.com/v4/letter/k/e47c2d/32.png) [@kknechtel](https://discuss.python.org/u/kknechtel)\ **Post date:** [July 9, 2024, 1:21pm UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/2 "2024-07-09T13:21:37Z")
Curiously enough, one of my friends, who I also don’t know to have a handle here, showed me the same article 🙂 Personally I wish more were written about _how_ we got to this point and _why_ the problems aren’t trivial (including ones that are neatly, almost accidentally sidestepped by others). ---
**Author:** ![jeanas](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/jeanas/32/8115_2.png) [@jeanas](https://discuss.python.org/u/jeanas)\ **Post date:** [July 9, 2024, 1:22pm UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/3 "2024-07-09T13:22:45Z")
I wrote this thing a while ago, but it’s in French: [L’installation et la distribution de paquets Python (1/4) - LinuxFr.org](https://linuxfr.org/news/l-installation-et-la-distribution-de-paquets-python-1-4) / [L'installation et la distribution de paquets Python (2/4) - LinuxFr.org](https://linuxfr.org/news/l-installation-et-la-distribution-de-paquets-python-2-4). ---
**Author:** ![sinoroc](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sinoroc/32/1607_2.png) [@sinoroc](https://discuss.python.org/u/sinoroc)\ **Post date:** [July 9, 2024, 4:35pm UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/4 "2024-07-09T16:35:00Z")
> [@kknechtel](#): > > Personally I wish more were written about _how_ we got to this point and _why_ the problems aren’t trivial There has been some. Let’s link them! The more we link them, the more likely people will find them and read them. The ones I recall: > **[Thoughts on the Python packaging ecosystem](https://pradyunsg.me/blog/2023/01/21/thoughts-on-python-packaging/)** > > My response to the discussion topic posed in Python Packaging Strategy Discussion Part 1 had become quite long, so I decided to move it to write a blog post instead. This post then started absorbing various draft posts I’ve had on this topic since... > **[Stargirl (@stargirl@hachyderm.io)](https://hachyderm.io/@stargirl/109697057391904145)** > > So You Want to Solve Python Packaging: A Practical Guide > > First, the technical: Python is used by vastly different groups of people, some that don't identify as "developers". Those groups often have disparate expectations about how packaging should... ---
**Author:** ![kknechtel](https://avatars.discourse-cdn.com/v4/letter/k/e47c2d/32.png) [@kknechtel](https://discuss.python.org/u/kknechtel)\ **Post date:** [July 9, 2024, 5:19pm UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/5 "2024-07-09T17:19:53Z")
I thought I’d read Pradyun’s post before, but now I’m not so sure. I’ll probably want to dig into it (again). I’m not a huge fan of the threaded-short-message style of discourse (Twitter, Mastodon etc.) generally, and would much rather have had those points as a blog article. But Stargirl’s main points are well taken. I guess it’s worth directly linking the previous PDO thread that got mentioned in the replies there, as well as the other blog post that came up a few times (since in particular, pointing out “how everyone else is doing it” doesn’t explain why - or if - we can’t follow suit, but is a very useful starting point for analysis): > [@Python Packaging Strategy Discussion - Part 1](https://discuss.python.org/t/python-packaging-strategy-discussion-part-1/22420): > > Summary of discussions: We requested the Packaging community for [feedback](https://discuss.python.org/t/rfc-survey-to-help-define-a-python-packaging-vision-and-strategy/15658) on the [proposed survey questions](https://discuss.python.org/t/your-feedback-required-python-packaging-user-survey/18070) The Python Packaging Survey was conducted in Sept-Oct. [The results are summarized here](https://drive.google.com/file/d/1U5d5SiXLVkzDpS0i1dJIA4Hu5Qg704T9/view). We asked maintainers/contributors on which forum they would like to discuss the Packaging Strategy. [The most popular option was Discourse](https://discuss.python.org/t/packaging-vision-and-strategy-next-steps/21513). Discussion cue: One of the common themes from the Packaging Survey was that the Python Packaging landscape is complex. Survey Respondent 1 said: “Unify the mult… > **[How to improve Python packaging, or why fourteen tools are at least tw](https://chriswarrick.com/blog/2023/01/15/how-to-improve-python-packaging/)** > > A journey to the world of Python packaging, a visit to the competition, a hopeful look at the future, and highlights from a disappointing discussion. ---
**Author:** ![barry](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/barry/32/42_2.png) [@barry](https://discuss.python.org/u/barry)\ **Post date:** [July 9, 2024, 6:04pm UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/6 "2024-07-09T18:04:45Z")
> [@kknechtel](#): > > I’ll probably want to dig into it (again). I just (re-read?) it and it’s a great article. Not to take away one iota of the great points Pradyun is making, but I think it underplays the perspective of the myriad of personas that are involved here, which is one of the fundamental reasons why IMHO, “fixing packaging” is so difficult. The article does touch on this under the [On formal UX analysis](https://pradyunsg.me/blog/2023/01/21/thoughts-on-python-packaging/#on-formal-ux-analysis) section. I say this knowing full well that Pradyun totally _does_ understand the complexities of the user cases and varying needs of all those personas. > [@kknechtel](#): > > I’m not a huge fan of the threaded-short-message style of discourse (Twitter, Mastodon etc.) generally, and would much rather have had those points as a blog article. But Stargirl’s main points are well taken. Agreed\[1\]! Quoting from Stargirl’s thread: > You gotta deal with the Python Core team and the steering council. They have consistently abdicated the details of packaging to the community. They aren’t, at this time, very interesting in taking over packaging and telling the community how to manage their dependencies. There are _decades_ of history for why and how we got in the current situation. Past active disinterest is certainly one component of that. While I can’t speak for my SC colleagues, or for the SC officially, I can say that I am personally of the opinion that we’re way past the point where this laissez-faire approach is serving us well. I’m not sure I can say what the shape of a more hands-on attitude toward packaging looks like, but I think we need to recognize that this is causing considerable pain for our users, and it’s our collective responsibility to commit to a compassionate response to making this better, even though it may take another decade of sustained effort to solve. * * * 1. on both counts ---
**Author:** ![sinoroc](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sinoroc/32/1607_2.png) [@sinoroc](https://discuss.python.org/u/sinoroc)\ **Post date:** [July 9, 2024, 6:12pm UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/7 "2024-07-09T18:12:02Z")
@kknechtel I thought you meant to share resources that are written by the makers of Python packaging ecosystem as a counterpoint to the usual rants that keep popping up again and again. I would not recommend re-sharing Chris Warrick’s article, it has too many inaccuracies in my opinion. This kind of articles articles keep getting “viral”, everyone has seen at least a couple of them. It feels to me like they attract too much negativity and are often inaccurate which does not help motivate the volunteers actually doing the work. ---
**Author:** ![kknechtel](https://avatars.discourse-cdn.com/v4/letter/k/e47c2d/32.png) [@kknechtel](https://discuss.python.org/u/kknechtel)\ **Post date:** [July 9, 2024, 7:38pm UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/8 "2024-07-09T19:38:46Z")
> [@sinoroc](#): > > I thought you meant to share resources that are written by the makers of Python packaging ecosystem as a counterpoint to the usual rants that keep popping up again and again. I did, originally. But now I see value in gathering all of these sorts of think-pieces. > [@sinoroc](#): > > would not recommend re-sharing Chris Warrick’s article, it has too many inaccuracies in my opinion. This kind of articles articles keep getting “viral”, everyone has seen at least a couple of them. It feels to me like they attract too much negativity and are often inaccurate which does not help motivate the volunteers actually doing the work. From what I could make of Chris’ post, it isn’t _inaccurate_ (aside from perhaps a biased summary of prior PDO discussion) - it just presents opinions that are less than sympathetic, and fails to anticipate how people might respond if they got what they think they want. (I have more thoughts inspired by what’s said there about venvs in particular; but they belong in a separate thread.) * * * A friend gave me another blog link I like in this context: [https://blog.williammanley.net/2022/02/23/pip-and-cargo-are-not-the-same.html](https://blog.williammanley.net/2022/02/23/pip-and-cargo-are-not-the-same.html) ---
**Author:** ![brettcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/brettcannon/32/34895_2.png) [@brettcannon](https://discuss.python.org/u/brettcannon)\ **Post date:** [July 10, 2024, 9:41pm UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/9 "2024-07-10T21:41:02Z")
My requisite [response to that XKCD comic](https://snarky.ca/deconstructing-xkcd-com-1987/). ---
**Author:** ![ofek](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ofek/32/1033_2.png) [@ofek](https://discuss.python.org/u/ofek)\ **Post date:** [July 12, 2024, 4:48am UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/10 "2024-07-12T04:48:47Z")
There is a condensed, UX representation of that comic at 4:25 🙂 [https://youtu.be/BgxklT94W0I?si=EGqYv7hvziFIfMWi&t=265](https://youtu.be/BgxklT94W0I?si=EGqYv7hvziFIfMWi&t=265) This is why I am adamant about not expanding the capabilities of pip and always push back against ideas to ship a full project manager with every Python distribution. Users should only be downloading a tool like Hatch and this tool will manage everything for you with a clean UX. Although Python has to support many more gnarly use cases compared to Cargo or NPM, there is no reason why everything should not be exposed through a centralized interface that lives independent of a Python installation. ---
**Author:** ![ncoghlan](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ncoghlan/32/14266_2.png) [@ncoghlan](https://discuss.python.org/u/ncoghlan)\ **Post date:** [July 12, 2024, 2:16pm UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/11 "2024-07-12T14:16:14Z")
While I was writing about CPython at the time, [Considering Python's Target Audience | Curious Efficiency](https://www.curiousefficiency.org/posts/2017/10/considering-pythons-target-audience/) also ties into @theacodes’s Mastodon thread about the diversity of use cases that Python’s packaging ecosystem needs to cover. Specifically, the [listed Python use cases](https://www.curiousefficiency.org/posts/2017/10/considering-pythons-target-audience/#use-cases-for-python-s-reference-interpreter) frequently also have different needs and expectations around what they want to get out of a package management tool (ranging from “I don’t want to deal with any dependencies that aren’t already preinstalled” all the way up to “I want an opinionated workflow management tool that tells me how to structure my entire development project rather than just installing the packages it is told to install where it is told to install them”). ---
**Author:** ![BrenBarn](https://avatars.discourse-cdn.com/v4/letter/b/74df32/32.png) [@BrenBarn](https://discuss.python.org/u/BrenBarn)\ **Post date:** [July 29, 2024, 5:57am UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/12 "2024-07-29T05:57:14Z")
Very interesting. Some discussion about these tools is, it seems, at cross purposes because there are multiple base functions of which each tool performs a subset: 1. environment manager 2. package manager 3. project manager Conda is an environment and package manager. pip is a package manager; venv is an environment manager. Poetry seems to be all three, but with the proviso that the project part is required (i.e., it cannot manage environments/packages independent of a defined project). Articles like this talk about each tool without making clear that they differ in which of these tasks they even attempt. I still think it makes most sense to start with an environment manager, because packages and projects can be layered on top of that, but not vice versa. Also I get a kick out of this remark: > The core tradeoff with `conda` is that it attempts to do package metadata the “right” way […] This can be onerous to adapt in a company that already has many internal python packages. This looks to me like the all-too-common tragedy of “this tool’s main/only disadvantage is that everyone isn’t using it already”. ---
**Author:** ![Kwpolska](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/kwpolska/32/10542_2.png) [@Kwpolska](https://discuss.python.org/u/Kwpolska)\ **Post date:** [July 29, 2024, 10:03pm UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/13 "2024-07-29T22:03:09Z")
> [@sinoroc](#): > > I would not recommend re-sharing Chris Warrick’s article, it has too many inaccuracies in my opinion. This kind of articles articles keep getting “viral”, everyone has seen at least a couple of them. It feels to me like they attract too much negativity and are often inaccurate which does not help motivate the volunteers actually doing the work. Instead of “too many inaccuracies”, it would be more constructive to offer advice on what is inaccurate or needs improving. ---
**Author:** ![ncoghlan](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ncoghlan/32/14266_2.png) [@ncoghlan](https://discuss.python.org/u/ncoghlan)\ **Post date:** [July 30, 2024, 3:27am UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/14 "2024-07-30T03:27:03Z")
> [@BrenBarn](#): > > Also I get a kick out of this remark: > > > The core tradeoff with `conda` is that it attempts to do package metadata the “right” way […] This can be onerous to adapt in a company that already has many internal python packages. > > This looks to me like the all-too-common tragedy of “this tool’s main/only disadvantage is that everyone isn’t using it already”. Not quite. [The Python Packaging Ecosystem | Curious Efficiency](https://www.curiousefficiency.org/posts/2016/09/python-packaging-ecosystem/#platform-management-or-plugin-management) goes into more detail, but the gist is that because `conda` is a general purpose cross-platform component management tool, it intrinsically doesn’t play nice with each platform’s _native_ component management, instead introducing an extra independent layer to the software stack. That can be a good or bad thing depending on what you’re doing, but it’s highly unlikely to be a _neutral_ thing (hence the divergence of opinions on `conda`’s utility). ---
**Author:** ![BrenBarn](https://avatars.discourse-cdn.com/v4/letter/b/74df32/32.png) [@BrenBarn](https://discuss.python.org/u/BrenBarn)\ **Post date:** [July 30, 2024, 5:17am UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/15 "2024-07-30T05:17:17Z")
That’s just another way of saying the same thing. 🙂 I’ve seen your article and I get what you’re saying, but I still don’t really agree. I can’t say I know every possible use case, but I think the number of cases where you really _need_ to “bring your own runtime” is much, much smaller than most people think it is. Most of the cases where people think they need that are just cases where they started out that way but would be better off if they hadn’t. ---
**Author:** ![ncoghlan](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/ncoghlan/32/14266_2.png) [@ncoghlan](https://discuss.python.org/u/ncoghlan)\ **Post date:** [July 30, 2024, 5:47am UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/16 "2024-07-30T05:47:55Z")
> [@BrenBarn](#): > > I’ve seen your article and I get what you’re saying, but I still don’t really agree. I can’t say I know every possible use case, but I think the number of cases where you really _need_ to “bring your own runtime” is much, much smaller than most people think it is. Most of the cases where people think they need that are just cases where they started out that way but would be better off if they hadn’t. There’s a very easy trap for software application developers to fall into: believing that all software development is standalone application development. That assumption simply isn’t true (software development is so diverse that there’s no one use case that makes up a majority of deployment scenarios). Folks deploying to cloud platforms should typically use the runtimes that the cloud provider offers. Operating system components should run on the runtimes built into the native packaging system rather than third party ones. Larger applications embedding Python should build their own Python or use a distro intended specifically for the purpose (such as `python-build-standalone`). Embedded processors should run MicroPython rather than a runtime aimed at desktop/server/mobile operating systems. `conda` is genuinely valuable for the use cases it covers \[1\], but that still leaves a vast array of legitimate use cases where it isn’t the right answer (and trying to use it anyway will mean you end up spending up more time try to make it work differently from the way it normally works than it would have taken to build something more suited to the use case from lower level building blocks). * * * 1. as long as you don’t have any non-ASCII characters in [your username](https://github.com/ContinuumIO/anaconda-issues/issues/10595), or work around the problem by installing it to an ASCII-only path ---
**Author:** ![aragilar](https://avatars.discourse-cdn.com/v4/letter/a/c57346/32.png) [@aragilar](https://discuss.python.org/u/aragilar)\ **Post date:** [July 30, 2024, 11:41am UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/17 "2024-07-30T11:41:04Z")
Some specific suggestions/updates (since you asked): - distutils/setuptools used to be able to upload (`python setup.py upload`), but this was removed (or at least deprecated), thereby decoupling uploads from building. - I think setuptools fully supports using pyproject.toml now (and dropping support for metadata in setup.py/setup.cfg would be a major breaking change, so I don’t expect that to happen any time soon) - I’m pretty sure wheel is now completely merged into setuptools, and the package is no longer needed (but that should be checked). However, what I think is the biggest issue is that what you are listing is fairly arbitrary (and hence misses a significant part of the picture): - venv is in the standard library (I suspect you meant virtualenv, which is still a bit odd given it’s a library, and most of the other projects use it underneath) - everything is mixed together, with build-backends and build-frontends conflated (I note that pymeson, scikit-build or pyscons are not listed, nor is build). - There’s no indication of which projects depend on which: virtualenv is used as a library by most of the tools, as is calling out to pip. - There’s also the wider issue of related-to-packaging tools: tox/nox, beeware, cx-Freeze, pyinstaller, and even things like docker, flatpak and more general purpose package managers (of which conda is only one). It also ignores why some solutions are not appropriate for some users, i.e. there’s no context provided, historical or otherwise. ---
**Author:** ![sinoroc](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sinoroc/32/1607_2.png) [@sinoroc](https://discuss.python.org/u/sinoroc)\ **Post date:** [August 9, 2024, 9:55pm UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/18 "2024-08-09T21:55:50Z")
If I recall correctly most of the points I found problematic in your article were addressed in Pradyun’s article. ---
**Author:** ![sinoroc](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sinoroc/32/1607_2.png) [@sinoroc](https://discuss.python.org/u/sinoroc)\ **Post date:** [August 25, 2024, 3:03pm UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/19 "2024-08-25T15:03:05Z")
> [@kknechtel](#): > > Personally I wish more were written about _how_ we got to this point and _why_ the problems aren’t trivial (including ones that are neatly, almost accidentally sidestepped by others). Another one. Seems somewhat accurate, but maybe some jabs here and there that might not be totally warranted (I only did a quick read). > **[What's the deal with setuptools, setup.py, pyproject.toml and wheels?](https://www.bitecode.dev/p/whats-the-deal-with-setuptools-setuppy)** > > Why? Why? Why? Oh, that's why. ---
**Author:** ![naught101](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/naught101/32/23235_2.png) [@naught101](https://discuss.python.org/u/naught101)\ **Post date:** [October 10, 2024, 1:07am UTC](https://discuss.python.org/t/2024-python-package-manager-overview/57711/20 "2024-10-10T01:07:32Z")
This ignores the fact that the python packaging landscape is so confusing that it is _very easy_ for any user to get into a state like the one described in the XKCD comic. Blaming the user is not a great look, when the tools available make it difficult to build a working system across multiple projects. [Next page](https://discuss.python.org/t/2024-python-package-manager-overview/57711.md?page=2)