# Python Packaging Strategy Discussion - Part 1

**URL:** <https://discuss.python.org/t/python-packaging-strategy-discussion-part-1/22420>\
**Category:** Packaging\
**Created:** [January 4, 2023, 11:13am UTC](https://discuss.python.org/t/python-packaging-strategy-discussion-part-1/22420 "2023-01-04T11:13:19Z")\
**Posts on this page:** 1\
**Showing post:** 7

<div class="post-metadata">

**Author:** ![h-vetinari](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/h-vetinari/32/1220_2.png) [@h-vetinari](https://discuss.python.org/u/h-vetinari)\
**Post date:** [January 5, 2023, 12:12am UTC](https://discuss.python.org/t/python-packaging-strategy-discussion-part-1/22420/7 "2023-01-05T00:12:29Z")

</div>

(I’m a regular contributor to conda-forge, a large(ly) parallel ecosystem of python packages that’s used widely in data-, ML- and science-heavy python projects. This does not make me a maintainer of the actual tools – `conda`, `mamba`, etc. – that are the UI for this ecosystem)

@smm, have you seen [this thread](https://discuss.python.org/t/pypackaging-native-content-about-the-state-of-using-native-code-python-packaging/22306) by @rgommers? It introduces a resource page that’s intentionally solution-free to establish a baseline understanding among the various different problems/needs/constraints in the python packaging ecosystem, particularly for those projects involving “native” code (i.e. code wrapped by, but not written in, Python).

> [@ofek](#):
>
> I’m fine with Hatch providing this unified UX since it really is almost there already.

With great respect for Hatch, “almost there already” is a big stretch IMO, given all the problems pointed out with (e.g.) native code. This is not Hatch’s fault (nor responsibility), but we’re (IMHO) emphatically _not_ close to declaring success.

As long as the ecosystem currently being served by conda cannot be folded back into “the One True Way”, we have not actually solved the schisms in python packaging (i.e. everyone can just use the same tool). Note that this is not some zealous attachment to conda as a tool or philosophy, but about not regressing the capabilities that are necessary to solve large classes of problems for the “data science persona” at scale. My impression is that this pragmatism is [shared](https://twitter.com/pwang/status/1095159128790626304) by many if not most in conda-land.

> [@brettcannon](#):
>
> While people bemoan Python’s packaging story as being too complex, it’s flexibility is what helped it become the glue language of the programming world.

Indeed, it is a blessing and a curse, but now we have to deal with it.

> [@pf\_moore](#):
>
> And I’d go further and say that for such a tool to be successful, it will be at least as important to decide what workflows we _won’t_ support, as what we will.

To this point, from my POV, the uncomfortable “math” here is to either:

1. solve most of the problems outlined in [https://pypackaging-native.github.io/](https://pypackaging-native.github.io/) (a gigantic undertaking)
2. define large parts of the data science ecosystem as out of scope (…)

Almost certainly, 2. won’t fly for the SC (who would want to define ~half their user base out of existence), and the wider PyPA community has consistently declared 1. as out of scope (unsurprisingly, given the monumental complexity resp. the available resources).

Both points are understandable for the respective stakeholders, but they are at odds with each other, and (IMO) the fundamental tension underlying the lack of tooling homogeneity.

As painful as 2. looks from a language governance POV, this is effectively what’s happening in various pockets of the ecosystem most affected by these problems (e.g. the [geospatial stack](https://pypackaging-native.github.io/key-issues/native-dependencies/geospatial_stack/)), where installation instructions often uniformly recommend an alternate (non-PyPA) installer, and wheels etc. are not provided.

Hopefully this can be mitigated with things like [PEP 668](https://peps.python.org/pep-0668/) (which would make it less “all-or-nothing” to use other package managers, and could more or less gracefully hand off installation of too-complicated packages from pip/wheels/PyPA to another package manager where necessary), but even achieving that is still a far cry from the “unification” that the survey comments cited in the OP are calling for.

---

_[View the full topic](https://discuss.python.org/t/python-packaging-strategy-discussion-part-1/22420)._