# Python Packaging Strategy Discussion - Part 1

**URL:** <https://discuss.python.org/t/python-packaging-strategy-discussion-part-1/22420>\
**Category:** Packaging\
**Created:** [January 4, 2023, 11:13am UTC](https://discuss.python.org/t/python-packaging-strategy-discussion-part-1/22420 "2023-01-04T11:13:19Z")\
**Posts on this page:** 1\
**Showing post:** 78

<div class="post-metadata">

**Author:** ![rgommers](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/rgommers/32/837_2.png) [@rgommers](https://discuss.python.org/u/rgommers)\
**Post date:** [January 11, 2023, 9:53am UTC](https://discuss.python.org/t/python-packaging-strategy-discussion-part-1/22420/78 "2023-01-11T09:53:10Z")

</div>

I agree with both Oscar’s comment and Steve’s reply. With the minor note that I don’t think _build system_ usage will ever converge. It’s conceivable _build backends_ do though, since they’re a pretty thin layer in between a couple of pyproject.toml hooks and invoking the actual build system. So it’s not inconceivable that, for example, scikit-build-core and meson-python would merge in the future and have a configuration option for whether to use CMake or Meson.

Overall we’re in decent shape here - there’s work to do on build backends and build systems, but nothing in the overall Python packaging design for that is currently blocking or in clear need of changes.

> [@rgommers](#):
>
> > [@steve.dower](#):
> >
> > To end with a single, discussion-worthy question, and bearing in mind that we don’t just set the technology but also _the culture_ of Python packaging: **should we be trying to make each Python user be their own system integrator, supporting the existing integrators, or become the sole integrator ourselves?**
> 
> I like @pf_moore’s answer a lot. The “want to be their own integrator” users are important (and over-represented on this forum), and that should continue to be supported. However, the average user doesn’t want to do this, they want things to work well without too much trouble. So I’d also go for supporting the existing integrators better.
> 
> Like @pradyunsg, I also have more thoughts than fit in a Discourse post - will go write a blog post too:)

Getting back to the big picture strategy discussion, here is that blog post: [Python packaging & workflows - where to next? | Labs](https://labs.quansight.org/blog/python-packaging-where-to-next). It’s an attempt at a comprehensive set of design choices and changes to make yes/no. There’s a long version, and a short version with only the key points. I’ll post that short version below.

The most important design changes for Python packaging to address native code  
issues are:

1. Allow declaring external dependencies in a complete enough fashion in  
`pyproject.toml`: compilers, external libraries, virtual dependencies.
2. Split the three purposes of PyPI: installers (Pip in particular) must not  
install from source by default, and must avoid mixing from source and binary  
features in general. PyPI itself should allow uploading sdist’s for  
redistribution only rather than intended for direct end user use.
3. Implement a new mode for installers: only pure Python (or `-any`) packages  
from PyPI, and everything else from a system package manager.
4. To enable both (1) and (3): name mapping from canonical PyPI names to other names.
5. Implement post-release metadata editing capabilities for PyPI.

Equally important, here are the non-changes and assumptions:

- Users are not, and don’t want to become, system integrators,
- One way of building packages for everything for all Python users is not feasible,
- Major backwards compatibility breaks will be too painful and hard to pull  
off, and hence should be avoided,
- Don’t add GPU or SIMD wheel tags,
- Accept that some of the hardest cases (complex C++ dependencies, hairy native  
dependencies like in the geospatial stack) are not a good fit for PyPI’s  
social model and require a package manager which builds everything in a  
coherent fashion,
- No conda-abi wheels on PyPI, or any other such mixed model.

On the topic of what needs to be unified:

- Aim for uniform _concepts_ (e.g., build backend, environment manager, installer) and a multitude of _implementations_,

- Align the UX between implementations of the same concept to the extent possible,

- Build a single _layered_ workflow tool on top (ala Cargo) that,

---

_[View the full topic](https://discuss.python.org/t/python-packaging-strategy-discussion-part-1/22420)._