Reposted here so folks are aware of it: https://twitter.com/packagingcon/status/1418544419687518209
I have the impression that various ‘downstream’ ecosystems that repackage code from PyPI either still rely on
setup.py, or write specific build instructions for packages to use Poetry or Flit. So I think it would be useful to spread the word about how we - the people who make packaging tools & standards for Python - envisage things working.
Is anyone already planning something like this? I don’t want to duplicate effort if someone else has already had the idea. @brettcannon @pf_moore @pradyunsg @uranusjr @dustin @pganssle @sdispater @njs (or anyone else :-)?
I’ve submitted a proposal, with the following abstract:
Python packaging has changed a lot in the last few years. New tools such as Poetry and Flit allow creating packages without the traditional
setup.pyfile, and new standards mean that
pyproject.tomlfiles are now the linchpin for building and installing Python modules. The wheel package format, which is somewhat older, has also gained a more central role.
I’ll explain what has changed, including a brief summary of what motivated these changes. Then I’ll discuss how you can use the new standard interfaces and formats, with a focus on people re-packaging Python packages into other distribution systems such as Conda, Spack, or Linux distribution repositories. I’ll introduce the low-level ‘build’ and ‘installer’ tools, and compare them to the more widely used ‘pip install’.
I’ve been pondering on giving an overview on Python packaging, focusing on why we end up where we are and the different approaches Python takes from some other ecosystems. Each topic takes roughly the form of:
- Raises of question why we do something different from everyone else.
- Historical reasons behind the difference.
- The advantage (to Python) of the approach chosen.
- If it’s possible to overcome the disadvantage, and how.
Currently I have 3+1 topics:
- Why is metadata distributed with each artifact (distribution) and can differ within the same version?
- Python packages predate indexes. PEP 508 direct URL is a thing.
- A user does not need a package server to host Python packages.
- Sort of. The community is gradually forming consensus some metadata shouldn’t differ. PEP 658.
- Why does PyPI not automatically builds wheels for every project (besides the cost issue)?
- Python does not have a “support tier” concept like many platforms.
- A platform is “supported” if you can manage to build a Python interpreter on it. Very good for the community to grow.
- Feature detection, but there are always edge cases.
- Why are Python package metadata stored directly beside the package installation, instead of living in a central location?
- That’s the only “official” place for third-party tools to store stuff, and all Python packaging tools are third-party.
- Easy discovery, especially easy to write tools to find them and guarantee the result is correct (if you find the code, you find the metadata).
- Are there real technical downsides…?
- Why does Python packaging do things weird?
- Python packaging contributors and Python contributors have surprisingly little overlap and historically receive relatively little effort.
- Packaging contributors have more freedom doing what we want.
- Overhead when discussing changes in the interpreter. Focus more on standards instead of specific tools to gather more people.
I have not submitted the proposal and honestly am still not entirely sure I want to. Update: I’ve submitted this with the title Python Packaging: Why Don’t You Just…? This page shows the proposal publicly if I’m not mistaken.
If you have a topic along a similar line that I do not cover, feel free to suggest one. I anticipate the conference will probably gather people interested in packaging from all sorts of communities, so the attendee may be interested in hearing why another community doesn’t do things like they do. I’ll also avoid sounding snobby to other communities (e.g. why are Python packages so difficult to compile? Actually they aren’t, but Python is more widely used on more things than most language so you see more exotic stuff).
I am pleased to announce that the talk introducing the floating cheeses (IPFS Python Wheels: Efficient, Secure and Reproducible Repository) has been accepted for PackagingCon. The list of accepted & confirmed talks so far can be viewed on the conference’s website.
I am a bit unhappy that it is using Eventbright, which shares participants data to third parties, to manage participation though.