Modernising my packages - am I thinking about this all wrong?

Packaging
25

By Laurie O via Discussions on Python.org at 29Mar2022 05:49:

Some content suggestions:

  • Mention that you need setuptools >= 61.0 for pyproject.toml support (maybe a version for each of the backends?)

Bumped. I don’t know enough about other tools to recommend versions;
suggestions welcome (with terse reasons, maybe).

  • Include references to further reading for package data-files, extension modules, testing, CI

I’ll see what I can dig up. This is not meant to be filled with detail -
being swamped with detail in a bazillion separate documents was what led
me to the conclusion that there’s no concise overview, which this is
meant to address.

I came to this wanting to update my ancient setuptools
setup(lots-of-arguments) incantation to modern approaches. And found
myself bogged down in PEPs which were both specific and vague and all
the other documents. I spent hours in that swamp.

My problem was that there was no big picture: what is required, with
what pieces. And links to the specs for the pieces.

  • Rename “Upload Artifacts” to “Build Artifacts” (or similar): the former to me sounds like an action, not a thing

Ok.

  • The default for build is to build an sdist, then use that to build a wheel. To me that sounds more resilient (passing --wheel builds the wheel directly from source, not testing the sdist)

It is; I build that way myself. I wanted to talk about the source and
built distributions separately though, so the incantation is specific to
the topic. I figure someone setting this up will at the least see the
help text for “build”.

How does this improve the packaging story over Packaging Python
Projects — Python Packaging User
Guide?
That seems to provide all the steps you provide, while being more
in-depth and including more steps. Is it that you think it’s too
detailed? Is it because it has mainly setuptools-specific configuration
and examples (I agree: now that setuptools supports pyproject.toml, it
should be updated for generic backends)?

I had a run at the tutorial again after the original post on this topic,
and found it wanting (for me) as before. It does talk a bit about what
it is doing.

It’s ok to bootstrap a single package with a specific tool (setuptools)
in a specific opinionated layout. Which is great for the new packager
with no opinions who just wants their package out the door and on PyPI.

However, I’ve got opinions and my repo is not much like the example. To
make my setup use the modern approach I need to understand what all
the bits are for and where they sit. And that overview is not apparent
to me at the PyPA site.

So this document is supposed to:

  • be short - a layout of the flow to distribute something, describing
    the pieces and their relationships
  • not be a tutorial; there’s a nice tutorial already
  • with some examples but not prescriptive except in the sense of
    prescribing “you need to make some distribution files”

So:

  • it has a point list of the objectives, starting at the author and
    arriving at the end user
  • it goes over each of those points in a little detail after the main
    list, to make them clear
  • it has some references to the places where relevant things are
    specified (PEPs, tools)

My remark above about the PEPs being both specific and vague? 517 and
518 were the ones that particularly gave me that impression. They are
written for people who already understand the larger picture and the
existing ecosystem. I can go to them to find specific information, but
they taught me little as a basis of “what do I need to do?”

This may sound like a litany of complaint, but my core issue is lack of
a concise overview. With an overview I know what needs doing, and what
things do. What the various bits of pyproject.toml are for.

'Soup: This is the one that Kawasaki sent out pictures, that looks so beautiful.
Yanagawa: Yes, everybody says it’s beautiful - but many problems!
'Soup: But you are not part of the design team, you’re just a test rider.
Yanagawa: Yes. I just complain.

Cheers,
Cameron Simpson cs@cskk.id.au

2 Likes