User experience with porting off setup.py

Aside, feel free to ignore:

Sorry to be blunt, but… I did not feel like reading much of the article. It has too much of a “yet another Python packaging rant” vibe to me and that made me stop reading. @indygreg, If your experience was truly that bad, I am sorry for you. It feels like everything that could have gone wrong went wrong. So bad that it is hard to believe. This seems like it frames Python packaging under the worst possible light. We have seen too many of such “Python packaging is bad” articles before. From what I have read I understand that the @indygreg did not aim to write a rant but it still feels like one to me. Immediately we could see how we fall in the same pattern of other users piling on (and the moderators needing to intervene). This is not good in my opinion. I am not sure we are getting much productive output from these articles. We know the documentation is not as good as we would like. Obviously we have no control over the results in the search engines. There is not much new in such articles.

Please, if you can, instead of writing in this thread, go to the repository for the packaging guide, review some issues and pull requests, fix the pages, write new guides. I did try some time ago but all my contributions got stuck for weeks or months so I gave up, while at the same time the people with permissions were active here stuck in endless rehashings in threads just like this one, it was disheartening. Since a few weeks there is a new active contributor on the guide so things are better which is absolutely great, but my window of opportunity for this is gone. StackOverflow is a better fit for my available time/energy resources, so this is where I try to do my “fixing packaging knowledge” work.

There are so many good, well written contributions with great, useful info here and on Packaging in general, but if this info does not end up in the guide, (virtually) no one will find it.

@indygreg, no offense intended. I encourage you to write the follow-up article "How to migrate from setup.py to pyproject.toml on the Python Packaging User Guide. :slight_smile:

8 Likes

The article is very significantly different from a “rant”, in that it’s essentially a script of the process. If Gregory had recorded a video of going through the process, that’s what we’d be getting to see.

Rants tend to be opinionated and disorganised, with no clear narrative (the “best” rants usually have a narrative based around an increasingly emotional response, e.g. shock jocks and modern “news” “reporters”, but the typical internet rant doesn’t even have this). They can usually be summarised down into tweet size without losing important information, and generally don’t provide any value other than indicating that there may be a problem somewhere.

I got three major points from Gregory’s post, which I suspected were true anyway but couldn’t provide any real evidence:

  • the right websites have the right information, but are not the obvious results (e.g. the most correct information about setuptools was not in pip’s docs, Stack Overflow, or poetry’s docs… but in setuptools’s docs)
  • choosing a “winner” on packaging.python.org adds confusion (more than it’s worth? Perhaps…)
  • error messages cannot be too detailed, but the blog posts they link to can be too detailed

And a specific issue would seem to be that we all rely too heavily on PEP numbers instead of feature names. Virtually every doc Gregory encountered starts by listing and linking to PEPs, instead of walkthroughs, tutorials or even definitions. It’s incredibly difficult to get a clear, succinct summary of the necessary context into every page a user might reasonably land on (especially when we assume they’re starting from a search engine or Stack Overflow post rather than following the structure we intended), but it really would seem that linking to a detailed proposal - most of which spend more time on edge cases, exceptions or rejected ideas than the actual feature! - is doing a real disservice.

All of these points can be seen clearly in Gregory’s post, and would be seen in a user study (e.g. where you’re watching over someone’s shoulder as they try to figure it out). It’s very unlikely that a rant would illustrate them as clearly.

25 Likes

[Whether the article is a rant or not is completely besides my main point. I clearly wrote that I do not believe it is intended to be one. I do not understand why we feel the need to expand on this. But since we are here, I will draw the attention to the second to last paragraph of the article.]


We are already 40 messages deep in this thread, and have made no apparent progress towards improving documentation. Why? What is the blocker really? Should we investigate this honestly? How come we are so bad at actually getting the info that we already wrote (in this thread alone or even in Gregory’s article there is a huge amount of useful, actionable info) into the packaging guide?

I wonder if a wiki would make sense. There wouldn’t be the hurdle of making pull requests and getting them reviewed.

a specific issue would seem to be that we all rely too heavily on PEP numbers instead of feature names”. This PR took 3 months to get accepted. This one 2 months. I believe that is an issue. Anyway I gave up on trying to contribute to the packaging guide. I know it is better now, and I might try again soon.

After Chris Warrick’s article back in January, I tried to extract some of the info I considered useful, and produced this. It is clunky and incomplete for sure, but when I refer to it on StackOverflow people find it useful and are thankful for it.

I want to encourage you all to copy your contributions from Packaging and paste them into the packaging guide where they can actually be seen. Do not overthink it. :slight_smile:

13 Likes

This. Please, for whatever is precious to you call these features
by their names not PEP numbers. It is like discussion of two
Reformed Theologians “Because of Mt 22:23 and of course consider
1Co 4:2, and you cannot miss Rev 8:6, it is obvious that you are
an idiot.” I hate it when they do it (and yes, I am a practising
Christian) and we don’t have excuse of such huge body of text.

Thank you,

Matěj

5 Likes

If that’s the way we want to do things, then I think we should follow up on @sinoroc’s suggestion of a wiki. If the idea is that anyone can go in and edit/add whatever they want, then I don’t see what is accomplished by doing everything through a github account and requiring the use of RST, other than raising barriers to entry.

That’s actually an appealing idea, but I’m not sure if it’s the best approach, both because of what @pitrou said about the lack of central direction, and because:

  1. The point of having official documentation is that it should represent something endorsed by the Python organization (ultimately the steering council, perhaps via delegates). If we just want individuals putting up whatever they want then we could just have links to a bunch of blogs.
  2. If the documentation is to integrate instructions on how to use third-party tools, then that likely requires coordination with the maintainers of those tools, to ensure that the pypa docs don’t get out of sync with what tools actually do. That is likely to be more feasible with something like the PyPC to act as a nexus for that coordination.
4 Likes

The confusion didn’t arrive from PyPA docs choosing a winner. It was that the story was inconsistent. The story of build, twine, and pip is great then it became inconsistent with the build-backends: why are hatchling, flit, setuptools, pdm mentioned altogether?

I agree with this being the responsibility of PyPC. Specifically it is PyPC that needs to decide on which story to tell. Otherwise, what do you want the contributors to do? I cannot expect to come in and contribute a modification that involves removing any recommendations that would be approved.


To be fair if the story is hatchling, this would unlikely help @indygreg as hatchling cannot handle extensions (IIRC). I agree there’s needs to be a separate migration guide for setuptools and a guide on extension packages in general. Perhaps just take a quick look at the sidebar of PyPA vs. scientific-python

5 Likes

I didn’t get that impression from the blog post. In fact perhaps the reverse, in the sense that there is too much discussion of many tools rather than a clear set of instructions that say “this is the tool you need to use”. In other words, as was mentioned in many other packaging threads, the necessary functionality is spread across many tools, making it hard to figure out which one to use.

And a specific issue would seem to be that we all rely too heavily on PEP numbers instead of feature names. Virtually every doc Gregory encountered starts by listing and linking to PEPs, instead of walkthroughs, tutorials or even definitions.

Agreed that this is a huge problem. In the end we can’t stop people from linking to PEPs, but one thing that might help is making sure that every PEP page on peps.python.org links to the official documentation of whatever feature the PEP introduced, maybe with a note saying “don’t link to this PEP as an explanation of how it works”. It should never be necessary to link to a PEP to get a specification of behavior; finalization of the PEP should include the creation of official documentation, so people should only need to look at the PEP if they want to understand the history of how that behavior came about.

6 Likes

I hope some of you have seen the pyOpenSci packaging guide. I like its approach of “here’s the basics, now we start to branch out based on what you want to do”.

So the problem here is that OP didn’t find a good explanation of the basics, leaving them unprepared to do their (non-trivial) task of migrating.

The solution could be changing our guides to be similarly branching as the one mentioned above. Clearly we can improve documentation for “I want to switch my extension module package building to the modern way”

The answer is that only direct invocation of setup.py is deprecated, so one only has to deal with 1. replacing those invocations with things like pypa/build and 2. build isolation. I feel like helping people with how to do the latter is the challenging part here.

PS: I like Ofek’s idea of decoupling wheel packaging backends and ways to build extension modules.

5 Likes

So you’re suggesting more strongly picking a winner? The problem here is that the user wasn’t trying to choose a new backend, they were trying to figure out how the one they already had fit into the new scheme (without realising that it’s now considered a “backend”, as opposed to an end-to-end tool).

Both the tutorial and the packaging flow documents introduce backends merely as configuration settings in the pyproject.toml, which IMHO is elevating the wrong concept. For someone planning to produce a package, choosing the backend is the key decision for them to make, and how they configure things follows directly from that (which is indeed mentioned, but it’s very easy to skim over the section because of the focus on the config file).

Renaming “Creating pyproject.toml” to “Choosing your build backend” and making no other changes is probably enough to emphasise the right thing. Someone who already has a build backend will more quickly realise that they don’t actually have to choose, whereas right now it can come off a bit like “Creating pyproject.toml means I should use what the tutorial recommends” which is exactly what happened in this case.

4 Likes

That might help but I think that there is a particularly common case here which is maintainers of existing projects that already use setuptools (or distutils or numpy.distutils). It is not helpful that this common case is not addressed directly. That is why I would suggest two guides aimed at people in this camp:

  1. Updating from setup.py to pyproject.toml with setuptools.
  2. Why you might or might not consider migrating to an alternative to setuptools.

Then these two guides are directed squarely at the majority of maintainers who would potentially modernise some project to use the new standards (using setuptools or using something else).

6 Likes

Ah, I see what you mean. Yes, I agree that is a problem.

What I mean by “pick a winner” is really “pick a winner for each use case”. In other words, have some kind of flowchart-like tutorial that tells people which tools are right for their use case. So maybe “I already have a setup.py and I don’t want to rock the boat” is a use case, and in that case the answer can be “use setuptools”.

Yes. In fact I would say the backend choice should be split out into a separate section.

I think this would go a long way toward guiding different user personas to the next step.

2 Likes

Yup, this is basically the ideal, from a PEP editor perspective. However, while we’ve improved this aspect including it in our checklists and guidance on marking a PEP accepted and final, either historically or even currently there isn’t really a necessarily been consistent pattern of always porting over the endearingly-relevant PEP content to a more suitable platform.

Packaging got partway there, and there’s been sporadic efforts to complete the job, but the people who’ve worked on it (certainly me, at least) have mostly ended up burning out fairly quickly for various reasons. IMO, I think we all recognize the better approach is to make sure PEP authors port their PEP over once it is accepted, which has been done in most but not all recent cases, but we still need to backfill a number of remaining specs.

As an example case, perhaps the most important is PEP 517/518/660, pyproject.toml and the backend interface, which I tried to tackle. The problem is that for historical reasons, the organization of the content in those PEPs has little correspondence to what would actually make sense in a canonical living standard for that today. I tried to do too much all at once to re-organize all of that material into a coherent, logic structure appropriate for a set of living standards, and the result was such a huge chunk of work as to be essentially unreviewable, so it stalled and I burned out.

In the end we can’t stop people from linking to PEPs, but one thing that might help is making sure that every PEP page on peps.python.org links to the official documentation of whatever feature the PEP introduced, maybe with a note saying “don’t link to this PEP as an explanation of how it works”.

We actually have a custom directive for that, canonical-doc, which adds a sticky banner in the PEP content and at the top of the screen with customizable text letting users who the PEP is a historical change proposal and linking to the canonical documentation instead. Additionally, we’ve included adding this as a requirement in the checklist to mark a PEP as “Final”. However, it is unfortunately not widely used yet, and we have many PEPs we need to backfill it to. Its basically just adding, e.g.

.. canonical-pypa-spec:: :ref:`packaging:declaring-project-metadata`

to the top of the PEP (or its more generic superclass, canonical-doc, for non-PyPA specs), which renders as

image

See PEP 621 to see that example in action.

That was something I was starting to try to tackle, but it never got far before other priorities intervened. However, it its something that its easy to help with, provided you have some familiarity with the PEPs and specs involved (unfortunately, most of the people who do are also those whose volunteer time is in most demand…).

And unlike embedding the metadata in setup.py, its straightforward to convert to pyproject metadata in pyproject.toml, since ini2toml was created explicitly for that purpose by the Setuptools maintainers to make that transition fairly painless.

1 Like
5 Likes

Thanks, I took a few minutes and turned it into a PR: Update Choosing a build backend section by zooba · Pull Request #1326 · pypa/packaging.python.org · GitHub

3 Likes

With regard to past cases, as you say it’s a matter of catching up. However, insofar as this is still “currently” going on, these do seem like cases where the issue is one of governance. Isn’t the solution to that for the decision-maker (e.g., PEP delegate) to take a firm stand and say “This PEP will not be marked as final until the implementation is complete”? (With the implementation for packaging-spec-type PEPs being just the docs.) Or are you saying these PEPs languish in “accepted” state and never get to “final”?

I certainly require that PEPs get moved to the standards documentation as part of the process of implementation. And I tend to insist on at least one reference implementation. But how far do you go here? Everyone is a volunteer, there are multiple projects that need to implement most standards, and many projects have a policy of not implementing standards before they are finalised.

It’s easy to suggest rules like this, but in practical terms we’re already doing about as much as we can here, short of taking control of packaging projects away from their maintainers (or making it effectively near-impossible to finalise any standards).

2 Likes

Someone feels like doing this (easy) one?

2 Likes

I have gone ahead and made a PR as well: Remove backend choice and just use Hatchling so the tutorial can be used without making any choices. by BrenBarn · Pull Request #1328 · pypa/packaging.python.org · GitHub . This conflicts with @steve.dower’s because I just axed the entire part of the tutorial that involves choosing a backend and made it use Hatchling. I’ve never even used Hatchling and I don’t care what the backend is, but (as mentioned on other threads) I don’t think a first-look “here is how to do this” tutorial should say anything about backend choice other than “look elsewhere if you want to do this any other way than exactly how this tutorial does it”. The user shouldn’t need to make any choices like that to follow the tutorial.

Note this means it won’t solve the specific problem discussed in this thread, where there’s a preexisting desire to use setuptools; but it will mean that people without such a precondition (which is probably the majority of people looking for packaging info) will get a single, linear, working example.

In your quote you left out the part where I said “the implementation for packaging spec PEPs is just the docs”. I’m not suggesting we wait for any tools to actually implement the spec, just wait until there is no information that is in the PEP that has not been properly added to the actual docs (and add a link to the relevant doc sections in the PEP making clear that the docs are the authoritative source for the spec, and not the PEP).