pradyunsg
June 22, 2022, 8:21am
#1
This has been something we’ve wanted to do for a while, and it’s not happened yet. I wanna chart a path onward to getting us there, now that Packaging PEPs | peps.python.org is a thing.
AFAICT, this means that we should:
Are there any broad strokes concerns with this plan?
6 Likes
pf_moore
June 22, 2022, 10:23am
#2
None from me, other than the usual “I’m pretty sure this has been the plan for ages now, but no-one has had the time to actually do the work”
2 Likes
pradyunsg
July 22, 2022, 12:34pm
#3
Poking this again, to give folks another chance to chime in before I start putting a concentrated effort toward this.
I say go for it and don’t wait. I’ve not heard anyone actually object to this overall idea and there’s been plenty of time to speak up (well beyond this thread’s lifetime).
2 Likes
AFAIK, after years of discussion of this, the only reason I know why it hasn’t happened was a simple lack of someone both able and willing to put the requisite Sphinx, technical writing, PEP/spec and packaging time, effort and expertise toward the problem. Now that we nearly simultaneously and independently have two such people, it seems the only question becomes how we’re going to most efficiently do the work my comment over on the issue .
And the PR, pypa/packaging.python.org#1111, for migrating PEP 517/518/660, is now live!
pypa:main ← CAM-Gerlach:add-pep-517-518-660-to-specs
opened 04:13AM - 25 Jul 22 UTC
@pfmoore @brettcannon @pradyunsg
As discussed on #955 , this (finally!) migr… ates the content of PEP 517, PEP 518 and PEP 660 to the specifications section and glossary, and updates the various references throughout the site, specs and glossary to the PEPs, their contents and the terms they define to refer to, and accurately reflect, the newly canonical content spec and glossary content.
Overall, while I did put a lot of effort into diligently taking advantage of the appropriate reST syntax/directives/roles and Sphinx cross referencing for consistent source formatting and output rendering, implementing editorial corrections for organization, clarity and consistency, and making some limited and motivated additions and updates to the non-normative portions (mostly added introductory/connecting material and updating/fleshing out examples), I attempted to be conservative and avoid any substantive changes to the actual content of the normative specification, which can be discussed and applied subsequent to this initial PR.
On the high level, they end up most neatly fitting into three separate specs:
* A brief new summary meta spec covering the `pyproject.toml` file as a whole, containing the high-level content from PEP 518 (as well as a bit from PEP 621/the declaring project metadata spec), since it services multiple purposes beyond just the original build-system table, so it wouldn't make much sense nor be reader-friendly to bury it inside a build interface spec. It links to the individual specs for each table (which also helps address the confusion and concerns of multiple recent readers who had difficulty finding it and its subtables, since the previous names were highly non-obvious to them).
* Broadening the existing "Declaring build system dependencies" specification to cover the `[build-system]` table as a whole, containing the `[build-system]` content from PEP 518 and the build-system top-level section from PEP 517. This section includes a lot more than just the `requires` key so it should not be limited to it, but can fairly cleanly be separated from the actual build interface specification itself, and also helps partition the content of potential interest to regular package authors versus build frontend/backend developers
* Finally, creating a new Build Frontend-Backend Interface specification that covers the hooks and the specified behavior of frontends and backends themselves from PEP 517 and PEP 660, as would be consulted by frontend and backend developers.
I've made some relatively modest adjustments to the [outline originally proposed in #955](https://github.com/pypa/packaging.python.org/issues/955#issuecomment-1161046484):
<details>
<summary>Final outline w/ mapping from existing PEP sections (click to expand)</summary>
* _The `pyproject.toml` configuration file_
* FIle format and location ([PEP 518 section](https://peps.python.org/pep-0518/#file-format))
* `build-system` table (high-level statement of purpose from the [PEP 518 section](https://peps.python.org/pep-0518/#build-system-table); links to _Specifying a project's build system_ spec for the actual details)
* `project` table (very brief statement of purpose, which links to the [_Declaring project [source] metadata_ spec](https://packaging.python.org/en/latest/specifications/declaring-project-metadata/))
* `tool` table ([PEP 518 section](https://peps.python.org/pep-0518/#tool-table))
* _Declaring a project's build system_
* `requires` key ([PEP 518 build-system section](https://peps.python.org/pep-0518/#build-system-table) and [PEP 517 Build Requirements section](https://peps.python.org/pep-0517/#build-requirements)
* `build-backend` key ([PEP 517 Source Trees section](https://peps.python.org/pep-0517/#source-trees) [which only contains ≈1 sentence about source trees])
* `backend-path` key ([PEP 517 In-tree Build Backends section](https://peps.python.org/pep-0517/#in-tree-build-backends))
* _Build front-end backend interface_
* Wheel hooks
* `build_wheel` (mandatory) ([PEP 517 section](https://peps.python.org/pep-0517/#build-wheel))
* `get_requires_for_build_wheel` ([PEP 517 section](https://peps.python.org/pep-0517/#get-requires-for-build-wheel))
* `prepare_metadata_for_build_wheel` ([PEP 517 section](https://peps.python.org/pep-0517/#get-requires-for-build-wheel))
* Sdist hooks
* `build_sdist` (mandatory) ([PEP 517 section](https://peps.python.org/pep-0517/#build-sdist))
* `get_requires_for_build_sdist` ([PEP 517 section](https://peps.python.org/pep-0517/#get-requires-for-build-sdist))
* Editable hooks
* `build_editable` ([PEP 660 section](https://peps.python.org/pep-0660/#build-editable))
* `get_requires_for_build_editable` ([PEP 660 section](https://peps.python.org/pep-0660/#get-requires-for-build-editable))
* `prepare_metadata_for_build_wheel` ([PEP 660 section](https://peps.python.org/pep-0660/#prepare-metadata-for-build-editable))
* Config settings ([PEP 517 section](https://peps.python.org/pep-0517/#config-settings))
* Build environment ([PEP 517 build environment section](https://peps.python.org/pep-0517/#build-environment)) and semi-normative portion of ([PEP 517 forntend recommendations section](https://peps.python.org/pep-0517/#recommendations-for-build-frontends-non-normative))
* Frontend requirements for editable installs ([PEP 660 Frontend Requirements section](https://peps.python.org/pep-0660/#frontend-requirements))
* [_PyPA Glossary_](https://packaging.python.org/en/latest/glossary/) (for terminology, linked in the above documents on first use)
* Build backend, Build backend, Integration frontend, Source tree ([PEP 517 terminology section](https://peps.python.org/pep-0517/#terminology-and-goals))
* Editable installation ([PEP 660 terminology section](https://peps.python.org/pep-0660/#terminology-and-goals))
* Updates to other related terminology to cross-link and reflect changes accordingly
</details>
After a fair amount of thought and experimentation, I've tweaked the current `Package Distribution Formats` heading to read `Building and Installing` heading and added/moved the new specs there, alongside the "Recording Installed Distributions" spec, given it is a much better fit for the new section than the crowded, core-metadata-focused "Package Distribution Metadata" section, particularly given it and the wheel spec that unpacks to it and the hooks that build it go hand in hand, and the new section now describes the build and install process from end to end.
On a procedural note, I made every effort to break this up into separate PRs or at the very least separate commits, as I normally would (and all the future specs should be). However, I ended up having to make this as a single atomic, squashed commit for practicality's sake, because otherwise there would be intractable circular dependencies between each PR, given the new specs, existing specs, glossary entries and other documents all heavily cross-reference and depend upon one another, and that would furthermore require a heavy cost in both time and quality/consistency, when this will be squashed down anyway, has been blocked on such for years and is much better spent migrating other specs and making further improvements to the packaging site in followup PRs.
I'll post a thread in the Packaging Discourse to announce this and solicit reviews of this, to both check for editorial correctness and ensure it accurately reflects the PEPs on which it is based.
Part of #1093
Resolves #955
— see