pypa:main ā CAM-Gerlach:add-pep-517-518-660-to-specs
opened 04:13AM - 25 Jul 22 UTC
@pfmoore @brettcannon @pradyunsg
As discussed on #955 , this (finally!) migrā¦ ates the content of PEP 517, PEP 518 and PEP 660 to the specifications section and glossary, and updates the various references throughout the site, specs and glossary to the PEPs, their contents and the terms they define to refer to, and accurately reflect, the newly canonical content spec and glossary content.
Overall, while I did put a lot of effort into diligently taking advantage of the appropriate reST syntax/directives/roles and Sphinx cross referencing for consistent source formatting and output rendering, implementing editorial corrections for organization, clarity and consistency, and making some limited and motivated additions and updates to the non-normative portions (mostly added introductory/connecting material and updating/fleshing out examples), I attempted to be conservative and avoid any substantive changes to the actual content of the normative specification, which can be discussed and applied subsequent to this initial PR.
On the high level, they end up most neatly fitting into three separate specs:
* A brief new summary meta spec covering the `pyproject.toml` file as a whole, containing the high-level content from PEP 518 (as well as a bit from PEP 621/the declaring project metadata spec), since it services multiple purposes beyond just the original build-system table, so it wouldn't make much sense nor be reader-friendly to bury it inside a build interface spec. It links to the individual specs for each table (which also helps address the confusion and concerns of multiple recent readers who had difficulty finding it and its subtables, since the previous names were highly non-obvious to them).
* Broadening the existing "Declaring build system dependencies" specification to cover the `[build-system]` table as a whole, containing the `[build-system]` content from PEP 518 and the build-system top-level section from PEP 517. This section includes a lot more than just the `requires` key so it should not be limited to it, but can fairly cleanly be separated from the actual build interface specification itself, and also helps partition the content of potential interest to regular package authors versus build frontend/backend developers
* Finally, creating a new Build Frontend-Backend Interface specification that covers the hooks and the specified behavior of frontends and backends themselves from PEP 517 and PEP 660, as would be consulted by frontend and backend developers.
I've made some relatively modest adjustments to the [outline originally proposed in #955](https://github.com/pypa/packaging.python.org/issues/955#issuecomment-1161046484):
<details>
<summary>Final outline w/ mapping from existing PEP sections (click to expand)</summary>
* _The `pyproject.toml` configuration file_
* FIle format and location ([PEP 518 section](https://peps.python.org/pep-0518/#file-format))
* `build-system` table (high-level statement of purpose from the [PEP 518 section](https://peps.python.org/pep-0518/#build-system-table); links to _Specifying a project's build system_ spec for the actual details)
* `project` table (very brief statement of purpose, which links to the [_Declaring project [source] metadata_ spec](https://packaging.python.org/en/latest/specifications/declaring-project-metadata/))
* `tool` table ([PEP 518 section](https://peps.python.org/pep-0518/#tool-table))
* _Declaring a project's build system_
* `requires` key ([PEP 518 build-system section](https://peps.python.org/pep-0518/#build-system-table) and [PEP 517 Build Requirements section](https://peps.python.org/pep-0517/#build-requirements)
* `build-backend` key ([PEP 517 Source Trees section](https://peps.python.org/pep-0517/#source-trees) [which only contains ā1 sentence about source trees])
* `backend-path` key ([PEP 517 In-tree Build Backends section](https://peps.python.org/pep-0517/#in-tree-build-backends))
* _Build front-end backend interface_
* Wheel hooks
* `build_wheel` (mandatory) ([PEP 517 section](https://peps.python.org/pep-0517/#build-wheel))
* `get_requires_for_build_wheel` ([PEP 517 section](https://peps.python.org/pep-0517/#get-requires-for-build-wheel))
* `prepare_metadata_for_build_wheel` ([PEP 517 section](https://peps.python.org/pep-0517/#get-requires-for-build-wheel))
* Sdist hooks
* `build_sdist` (mandatory) ([PEP 517 section](https://peps.python.org/pep-0517/#build-sdist))
* `get_requires_for_build_sdist` ([PEP 517 section](https://peps.python.org/pep-0517/#get-requires-for-build-sdist))
* Editable hooks
* `build_editable` ([PEP 660 section](https://peps.python.org/pep-0660/#build-editable))
* `get_requires_for_build_editable` ([PEP 660 section](https://peps.python.org/pep-0660/#get-requires-for-build-editable))
* `prepare_metadata_for_build_wheel` ([PEP 660 section](https://peps.python.org/pep-0660/#prepare-metadata-for-build-editable))
* Config settings ([PEP 517 section](https://peps.python.org/pep-0517/#config-settings))
* Build environment ([PEP 517 build environment section](https://peps.python.org/pep-0517/#build-environment)) and semi-normative portion of ([PEP 517 forntend recommendations section](https://peps.python.org/pep-0517/#recommendations-for-build-frontends-non-normative))
* Frontend requirements for editable installs ([PEP 660 Frontend Requirements section](https://peps.python.org/pep-0660/#frontend-requirements))
* [_PyPA Glossary_](https://packaging.python.org/en/latest/glossary/) (for terminology, linked in the above documents on first use)
* Build backend, Build backend, Integration frontend, Source tree ([PEP 517 terminology section](https://peps.python.org/pep-0517/#terminology-and-goals))
* Editable installation ([PEP 660 terminology section](https://peps.python.org/pep-0660/#terminology-and-goals))
* Updates to other related terminology to cross-link and reflect changes accordingly
</details>
After a fair amount of thought and experimentation, I've tweaked the current `Package Distribution Formats` heading to read `Building and Installing` heading and added/moved the new specs there, alongside the "Recording Installed Distributions" spec, given it is a much better fit for the new section than the crowded, core-metadata-focused "Package Distribution Metadata" section, particularly given it and the wheel spec that unpacks to it and the hooks that build it go hand in hand, and the new section now describes the build and install process from end to end.
On a procedural note, I made every effort to break this up into separate PRs or at the very least separate commits, as I normally would (and all the future specs should be). However, I ended up having to make this as a single atomic, squashed commit for practicality's sake, because otherwise there would be intractable circular dependencies between each PR, given the new specs, existing specs, glossary entries and other documents all heavily cross-reference and depend upon one another, and that would furthermore require a heavy cost in both time and quality/consistency, when this will be squashed down anyway, has been blocked on such for years and is much better spent migrating other specs and making further improvements to the packaging site in followup PRs.
I'll post a thread in the Packaging Discourse to announce this and solicit reviews of this, to both check for editorial correctness and ensure it accurately reflects the PEPs on which it is based.
Part of #1093
Resolves #955
As discussed on pypa/packaging.python.org#955 , this (finally!) migrates the content of PEP 517, PEP 518 and PEP 660 to the specifications section and glossary, and updates the various references throughout the site, specs and glossary to the PEPs, their contents and the terms they define to refer to, and accurately reflect, the newly canonical content spec and glossary content.
Overall, while I did put a lot of effort into diligently taking advantage of the appropriate reST syntax/directives/roles and Sphinx cross referencing for consistent source formatting and output rendering, implementing editorial corrections for organization, clarity and consistency, and making some limited and motivated additions and updates to the non-normative portions (mostly added introductory/connecting material and updating/fleshing out examples), I attempted to be conservative and avoid any substantive changes to the actual content of the normative specification, which can be discussed and applied subsequent to this initial PR.
Iād appreciate anyone interest to review the PR, to both check for editorial correctness and ensure it accurately reflects the PEPs on which it is based. Please see the issue and PR threads for more detail on the final high-level organization and motivations behind it, and other considerations to keep in mind when reviewing. Thanks, and hope to get this finally merged soon!
6 Likes
JDLH
July 25, 2022, 7:28am
2
Awesome! Thank you for this, it looks like it was a lot of work. I have started to review it.