PEP 621: round 3

It is with pleasure that I am formally accepting PEP 621. Thanks to @brettcannon and the other authors for their work in developing the PEP, and to the “cast of thousands” who contributed to the discussions.

After reflection, I have decided that the PEP is initially accepted as provisional, to allow time for any issues identified as tools implement the spec to be ironed out. But I am strictly limiting the provisional status to a maximum of (just over) three months, so the PEP will be declared final on 1st March 2021. If we have fully working implementations of the PEP sooner than that, I’m happy to move the status to final before the deadline.

13 Likes

PEP 1 doesn’t spell it out, but I agree it makes sense for the BDFL-Delegate to approve leaving the Provisional state (reverting to the SC if the original delegate is uncontactable).

And for this specific case, I agree that the Provisional qualification can now be removed - the proposal has been successfully implemented and deployed.

1 Like

Thanks! Glad the PEP worked out in the end!

Would you like me to hold off on a PR for packaging.python.org to link to the PEP from there (or fully copying it over so the official version is kept on packaging.python.org entirely)?

I don’t think there’s a need to hold off. Sure, there’s a possibility of change due to the provisional status, but I certainly don’t think we should delay having official documentation until the provisional status is removed.

I’d prefer it if the official version were kept on packaging.python.org, rather than just linking to the PEP, but that is a non-trivial amount of extra work, so I’m perfectly fine if you prefer to just link to the PEP for now.

2 Likes

I’m happy to put in the work to move it all over to packaging.python.org since it will make it easier to read due to having less comparison details to other tools and such.

1 Like

Quick update: I marked the PEP as provisionally accepted. Next step is creating the page for the spec on packaging.python.org.

1 Like

PR for the spec on packaging.python.org: https://github.com/pypa/packaging.python.org/pull/797.

1 Like

As an update to this, PEP 621 is now officially declared as Final. @brettcannon can you update the PEP to reflect this?

2 Likes

PEP 621: mark as final by brettcannon · Pull Request #1857 · python/peps · GitHub has the update to final.

3 Likes

I’ve just released Flit 3.2 with experimental support for PEP 621. It’s also using PEP 621 itself (is it the first package to do so? :slightly_smiling_face: )

This is largely undocumented for now - the idea is that early adopters can try it out before everyone switches over. I’m asking that anyone who does try it out declare that only Flit 3.2.x should build their project, in case the support needs to change for 3.3 - i.e. start pyproject.toml like this:

[build-system]
requires = ["flit_core >=3.2.0,<3.3"]
6 Likes

(There is pdm already that I know of.)

I released whey (as in curds and whey, used to make wheels of cheese; Home - whey 0.0.6 documentation), a PEP 517 backend with PEP 621 support, on the 23rd Feb. I’ve used it for several of my projects already.

I’m curious how flit handles the case where one author has only a name and one has only an email like in the example? It wasn’t obvious to me what should go in which core metadata field.

1 Like

OK, I don’t get to lay claim to the first PEP 621 package, then. Congrats to both of you!

Fairly naively. Following the recommendation in PEP 621, authors with email addresses go in Author-Email, and authors with just a name go in Author. So in that case, it wouldn’t be obvious in the resulting METADATA file that they were meant to be for two different people. You can see the relevant code in flit here.

Once PEP 621 is more widely adopted, someone who wants to automate getting author metadata might be better off fetching sdists and parsing the pyproject.toml, rather than trying to make sense of the essentially free-text fields in a METADATA file. So I’m not going to lose too much sleep over the potential ambiguity once it’s translated into METADATA.

1 Like

Either of you care to share how PEP 621 has worked for you? Other than all of us (re)discovering ambiguities in METADATA, has PEP 621 made sense, seemed to work well, etc.? I.e. did I help write a horrible PEP or does it seem okay? :wink:

1 Like

So far, I’ve been pretty happy with it. We’ll see how it goes as more people try to use it.

One question that I wondered about: what should a tool like Flit do if there are extra keys in the [project] table which it doesn’t recognise? I.e. is it meant to be extensible, and if so, will it be OK for implementations to ignore new keys, or should they bail out if there’s anything they don’t understand? For now, I’ve gone for a warning (“Unexpected names under [project]”) but no error.

Flit isn’t doing much with the license field at present - it includes a file if one is specified, but doesn’t put anything in the metadata for it. I understand we’re waiting on PEP 639 to define more precise ways of describing licenses in metadata (let’s not get into that question on this thread).

1 Like

The PEP says on this subject:

Tools MUST specify fields defined by this PEP in a table named [project]. No tools may add fields to this table which are not defined by this PEP or subsequent PEPs.

That to me says “error”, but you’re right it doesn’t explicitly say “error out”. @pf_moore what do you think?

IMO, the section you quoted, in conjunction with the paragraph just above that

When specifying project metadata, tools MUST adhere and honour the metadata as specified in this PEP. If metadata is improperly specified then tools MUST raise an error to notify the user about their mistake.

says to me that tools should flag an error if they see anything not defined by the PEP.

However, I think we could have an issue soon, because there’s no versioning of the format, so when a new metadata field is added (for example, License-Expression) we’re going to have to work out how to add that to pyproject.toml in a backward compatible manner. I’m sorry I didn’t spot this when I was reviewing the PEP for approval.

Getting a fix for this issue into the spec before too many people have implemented it might be worthwhile - we may have a small window where we can reach out to tools that have implemented the existing PEP without too much difficulty, so we may be able to manage the backward compatibility requirements somewhat.

We should also consider who is responsible for updating the PEP 621 definition when a new metadata item is added - maybe we should formally document the process for adding new metadata items, so it’s clear that pyproject.toml needs to be considered as well as the core metadata specs.

2 Likes

Do we need to?

Older tool versions not understanding newer keys and failing because of them is a much cleaner split (for explaining and understanding) than having it work the wrong way due to a subtlety.

As an example from our domain, older versions of pip don’t understand yanking (which causes it to not ignore those versions). This regularly shows up in discussions online and in pip’s issue tracker as a source of user confusion.

The trouble of the chicken-egg problem for adoption of new keys in this case is… fine?

The most troublesome tooling that would fail due to this is build tools, and those are opted into by the user (and their versions can be constrained with build-system.requires). Realistically, any supporting tooling would likely not be thoroughly validating all the values anyway, so it should be A-OK on that front as well?

FWIW, once a tool supports the newer keys of the format, it’ll still be able to read from older [project] tables; given that all new keys will definitely be optional. (if they aren’t, we can add a new key to denote the version when we make that change).

This should all be fine if you’re constraining the backend versions in build-system.requires, which is already showing up as best-practice among the various build backends anyway.

Mostly, I agree. But if flit (for example) were to say “projects specify their metadata using the [project] key in pyproject.toml (insert link to spec here)”, and a user looked at that spec, found license-expression, added it and got an error, they’d be justified in being confused and annoyed.

Maybe all that’s needed is for flit to document that it supports metadata 2.2, and when flit 3.3 is released, update the docs and have a changelog entry saying “Now supports metadata 2.3”. Along with a requirement that additions to the metadata spec must also update Declaring project metadata — Python Packaging User Guide to say what the pyproject.toml name of the new field is, that’s probably sufficient.

Yeah, OK. I was probably over-reacting (it was late when I posted!)

1 Like