Delays due to moon launch responsibilities are acceptable I guess
As long as heās not saying Laika ate his homework I guess thatās a good enough excuse.
Any update?
Hey @CAM-Gerlach congrats for sending the rocket towards the Moon and not crashing it!
@thatch and I are back on the saddle and willing to help usher this through the finish line.
You have done an enormous amount of work there!
What can we do to help?
What is your latest PR?
One thing that came up in hatchās issue tracker: Custom files under `dist-info` as part of a build hook Ā· Issue #620 Ā· pypa/hatch Ā· GitHub
Currently, this PEP defines a directory for storing license files under the .dist-info folder, in a sub-folder. In the discussions back when installerās scope/API was being figured out, @dholth pointed out that (at least, by design) the files in dist-info should be treated like a dictionary of name-of-file: text-contents-of-file
with no directories expected in there (and, that was somewhat baked-into the API design of installer).
Iām mentioning this to elevate from those discussions that this is a semantic change weād be making to how dist-info works, which could lock us out of potential data structures for storing metadata info in that, and other stuff like that, in the future.
I donāt personally have strong opinions on this; the pip/installer side of this is relatively straightforward to deal with, and Iām fairly certain that they already do ā but Iām not sure if everyone is fine with making this change for tooling. Itās certainly worth calling out in the PEP as well.
I also donāt have a strong opinion here, beyond that we shouldnāt just make assumptions, we should agree and document a position, one way or another.
The spec for the .dist-info
directory says āAdditional installer-specific files may be presentā. That doesnāt explicitly prohibit directories, but it could be viewed as implying āfiles only, no directoriesā. The wheel spec says nothing about this, as far as I can tell.
Does anyone know of any existing wheels or installed projects that have subdirectories in .dist-info
? My database of PyPI data doesnāt currently include wheel contents, so I canāt check this myself.
If weāre allowing custom (whether backend or user defined) files in .dist-info
, then we have a potential backward compatibility issue whenever we add new files. For example, the proposal here for a licenses
directory would be incompatible with a custom file called licenses
. This isnāt mentioned in the PEPās āBackward compatibilityā section, by the wayā¦ A way round this would be to reserve all top level names (maybe explicitly noting that setuptools adds a number of non-standard names, which will remain for compatibility) and require all non-standard metadata files to go in a custom
subdirectory. That means accepting that subdirectories are allowed.
Directories would be fine.
I would like to see new keys in the .data/ directory to mark categories of files.
Hey all, is there any update on this? Iāve now ran into multiple situations where having this PEP active would be helpful.
I sat down with @CAM-Gerlach at PyCon US sprints yesterday, and we made progress on the draft updates.
Has there been any update on this?
Is there anything I can do to help?
Has there been any update on this?
I think as this point @CAM-Gerlach should either withdraw the PEP so someone else can draft a new one, or find a co-author who can drive this to completion. As PEP delegate, Iām giving CAM a week to decide which he prefers (Iām not giving explicit time for another draft as the amount of time and pinging has been enough to show CAM simply doesnāt have the bandwidth to drive this right now).
Sorry, Iād DMed Brett as well as some interested co-authors a couple weeks ago, but after getting confirmations from each of them, following up with them further on Discord and letting them know Iād post this here, I didnāt realize Iād accidentally left this as a draft and not submitted it. My faultā¦original message follows:
Iām very sorry to have disappointed everyone yet again with my continued slow progress and delinquency of my responsibilities on PEP 639, and I appreciate all your pings and attempts to get things moving again. While Iām sorry he had to post it, Iām of course very thankful for Brett and others here having been proactive in helping move things along without me given the growing interest in seeing it through.
The good news is that weāve found not one but two new motivated and very well-qualified authors to help drive this PEP to completion. Please welcome @sethmlarson and @ksurma on board! As Iām sure many of you know, Seth is the new Security Developer in Residence with the PSF, and reached out to Brett and myself to help with this PEP due to its SPDX support helping provide metadata important for SBOMs and in turn, supply chain security. Meanwhile, Karolina works along with Petr Viktorin at Red Hat on Python packaging, and her and her team are already making considerable use of initial implementations of this PEP to more easily ingest Python package license files and metadata, and reached out to me following our discussion at the Python core dev sprint looking to help with this PEP so they can further that goal.
Iāve been following up with them to help get them up to speed on the current status of the PEP and the remaining steps needed to get it over the finish line. While Iād been slowly working through a thorough (if content-preserving) rewrite incorporating editorial improvements, more concise language and a glossary with more consistent, cross-linked terminology and using it throughout the PEP, Iām going to just cut that off at the end of the Specifications section where I am now and just submit what I have, so it doesnāt block further work, and of course update the PEP to formally add them as authors.
Iāll still be as involved as I can be supporting them, contributing my relevant expertise and taking on some of the work as time allows, but Iāll no longer be a blocker to continued progress, and theyāll be able to move the PEP forward however needed.
Hereās the remaining priority work items that Iām aware that weāll likely be tackling over the next couple months:
- Review the ātoolā definitions and validation requirements from the perspective of ensuring a conforming implementation is practical and not over-burdensome for maintainers, while still covering the most important points.
- Make the PEP much shorter and more concise (its the longest current PEP, which is a big barrier to it being useful to reviewers and readers)
- Move the non-core content to separate documents and just linking them
- Edit down the remaining core PEP text to be less verbose and more focused
- Make a few further small content tweaks requested by people on the thread since my last big revision:
- Add mention of license subdir in dist-info being not being technically backward compatible with at least theoretical implementations
- Add rejected idea (for now) of a single file concatenating license file text
- Mention that the design is REUSE-compliant and important for SBOM initiatives
- Link mesonbuild/meson-python#270 and Red Hatās existing work as additional evidence of existing implementation of the proposed draft
Additionally, Seth and Karolina will each be providing and incorporating additional valuable input and feedback from their organizations, their many contacts and the community, to ensure the PEP delivers on its stated benefits and is practical to implement and make use of in the real world.
Please give a warm welcome and three cheers for Seth and Karolina for helping pick up the torch that I so carelessly dropped much too long ago, and we look forward to sharing more updates and PRs as they get up to speed on the PEP and the remaining work items. And thanks to everyone in the community for keeping the flame alive in the meantime!
This is great news! Please Seth and Karolina let me know if I can be of any assistance.
I have a vested interest in getting this finished finally as I implemented support for this in Hatchling since the very first draft and I get issues opened sometimes about the string version of license not matching existing schema validators
Hey, thanks!
So actually, as a tool author who has made the effort to fully implement the PEP in a popular build backend and frontend, it would be particularly valuable if you could share some feedback on what specific nominal requirements and recommendations of the specification you implemented and which you didnāt, and which seemed reasonable and straightforward vs. unnecessary or onerous for backends. We donāt want them to be too burdensome for some backends in practice, and reducing complexity there would also help simplify the specification.
Additionally, as a corollary, it would also be helpful to have some feedback on the distinctions made between different types of tools (backends, publishing, installers, etc), and whether each of the various differing requirements for each of them make sense or could be loosened or harmonized, since it seems that could use a bit of a rethink and simplification pass too.
And of course, any further author or user feedback is always helpful, thanks!
Unfortunately a new draft has not appeared 3 months later. I now have lock files to contend w/ while staring down becoming a parent for the first time next month. As such, this may need to wait until June when I am back from family leave or I need to withdraw from being the PEP delegate. But if I do remain the delegate then I am going to require a new draft before I return in June or else Iām going to have the PEP be withdrawn.
Would someone be so kind as to jot down what needs doing? I am willing to do the work but do not have the time to investigate what needs doing.
Just as a heads-up, Hatchling has supported this since forever and setuptools began outputting the License-File
that this PEP introduces 3 years ago.
Either PEP 639 needs to be withdrawn due to inactivity and a new PEP that just clearly outlines what the spec is, or PEP 639 needs an update for the exact spec.
If Hatch and Setuptools are already doing something, then just write a PEP of what that looks like.