I do ask that people read both PEPs before commenting, even if you have been following the discussions to make sure your knowledge is current.
Also note that neither PEP is final. If a change to one would sway you to another side then it’s okay to bring up. But I do ask that if you want to discuss something very specific to a PEP that you do so on their respective topic. Please consider this topic about choosing between the PEPs, not explicitly trying to improve either PEP.
Honestly, both PEPs make me want to choose the setup.cfg format as it’s the only one with neither quotes, braces or commas
Neither PEP seems to mention anything about the transformation into metadata, apart from the TOML one showing about ~50 LoC for converting the format into a PEP 508 string (and a statement that “tools will need” to do this).
More than that, neither includes anything that sways me one way or the other, so I’m still of the opinion that one fewer specification is better, and we should stick with PEP 508 string literals (and ideally just merge the spec text from 631 straight into 621, but if it needs to stand alone then so be it).
from me too. I find the table format being harder to read for the simple case, and the burden of introducing a new format beside PEP-508 onto the ecosystem gives me the impression that it’s not worth it. I feel like we already have a solution that works in form of PEP-508, so I see no reason to introduce a new one. Both formats require end users to lookup the syntax for anything more complicated, and in such cases IMHO status quo wins.
That would be very cool, though I have no real idea what the syntactic complications of that look like in TOML.
I’m not a fan of the separate file though. Seems too likely to become secretly dynamic. Build tools can still support a separate file in whatever format they like, provided it’s marked as dynamic in this file.
I kind of agree, but that’s more of a comment on PEP 621 (and the TOML format in general) than on the dependency specification aspect alone.
I think it’s a valid point to suggest that some people may find punctuation-heavy TOML less attractive than plain setup.cfg, and for those people, PEP 508 strings minimise the amount of TOML they have to put up with in the common case of dependencies.
But on the other hand, those users may be happy with just using setup.cfg (at least for setuptools projects) for their metadata, and TOML-wanting users can use another build-backend with TOML (eg Poetry). As long as the dependencies (and other metadata) can be dynamic, there are likely to indefinitely be tools which support specifying in other formats.
The whole point of PEP 621 is to standardise on one configuration format. Obviously, setup.cfg won’t go away in the short term, even for metadata, as it will need to be retained for backward compatibility, but suggesting that if people don’t like TOML for dependencies, they can continue using setup.cfg suggests to me that we’re not committed to promoting the new standard
It sounds like you’re saying that the proposal is “PEP 633, and if people don’t like that then just use setup.cfg or some other tool-specific format”. If this was about PEP 621, then I’d say that pretty much says the PEP isn’t worth it. As it’s specifically about dependency specification, letting users throw out the whole of PEP 621 because they don’t like how dependencies are specified is what this topic is trying to avoid…
Sorry, mis-communication on my part. I’m not suggesting that, rather stating the current fact that regardless of the choice between PEP 631 (PEP 508 strings) and PEP 633 (exploded TOML), users can (and in my opinion, likely will) mark dependencies as dynamic and specify dependencies elsewhere in another format.
If anything, I agree with you, and I think there should only one place where dependencies are statically specified. Unfortunately, it’s impossible for dynamic to know the difference between static dependencies defined elsewhere and actual dynamic dependencies.
If that’s your opinion, OK, but in my view if that’s what people do then PEP 621 has failed. Certainly people may stick with setup.cfg, that’s normal (not everyone needs to be an early adopter of a new standard), but once people make the transition to specifying their metadata in pyproject.toml, I’d be very disappointed if they used dynamic to mean “specified somewhere else because I don’t like this format” rather than for its intended use of “cannot be defined statically, needs build-time information to work out the value”.
If folks are viewing dynamic as an escape hatch for “I don’t like what we as a group settled on”, then we’re not better off than status quo where every build tool is already inventing their own format. That’s exactly the problem we’re trying to solve with PEP 621.
If we decide to go ahead with the TOML format and setuptools/flit do not support it, that’s bad and PEP 621 has failed. Similarly, if we decide to go ahead with the PEP 508 format and poetry does not support it, that’s bad and PEP 621 has failed.
I will be honest here: if the PEP 508 format is chosen for PEP 621, Poetry will not be an early adopter of it. I would consider it a regression (both from a user experience and programmatic standpoints) compared to what Poetry provides today and it would drag its development and planned features behind, so there would be no incentives for us to adopt it in this form.
The user base of Poetry is growing each day and having to tell people that they should specify their dependencies the old way from now on would be a major step back for the project and would do more harm than good.
it would drag its development and planned features behind
What planned features would these be and why would using the PEP 508 format not work with them? This could be really good context to mention in PEP 633 as “things doing this would enable” since that can serve as a +ve point for that proposal.
So you’re basically saying that if we don’t agree to the TOML format for dependencies, Poetry will ignore PEP 621? If Python’s key packaging tools, and more specifically one maintained by a co-author of PEP 621, won’t adopt the agreed standard, then it devalues the standards process to a huge extent.
That’s a fairly strong “do it my preferred way or I’ll sabotage the process” ultimatum Did you mean to come across that strongly? I’m hoping I misunderstood, and that you’re actually just saying that you’d need a gradual transition period, rather than that you’d retain your own format indefinitely.
If we can’t agree that the participants in the PEP process will support the standard that comes out of that process, then we should probably simply withdraw all of PEPs 621, 631 and 633 and find something more productive to spend our time on
I only said that Poetry won’t early adopt the PEP, that’s all. It won’t adopt it short-to-mid term because we already have a roadmap that will likely span 1 to 2 years before we even begin to be able to see how the PEP can integrate with Poetry.
Poetry already has a strong user base and any change can not happen overnight once the new PEP is accepted. It will require design, CLI, UX, documentation changes and guidance to make the transition as smooth as possible. And as someone that maintains open source projects I think you know that this is a ton of work that will fall upon us. This will be made even harder if the PEP 508 format is chosen because Poetry has been designed around the TOML format, and since I consider the PEP 508 format to be a lesser format when it comes to dependency specification, yes, I’ll admit the motivation will not be strong to make the transition. It will eventually happen I guess, since not sticking to standards could hurt Poetry in the long run but I’ll reiterate that I would consider it a regression compared to what Poetry provides today.
Clearly and I am not denying that. But for a project of the magnitude of Poetry adhering to a new standard is proving more difficult and time consuming that not doing it so it will take time. A lot of it.
And honestly, honoring the promises, and the roadmap, made to the users of Poetry is more important to me than jumping on this new PEP immediately.
I disagree with the premise of the discussion happening here that if PEP 621 is not universally adopted or not universally and wholly adopted by all backends it has failed. Most of the benefits accrue to the ecosystem as long as some popular backends adopt PEP 621, and some of the benefits accrue even if no existing backend adopts it:
New backends can adopt the format by default — even extremely niche backends — which makes it much easier to just slot in a configuration format.
We can still use PEP 621 in the tutorials on packaging.python.org, and any backend that opts in gets their documentation for free. As long as setuptools adopts the format (and setuptools is looking to add support for specifying configuration in pyproject.tomlanyway and has to pick some format), then we’re certainly no worse off, and we’re providing a leg-up to any new competitor backends who can simply adopt PEP 621 and much of their education work is done for them.
Frankly, I don’t think it hurts anything if poetry doesn’t adopt PEP 621, because poetry's whole raison d’être is to re-invent packaging for Python. It is good that they’re trying to get on board with the standards stuff now, but they seem to be doing fine in their own universe.
With regards to the actual specs, I never thought that the process of designing a spec for the exploded table format was going to change my mind, since I am firmly convinced that PEP 508 is simply not bad enough — even if there were a perfect alternative to it, there was no way it was going to be so much better than PEP 508 as to justify the proliferation of a second way of specifying this information.
Actually seeing the designs, though, I’m more convinced than ever that PEP 508 is the right way to go:
There are multiple ways to specify dependencies: I feel like people will find it confusing that sometimes people use x = {version = ">=3.5"} and sometimes use x = ">= 3.5", if they even understand that the two mean the same thing.
Nearly all the “hard parts” of the parsing are still present — the reference implementation in PEP 633 simply converts the structure into a PEP 508 string (at which point you are at the same point in the parse as PEP 631). If you want to pull information about your dependencies out, you still need to handle all the ">=" operations and parse out the markers. The only things that are easier are pulling out URLs and extras — extras are pretty trivial to parse, URLs maybe somewhat less so.
The hardest parts to write are still mostly present as well. markers is just a freeform text field (as it pretty much must be, since it allows for logical operations). The only advantage that I see PEP 633 has is that specifying URL and VCS dependencies is somewhat nicer — you still need to know that it’s git=... and revision=... (and I would have to look it up every time), but it’s slightly easier to remember that than git+...@revision.
Actually looking at it, I’m not sure I would choose PEP 633 even in the absence of PEP 508 (though admittedly it would be a much closer race). In the landscape as it actually exists, I think PEP 508 is the obvious choice.
I thought that Poetry’s raison d’être was rather to be a single-stop tool for developing, managing dependencies and packaging and not necessarily a wholly different way to do things. In fact, the main reason I have for not using Poetry after considering it several times is that I just don’t want to have to learn a new framework that is mostly incompatible with setuptools/pip.
@sdispater, to be clear: if the exploded table format is chosen for PEP 621, would it be more incentive to move to the new standard? It is a bit strange that the syntax of dependency specification should be the one thing that would slow down Poetry’s adoption of the new standard, especially since code to parse PEP 508 strings already exists.
as it’s the only one with neither quotes, braces or commas
from me too. I find the table format being harder to read for the simple case, and the burden of introducing a new format beside PEP-508 onto the ecosystem gives me the impression that it’s not worth it. I feel like we already have a solution that works in form of PEP-508, so I see no reason to introduce a new one. Both formats require end users to lookup the syntax for anything more complicated, and in such cases IMHO status quo wins.
Did you mean to come across that strongly? I’m hoping I misunderstood, and that you’re actually just saying that you’d need a gradual transition period, rather than that you’d retain your own format indefinitely.
