Some developers (eg. the author of flake8) have been reluctant to embrace pyproject.toml due to syntax issues which looked likely to be addressed in TOML 1.1. However unless something changes, it seems unlikely that TOML 1.1 will ever see the light of day given the lack of resources to go through the pending PRs.
What does it mean for pyproject.toml if TOML never moves beyond 1.0? If the owner of toml-org remains unresponsive, should the TOML specification repository be forked and if so to where? Should the development of TOML, given it was (as far as I know) popularised by its usage in pyproject.toml, be moved under an appropriate Python packaging group eg. PyPA?
I guess it depends on what’s actually missing… JSON hasn’t changed ever, and no-one worries about that. tbh I kind of had the impression that TOML was like… done. Can you (or someone) put together some list of what the big motivating features for TOML 1.1 would be?
Huh, skimming over those, I think the situation is that Pradyun actually can do a 1.1 release; it’s just been bottlenecked on him finding the time? If the only issue is he doesn’t have the admin bit to rearrange repos or add new maintainers, that’s an issue on some time-frame, but probably not urgent and probably something where we can let him decide how he wants to handle it?
I don’t see how time can solve @pradyunsg’s lack of admin permissions to add new maintainers since it requires the owner of toml-org to make changes. I also think that it’s rather unfair to expect one person to handle all the maintenance alone particularly if they have other important projects to look after. Ideally the toml-org owner would respond, but it appears he has moved on and isn’t responding.
One thing I think it’s important to mention is that while JSON may not have changed for a while, it certainly had a number of releases before it reached a state of completion. Similarly with YAML 1.2 (YAML™ Specification Index). I think it is reasonable that TOML requires further releases before it is complete.
“JSON was first presented to the world at the JSON.org website in 2001. A definition of the JSON syntax was subsequently published as IETF RFC 4627 in July 2006. ECMA-262, Fifth Edition (2009) included a normative specification of the JSON grammar. This specification, ECMA-404, replaces those earlier definitions of the JSON syntax. Concurrently, the IETF published RFC 7158/7159 and in 2017 RFC 8259 as updates to RFC 4627.” (https://www.ecma-international.org/wp-content/uploads/ECMA-404_2nd_edition_december_2017.pdf)
It’s also worth mentioning that some people still do think it would be better to make more changes (such as allowing trailing commas). The fact that no one changed a format to make it better doesn’t mean it’s the best it can be, any more than the fact that no one repaired a run-down house means it’s in pristine condition.
Interesting analogy. I wasn’t aware time and the elements could wear out digital stuff.
The fact that no one changed the TOML spec recently suggests either the author either thought the paid of further change want worth the cost, or tired of the project and moved onto other things. Changing it once stabilized (and presumably available in multiple environments) might not be worth the compatibility problems.
Well, if you prefer it can be “just because someone built half a house and stopped before finishing the roof and floor doesn’t mean it’s habitable”. I just mean that just because no one’s fixing something doesn’t mean it ain’t broke.
Building further on that analogy, I’d like to point out that my family house was never “finished” per the original drawing. That doesn’t mean it’s uninhabitable — on the contrary, I’m grateful that it was never completed because it would be a complete overkill.
I’m not saying that TOML is perfect but I’m not convinced that making major changes for what I understand it’s mostly convenience is worth breaking backwards compatibility. Two important points are:
We’ve “just” managed to get tomllib in, and some projects still rely on the ancient unmaintained toml library. TOML 1.1 would mean that many projects suddenly need to switch back to an external implementation, and then start using tomllib again in a future version of Python.
Versions are hard. People are likely to end up using TOML 1.1 feature unknowingly and breaking stuff that still uses a TOML 1.0 parser.
@mcarans I’m not sure what you’re concerned about here and why you’re signal boosting this thread in multiple places now.
Then pyproject.toml won’t evolve beyond TOML 1.0. I don’t think TOML 1.0 is severly lacking as many speculate/opine – it has limitations and tradeoffs but that’s literally every technology ever and it’s fine for digital infrastructure to not evolve stupidly quickly anyway.
Maybe. This forum isn’t the right place to discuss that.
No. TOML is not a PyPA or Python packaging thing – it’s a language spec that is maintained by a completely separate group. It is a configuration file format, and isn’t tied to PyPA or (for that matter) any specific programming language).
There is json5, which is one groups attempt at a new iteration of JSON.
That being said, I think TOML in it’s current iteration is fine for our use cases. Maybe there’s some things that could be better, and if so hopefully it continues to improve, but if it doesn’t, then I think it’s still perfectly adequate.
I only raised it in one place - on the issue about finding additional maintainers for TOML where I thought some watching that issue might be interested. I understand you’d prefer not to turn it into a bigger discussion.
I think that could have been an argument for using a well established existing format for pyproject rather than a less well known still evolving format, but anyway I really hope the improvements make it into TOML and then into pyproject.toml. Thanks for making the milestone which makes it much easier to track what is going on.
I don’t think it’s particularly useful to re-litigate a decision made in 2016, unless we think that the current situation is problematic enough to warrant the ecosystem wide pain of migration . However, I thought maybe it would be fun to poke at the question a bit and see how well that decision held up.
At the time the only really serious contenders we had were TOML, JSON, configparser flavored INI, YAML, and Python Literals (via ast.literal_eval()), and most of them were reviewed by @njs, and out of that review it became obvious that all of the existing formats had serious short comings, and the state of TOML, even in 2016 at v0.4.0, was in a much better position for our use cases.
AFAIK, TOML is the only one of those, since 2016, to have evolved at all, so all of the shortcomings that existed in 2016 for those other formats still exist, but TOML has evolved to get even better in the intervening 7 years.
It is possible that TOML will now end up in the same state that the other options at the time were in, and end up effectively frozen and in a “finished” state… and that’s just fine? Maybe there are some situations where a better format could be more readable, but we haven’t found that in the packaging space yet, and if it’s bad enough, then those projects don’t have to use pyproject.toml and can use their own file in whatever format they prefer.
Hopefully that doesn’t happen, and TOML continues to evolve to support more use cases in clean ways, but I think, even if we were remaking this decision today, given the options we had then, I think we would still make the same decision.