Adopting/recommending a toml parser?

pip recently switched from pytoml to toml, and flit might well follow it. But I notice that toml has not had a release since 2018, and the pip PR I linked contains a discussion about a newer TOML syntax which Poetry would like to use, but pip would be unable to parse.

@dholth suggested that PyPA should take over maintenance of a toml parser. I think it’s a good idea to ensure there’s one package we can wholeheartedly recommend, because we’ve adopted the format in packaging standards (PEP 517 & 518). I think this should ultimately be a candidate for adding to the standard library, maybe once the TOML specification has reached version 1.0.

The three contenders that I’m aware of:

  • toml (github): last release October 2018, last commit January 2020. Newly vendored in pip.
  • pytoml (github: last release July 2019, last commit July 2019. Marked as unmaintained in the readme.
  • tomlkit (github): last release April 2020, last commit April 2020. Supports newer TOML features than the other two. Extra features to work with comments & layout, beyond simple parse/dump.

Tomlkit is clearly the most actively maintained, and it belongs to @sdispater, who’s also the author of Poetry. However, I imagine that the ability to preserve and manipulate style adds a considerable amount of complexity relative to a library which parses TOML and discards the style. I find it hard to imagine that being added to the standard library, for instance.

4 Likes

I will mention that once TOML hits 1.0 I plan to talk to python-dev about somehow getting a TOML parser into the stdlib. This might also lead to a reckoning over what the future of the stdlib should be. :grin:

3 Likes

I chose pytoml for enscons because its source code seemed easier to read than toml, I understand @takluyver had the same impression. We all chose TOML as a file format because you can have interoperable parsers. It’s not a lot of code.

Doesn’t @pradyunsg maintain one?

Reading through the feature list and implementation I personally find tomlkit to be the best implementation. IMHO ability to pass the roundtrip (read and then write equals the same file) should be offered.

I don’t think so. IIUC @pradyunsg is one of the core members of the TOML spec, but they don’t maintain a reference implementation.

2 Likes