# Adopting/recommending a toml parser? **URL:** \ **Category:** Packaging\ **Created:** [May 3, 2020, 12:16pm UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068 "2020-05-03T12:16:40Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![takluyver](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/takluyver/32/513_2.png) [@takluyver](https://discuss.python.org/u/takluyver)\ **Post date:** [May 3, 2020, 12:16pm UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/1 "2020-05-03T12:16:40Z")
pip recently [switched from `pytoml` to `toml`](https://github.com/pypa/pip/pull/8045), and flit might well [follow it](https://github.com/takluyver/flit/issues/255). But I notice that `toml` has not had a release since 2018, and the pip PR I linked contains a discussion about a newer TOML syntax which Poetry would like to use, but pip would be unable to parse. @dholth suggested that PyPA should take over maintenance of a toml parser. I think it’s a good idea to ensure there’s one package we can wholeheartedly recommend, because we’ve adopted the format in packaging standards (PEP 517 & 518). I think this should ultimately be a candidate for adding to the standard library, maybe once the TOML specification has reached version 1.0. The three contenders that I’m aware of: - [`toml`](https://pypi.org/project/toml/) ([github](https://github.com/uiri/toml)): last release October 2018, last commit January 2020. Newly vendored in pip. - [`pytoml`](https://pypi.org/project/pytoml/) ([github](https://github.com/avakar/pytoml): last release July 2019, last commit July 2019. Marked as unmaintained in the readme. - [`tomlkit`](https://pypi.org/project/tomlkit/) ([github](https://github.com/sdispater/tomlkit)): last release April 2020, last commit April 2020. Supports newer TOML features than the other two. Extra features to work with comments & layout, beyond simple parse/dump. Tomlkit is clearly the most actively maintained, and it belongs to @sdispater, who’s also the author of Poetry. However, I imagine that the ability to preserve and manipulate style adds a considerable amount of complexity relative to a library which parses TOML and discards the style. I find it hard to imagine that being added to the standard library, for instance. ---
**Author:** ![brettcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/brettcannon/32/34895_2.png) [@brettcannon](https://discuss.python.org/u/brettcannon)\ **Post date:** [May 3, 2020, 3:44pm UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/2 "2020-05-03T15:44:52Z")
I will mention that once TOML hits 1.0 I plan to talk to python-dev about somehow getting a TOML parser into the stdlib. This might also lead to a reckoning over what the future of the stdlib should be. 😁 ---
**Author:** ![dholth](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dholth/32/952_2.png) [@dholth](https://discuss.python.org/u/dholth)\ **Post date:** [May 3, 2020, 4:57pm UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/3 "2020-05-03T16:57:51Z")
I chose `pytoml` for [enscons](https://github.com/dholth/enscons) because its source code seemed easier to read than `toml`, I understand @takluyver had the same impression. We all chose TOML as a file format because you can have interoperable parsers. It’s not a lot of code. ---
**Author:** ![steve.dower](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/steve.dower/32/56_2.png) [@steve.dower](https://discuss.python.org/u/steve.dower)\ **Post date:** [May 4, 2020, 7:54am UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/4 "2020-05-04T07:54:36Z")
Doesn’t @pradyunsg maintain one? ---
**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\ **Post date:** [May 4, 2020, 11:39am UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/5 "2020-05-04T11:39:30Z")
Reading through the feature list and implementation I personally find tomlkit to be the best implementation. IMHO ability to pass the roundtrip (read and then write equals the same file) should be offered. ---
**Author:** ![uranusjr](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/uranusjr/32/103_2.png) [@uranusjr](https://discuss.python.org/u/uranusjr)\ **Post date:** [May 4, 2020, 12:13pm UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/6 "2020-05-04T12:13:18Z")
I don’t think so. IIUC @pradyunsg is one of the core members of the TOML _spec_, but they don’t maintain a reference implementation. ---
**Author:** ![bobfang1992](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bobfang1992/32/3025_2.png) [@bobfang1992](https://discuss.python.org/u/bobfang1992)\ **Post date:** [June 19, 2020, 11:12pm UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/7 "2020-06-19T23:12:28Z")
Hi I just want to mention that I have wrapped around a C++ single header library to provide a fast TOML: parser in Python. The repo is here: [https://github.com/bobfang1992/pytomlpp](https://github.com/bobfang1992/pytomlpp) ---
**Author:** ![takluyver](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/takluyver/32/513_2.png) [@takluyver](https://discuss.python.org/u/takluyver)\ **Post date:** [June 20, 2020, 9:13am UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/8 "2020-06-20T09:13:24Z")
Thanks Bob. For my purposes, the ease of managing a pure Python package outweighs the performance benefit of a compiled parser, but it’s great to have a compiled option if people need that performance. ---
**Author:** ![bernatgabor](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bernatgabor/32/3003_2.png) [@bernatgabor](https://discuss.python.org/u/bernatgabor)\ **Post date:** [June 20, 2020, 9:19am UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/9 "2020-06-20T09:19:07Z")
If pure speed is what one is looking for there’s [https://github.com/samuelcolvin/rtoml](https://github.com/samuelcolvin/rtoml) too. I tried to compare pytomlpp to it, but did not manage to make pytomlpp work on MacOs. ---
**Author:** ![pitrou](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pitrou/32/28_2.png) [@pitrou](https://discuss.python.org/u/pitrou)\ **Post date:** [June 21, 2020, 3:56pm UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/10 "2020-06-21T15:56:14Z")
> [@bobfang1992](#): > > The repo is here: [https://github.com/bobfang1992/pytomlpp](https://github.com/bobfang1992/pytomlpp) Interesting, but [toml++](https://marzer.github.io/tomlplusplus/) uses C++17 – an extremely recent version of the C++ language – , while PEP 7 mandates the use of [“C89 with several select C99 features”](https://www.python.org/dev/peps/pep-0007/#c-dialect). So I’m afraid integrating `pytomlpp` into the stdlib would be a no-no, and would remain so for at least the next 10 years. As for speed, well, the dominant use case for TOML is configuration files, and I don’t think those are usually performance-critical (unless you’re batch-processing thousands of them, perhaps 🙂 ). ---
**Author:** ![bobfang1992](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bobfang1992/32/3025_2.png) [@bobfang1992](https://discuss.python.org/u/bobfang1992)\ **Post date:** [June 22, 2020, 7:01am UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/11 "2020-06-22T07:01:20Z")
Hi, thanks for introducing rtoml to me. Did not see this one before. I am trying to resolve the issue on Mac OS recently and once done I will provide a benchmark. Thanks! ---
**Author:** ![bobfang1992](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bobfang1992/32/3025_2.png) [@bobfang1992](https://discuss.python.org/u/bobfang1992)\ **Post date:** [June 22, 2020, 7:05am UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/12 "2020-06-22T07:05:46Z")
Hey thanks for the reply. Yeah I never expected it will be part of stdlib, as it also requires pybind11 which is not part of stdlib. My aim is to provide an alternative route when perforemance is sensitive. As for apps that care about toml parsing performance, I worked in a quant fund where toml is used for configuring trading strategies and we have thousands of them and when the app starts it reads in all of the toml configs which take quite a while. This is the mian montive for me to start this project in the first place. I agree this is a quite niche scenario but pytomlpp provides other benefit as well I think – fully compatiable with toml1.0rc1 and stick to only python native types for example. Thanks! ---
**Author:** ![bobfang1992](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/bobfang1992/32/3025_2.png) [@bobfang1992](https://discuss.python.org/u/bobfang1992)\ **Post date:** [June 22, 2020, 7:08am UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/13 "2020-06-22T07:08:09Z")
Hi thanks! Yeah I agree this should not be part of the stdlib, but just thought if I mention it here then when people need a performant toml parser they can discover it! ---
**Author:** ![EpicWink](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/epicwink/32/17968_2.png) [@EpicWink](https://discuss.python.org/u/EpicWink)\ **Post date:** [June 22, 2020, 10:23pm UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/14 "2020-06-22T22:23:30Z")
What’s the easiest way for allowing the user to choose between packages which provide the same interface without code change? Is that possible? Is it a security risk? ---
**Author:** ![sinoroc](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/sinoroc/32/1607_2.png) [@sinoroc](https://discuss.python.org/u/sinoroc)\ **Post date:** [June 23, 2020, 8:35am UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/15 "2020-06-23T08:35:49Z")
This might be [`Provides-Dist`](https://packaging.python.org/specifications/core-metadata/#provides-dist-multiple-use), in theory only though, since there doesn’t seem to be any tool that implements it. Discussed here, recently: - [Options to build the same package in different ways (with different build dependencies)?](https://discuss.python.org/t/options-to-build-the-same-package-in-different-ways-with-different-build-dependencies/4458) - [Idea: selector packages](https://discuss.python.org/t/idea-selector-packages/4463) ---
**Author:** ![hroncok](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/hroncok/32/18696_2.png) [@hroncok](https://discuss.python.org/u/hroncok)\ **Post date:** [June 25, 2020, 7:46am UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/16 "2020-06-25T07:46:09Z")
> [@takluyver](#): > > But I notice that `toml` has not had a release since 2018 There is now 0.10.1 from May 14, 2020. ---
**Author:** ![dfundingsland](https://avatars.discourse-cdn.com/v4/letter/d/8edcca/32.png) [@dfundingsland](https://discuss.python.org/u/dfundingsland)\ **Post date:** [December 9, 2020, 4:29am UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/17 "2020-12-09T04:29:20Z")
It seems that date is getting close. As of October 7, there is a pre-release candidate (1.0.0-rc.3) for TOML. ---
**Author:** ![fdrake](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/fdrake/32/19_2.png) [@fdrake](https://discuss.python.org/u/fdrake)\ **Post date:** [December 9, 2020, 5:36am UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/18 "2020-12-09T05:36:07Z")
Again, as the time approaches… I’m looking forward to seeing what @brettcannon is thinking on the topic of the standard library. While “batteries included” served us well for many years, we’re beyond it being an advantage now. ---
**Author:** ![brettcannon](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/brettcannon/32/34895_2.png) [@brettcannon](https://discuss.python.org/u/brettcannon)\ **Post date:** [December 9, 2020, 7:10pm UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/19 "2020-12-09T19:10:15Z")
I would also like to here what @brettcannon thinks on the topic of the standard library. 😉 (Seriously, it fluctuates so I don’t even know where I’m going to land after the topic is discussed.) ---
**Author:** ![pganssle](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/pganssle/32/245_2.png) [@pganssle](https://discuss.python.org/u/pganssle)\ **Post date:** [December 10, 2020, 8:09pm UTC](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/20 "2020-12-10T20:09:23Z")
> [@fdrake](#): > > While “batteries included” served us well for many years, we’re beyond it being an advantage now. I agree with this sentiment in general, but it really is hard for stuff like `setuptools` and `pip` to take on dependencies, which is why they vendor everything. Considering the universal nature of `pyproject.toml`, it would be very nice to be able to parse these things without taking on a dependency. Of course, it’s not likely that we’ll drop support for Python 3.9 for many years from now, and maybe by then the need for `pip` and `setuptools` to vendor all their dependencies will have been considerably alleviated, so we’ll gain very little advantage to it. That said, the other time I find it convenient to work without any dependencies are for little scripts where I essentially replace what would have been a bash script with a Python script — I tend to use _only_ the standard library for this sort of thing, since it’s inconvenient to declare dependencies and build a virtualenv for single standalone `.py` files. I can imagine a TOML parser / emitter will increasingly be valuable for these sorts of things as well, so I’m +1 on including it, even from someone who is mostly in the “kernel / minimal python” camp. [Next page](https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068.md?page=2)