Clarify whether `tomllib` preserves table key order from input

Ideas
1

Currently the tomllib documentation doesn’t say anything about dict key order. FWIU TOML does not guarantee any particular order:

Key/value pairs within tables are not guaranteed to be in any specific order. (from TOML: English v1.1.0)

However, neither does JSON, yet the json documentation specifies that the order is preserved:

Note: This module’s encoders and decoders preserve input and output order by default. Order is only lost if the underlying containers are unordered.

I’m wondering if we can give the same guarantees for tomllib. And if we don’t want to do that, perhaps we should make it explicit before users start relying on that.

2 Likes
2

I wouldn’t add a guarantee like this, if a future toml version specifies something about key order that’s incompatible with a guarantee provided here, what would the transition plan be for the standard library’s behavior?

There’s already a lack of guarantee here, neither the parser nor toml guarantee it, users relying on it already shouldn’t be.

3 Likes
3

Then we would document the change and likely propose a way to select the version to expose. Or say “we only expose TOML vX.Y”.

Now, since Python’s dicts are guaranteed to preserve insertion order, then we are implicitly guaranteeing this as well (AFAIR, we’re using dicts for that? and if not, then we can say that we don’t guarantee anything as well; it’s better than letting the user assume something). And if future versions of TOML change the specs by saying “TOML guarantees alphabetical ordering in the keys”, then we would just reorder the dict before returning the result, along with a versionchanged directive, possibly options to preserve parsing/insertion order, and so on.

3 Likes
4

We aren’t implicitly guaranteeing it. There is no written guarantee that the objects inserted into the dict by the parser are inserted in streaming order. The dict order guarantee on it’s own is not enough to implicitly guarantee this, and it is only the result of that guarantee + the implementation details of the parser.

I would rather leave this as something not guaranteed. Applications that need the order guaranteed shouldn’t be using toml tables, instead using lists or another format.

Adding guarantees over what a format provides creates the potential for diverging behavior in application rewrites, as well as inter-language tooling. Rust’s toml crate does not add this guarantee, and there are tools for python actively written in Rust parsing toml files for python projects.

3 Likes
5

I don’t know why we should look at what other languages do. It’s not about TOML, it’s about tomllib so if we want to expose this implementation detail it’s fine. If the insertion order is not the same as the parsing order, we can simply say that we don’t guarantee the ordering. Since TOML specs explicitly say it, we can also restate it.

6

tomllib diverging from what other languages guarantee would create a a situation where if someone relies on that divergence, they can’t use toml as a cross-language format. the example I’d give here is that tools like uv and ruff are written in rust, and are intended to be able to be used as drop-in replacements for other tools that read pyproject.toml adding that constraint after the fact now also constrains drop-in replacements, both of tomllib (implementations of tomllib in non-cpython implementations of python) and in other tools that consume it.

As part of the standard library, I don’t think that tomllib shouldn’t guarantee anything TOML doesn’t, it’s the default tool people will reach for in python for reading TOML files, and the file format doesn’t guarantee it.

4 Likes
7

And thus, it’s better to lay down that the order is not guaranteed to be preserved then. As mentioned, we already do it for JSON (that is, guarantee something that the specs don’t). We can also do it for tomllib (if this makes kind of implementation details makes sense). I don’t know if JSON explicitly says that there is no guarantee though. But since TOML explicitly says it, then it’s better to also restate it so that people don’t make wrong assumptions.

EDIT: maybe I wasn’t clear in my previous posts but I’m neither for nor against documenting the key ordering. We can say that it’s an implementation detail and not guaranteed by the specs, or simply state that there is no key ordering guarantees like the specs. At least, I want the docs to be clear on that (as it’s burried quite well in TOML specs).

tomllib diverging from what other languages guarantee would create a a situation where if someone relies on that divergence, they can’t use toml as a cross-language format

Honesly, it’s not really an issue if we clearly document that it’s what Python offers and not what the specs guarantee. The question is whether to document it loudly as an implementation detail, or state that there is no table key ordering that is guaranteed.

If we want to prevent future complains, we can just restate the TOML docs about ordering guarantees (we already say that we return a dict and it’s easy to assume that your TOML document will be simply parsed as a dicts with multiple levels and thus one could expect the implementation to preserve insertion order).

For JSON, I think we mentioned the order preserving because of the sort_keys=False parameter (so that users understand that they can also force another order).

I don’t think that tomllib shouldn’t guarantee anything TOML doesn’t,

Sometimes, the stdlib offers more than just plain specs. For instance, all the HTTP-related modules are meant to be really RFC compliant, but sometimes they either restrict or offer more. So it can guarantee more things than what is expected (usually it’s behind a gate though).

I’d give here is that tools like uv and ruff are written in rust, and are intended to be able to be used as drop-in replacements for other tools that read pyproject.toml adding that constraint after the fact now also constrains drop-in replacements

Strictly speaking, having an additional constraint doesn’t make your tool less usable. If you decide to rely on something that is guaranteed by the implementation but not by the specs, it’s the user’s decision. And it would be wrong to assume that a simple drop-in replacement would be correct. We are still compliant with the specs strictly speaking (fixing an ordering in the implementation doesn’t conflict with “there is no order guaranteed”).

To summarize: I’m ok for restating the “no ordering is guaranteed”. If we once decide to extend the interface and change the behavior, it’s also fine (e.g., adding an “sort_keys” parameter in tomllib.load()), as long as we are clear on this “extension”.

1 Like
8

How would a typical use-case for a TOML key order guarantee look like?

I never needed it, but I could imagine that the JSON guarantee might be useful when the JSON format is used for a program to program communication. When the source program has a fixed key order, the recipient could rely on it.

OTOH, TOML files are mostly created by humans to store some config data. I don’t think it’s a good idea to expect or even to dictate the key order. And if an ordered configuration data is required, a list defines it clearly.

9

It makes it so that if you read and write a file, you get an empty diff, which is very nice.

4 Likes
10

There’s no writer for TOML in the stdlib, so that’s not a concern here.

3 Likes
11

Arguably, if there were a writer for toml in the stdlib, and one of its objectives were to actually preserve empty diffs in round-trips, the current implementation of tomllib.load(s) would be insufficient despite dict ordering, because TOML explicitly supports comments, but those are discarded currently.

1 Like
12

The same as it is right now (i.e. IIRC people are still arguing), given that tomllib is currently implementing TOML 1.0.0 (as stated on top of tomllib doc).

Sure. However, this is not stated explicitly (and to be honest, the TOML spec is not exactly 100% precise there either), so there is space to mistakes. Especially given that the order-preserving behavior is something ubiquitous to Python.

Funny you should mention that, given that I’m asking this question precisely because uv is using a TOML file where dictionary key order is significant, so the answer here pretty much determines whether we can use tomllib to parse this file, or if we need to implement a separate “extended” TOML parser that supports this feature.

Looking at the relevant issue (Document order-preserving dictionary output in json), looks like people actually found the behavior helpful.

Depends on the format. Sometimes an ordered dictionary is the simplest and least confusing thing you can give to a user. Compare, say:

[ordered-thing]
first = "..."
second = "..."

vs.

[[ordered-thing]]
key = "first"
value = "..."

[[ordered-thing]]
key = "second"
value = "..."

Yes, the second form is 100% TOML-compliant but it’s a lot more typing for a human to do, and I dare say it’s less readable. And I really can’t think of a much better alternative.

I disagree here. We are explicitly recommending tomli-w to be paired with tomllib, at the very top of the tomllib documentation.

2 Likes
PEP 825: Wheel Variants, Package Format (split from PEP 817)
13

That sure sounds like a bug in uv, not something to support. The format doesn’t specify that you can rely on key order, therefore it’s a mistake to rely on key order.

I’d rather explicitly document that lack of guarantee because it’s not the job of a format parser to do something the format doesn’t, that’s exactly how you get into problems with incompatibility between implementations, and something that uv shouldn’t be doing either.

14

Actually, on opening it, that seems like an issue with the wheel variant spec? Why is ordering in the variant spec relying on something toml doesn’t provide instead of structuring it appropriately for format?

15

The spec doesn’t define that particular file format, uv does as its implementation detail.

16

If that’s an implementation detail of uv and not at all relevant to interoperability as you claimed in the other thread, then I don’t see how it’s relevant to the python standard library, or why you would link it and say it’s what prompted you to discuss this.

In any case, I don’t think it’s proper to layer a guarantee ontop of a format at the level of a parser that potential people on the other end of the wire can’t guarantee from the format alone. It should be possible to get the “right” order using any compliant parser, even if that means application logic after parsing.

Add an “order” field to values that need to be ordered, or use a list, or use something other than toml if toml isn’t suiting your needs.

17

I don’t think it’s anything about changing the implementation. It’s about whether we document the implementation details or not.

In any case, I don’t think it’s proper to layer a guarantee ontop of a format at the level of a parser that potential people on the other end of the wire can’t guarantee from the format alone

It’s not uncommon in the docs. We do sometimes relax/slightly differ from RFCs for instance when it relates to HTTP/email modules.

And why would people not be interested in knowing that the stdlib parser does more than necessary? If we add a stronger guarantee because from CPython PoV it’s easier for us to do it as is, then it’s fine. I wouldn’t spell it as a language guarantee since dict ordering is (AFAIR) only guaranteed as CPython implementation detail and not a language specification. So using tomllib with an alternate implementation may give different outputs (if their dictionaries are not ordered by default). So, better indicate that on the docs. Then users can use our parser for this instead of trying to rely on another parser (now, I still don’t know if insertion order is preserved even on CPython for tomllib so this needs to be checked).

1 Like
18

Users using TOML as a format shouldn’t be relying on something the format doesn’t guarantee. Relying on it means their files behave differently in a material manner when parsed by different format compliant libraries.

Guaranteeing it in CPython closes off future changes when the format provides no reason to do so, and may lead people to not structure their data appropriately for the actual format they are using.

Such a guarantee can only hurt future users that relied on it, so if you want to note that it’s an implementation detail and should not be relied upon, that’s fine, but guaranteeing it isn’t in the benefit of maintainers or users.

19

Speaking from my experience with other file formats, I think guaranteeing this is a bad idea.

tar and zip are both examples of cases where various people using the format didnt strictly comply to one set of standards, and the formats are hazardous to use (yet ubiquitous anyhow) as a result.

While it might seem easy to guarantee something because it’s the internal behavior, at the end of the day, the internal behavior should be just that, internal.

I don’t want to have to keep replacing file formats just because the authors of the formats didnt put in features that prevent this (like saying that all parsers must shuffle the order of keys) that have additional costs to them because other people layered on additional functionality into the format, but continued calling it the same as the prior format.

Relying on key order is not part of toml, and not something a parser of toml should guarantee, lest they encourage users of the parsers to rely on that guarantee and make the use incompatible with the actual toml spec (They would break if the key order were different)

4 Likes
20

I’d suggest that someone adds a comment to the PR noting that the TOML format doesn’t guarantee that order is preserved, so uv shouldn’t be making that assumption (even if the library they are using does preserve order).

I agree that PR isn’t relevant to this discussion.

Given that TOML doesn’t guarantee that order is preserved, I’d explicitly note in the tomllib documentation that order isn’t guaranteed to be preserved. This could be a footnote to the dict entry in the conversion table in the docs, saying something like

The TOML spec does not guarantee that the order a table is written in the TOML file will be preserved. Therefore, when using a dict read from a TOML file, the user must not assume that the dictionary key order has any particular meaning.

In answer to the question in the OP, I do not think we should guarantee order, regardless of the fact that the json module does.

3 Likes