Can I rely on dict insertion order as stated in the release notes?

Python Help
1

I was reading Store set items in an orderly manner, on a first-come, first-serve basis and saw:

This shocked me, since I had the impression that this is guaranteed by the language and no longer considered an implementation detail. Much of my code now relies on this.

Searching the net, I found this in the release notes of Python 3.7:

the insertion-order preservation nature of dict objects has been declared to be an official part of the Python language spec.

Following the link, I am not sure whether that’s a correct interpretation of the message from Guido, but I am not a native speaker and I hope, somebody would have challenged that entry in the release notes if it was not factual.

1 Like
2

I think there is a lot of code that relies on this today, and that the python developers are knowing it quite well, and also that the current data structure (that was meant for performance improvement and brought the insertion order preservation “for free”) is perfectly robust.
→ You can certainly rely on this feature for the future.

3

It’s a guarantee. As of Python 3.6, it was an implementation detail for CPython dictionaries (as part of the compact dict representation - its primary goal was performance and memory usage, with iteration order stability being a side benefit); the order of attributes in a class was guaranteed:

As of 3.6, though, other Python implementations were not required to maintain order for other dictionaries; it would be entirely reasonable to have a high performance non-order-preserving dictionary for most purposes, and a lower-performance one for class dictionaries that maintains order. That changed in 3.7, with all compliant Python implementations being required to retain insertion order for all dictionaries.

You CAN rely on this now.

It’s correct. The linked-to post is a very short one whose purpose is to select from among available options, but if you read through the entire discussion thread (or, like me, you were there at the time), it is talking about exactly that. The order of iteration of a dictionary is guaranteed as a language feature.

3 Likes
4

If you need to rely on insertion order on Python 2 or <=3.6, just use a collections.OrderedDict.

Pop quiz: what’s the subtle difference?

5

I’m not sure. There are quite a few differences, and I’m not sure which one counts as “subtle” :slight_smile:

1 Like
6

The question’s even harder then: “which one was I thinking of?”!

7

The difference is the difference! :wink:

I.e. dicts with different order - but otherwise the same content - compare equal, OrderedDicts don’t:

>>> from collections import OrderedDict as odict
>>> dict(x=17, y=23) == dict(y=23, x=17)
True
>>> odict(x=17, y=23) == odict(y=23, x=17)
False
2 Likes
8

But that’s explicitly mentioned in the documentation, so that can’t be the subtle one.

9

I don’t know, but the one I am thinking of is equality checks. Two dicts with the same elements in different orders are equal; two OrderedDicts wouldn’t be.

>>> {1:2, 3:4} == {3:4, 1:2}
True
>>> OrderedDict({1:2, 3:4}) == OrderedDict({3:4, 1:2})
False

But I wouldn’t call this a subtle difference, as this is a significant distinction in how important order is - for a regular dict, it’s retained but not crucial, but for OD, it’s an integral part of the data. This is also why OD has a move_to_end method. Still, maybe it’s what you were thinking of?

2 Likes
10

I take that as a compliment. But I’m not that subtle!

11

Yep spot on. Well done you and @doerwalter!

Anyway I’m not sure “subtle difference” to dict is the best description of these either, but a couple of other gotchas when dealing with OrderedDicts I’ve come across are:

In Python 2, no matter what OrderedDict, any other class or any function tries to do about it, key word args become a kwargs dict, and ever after lose their order:

>py -2.7 -c "from collections import OrderedDict; print(OrderedDict(a=1,b=2,c=3))"
OrderedDict([('a', 1), ('c', 3), ('b', 2)])

Secondly, even in Python 3, care is needed with reprs of subclasses of OrderedDicts:

from collections import Counter, OrderedDict

class OrderedCounter(Counter, OrderedDict):
    pass

assert repr(OrderedCounter({'a': 1, 'b': 2, 'c': 3})) == "OrderedCounter({'c': 3, 'b': 2, 'a': 1})"

The customised __repr__ method from the old recipe in the docs was added for good reason:

class OrderedCounter(Counter, OrderedDict):
     'Counter that remembers the order elements are first encountered'

     def __repr__(self):
         return '%s(%r)' % (self.__class__.__name__, OrderedDict(self))

The ordering of a Counter’s items can be important when doing calculations involving Histograms.