Copy a dictionary, except some keys

Ideas
1

How common is the need of getting a copy of a dict, except some of its keys?

It happens to me quite frequently that I need a similar dict to what I have, but without some of its keys. Of course I can’t just remove the keys, as other parts of the code still uses the full dict.

I end up doing stuff like this:

new_dict = {k: v for k, v in old_dict.items() if k not in {'key1', 'key2'}}

What if we could do just…?

new_dict = old_dict.copy(avoid_keys=('key1', 'key2'))

Alternatives:

new_dict = old_dict.copy(avoid=('key1', 'key2'))  
new_dict = old_dict.copy(except=('key1', 'key2'))

Thanks!

2

I don’t know if this is common enough to warrant a method on dict, but I would also consider drop, ignore, and ignore_keys as argument names.

1 Like
3

You can make your dict comprehension more concise by using the keys() dictionary view:

new_dict = {k: old_dict[k] for k in old_dict.keys() - {'key1', 'key2'}}

These act like sets, so you can use set subtraction.

I agree with Brett that I can’t see this being common enough to complicate the dict.copy() method.

1 Like
4

The first version of PEP 584 proposed that dicts could support
the - subtraction operator to do precisely this:

new = d1 - d2

would return a new dict based on a copy of d1 except for the keys of d2.

PEP 584 has currently dropped that operator to concentrate only on dict
addition, but I expect that somebody will propose dicts support the full
set API including subtraction.

5

I expect this to be really slower, as you’re accesing the old dict for every key! I’ve run both snippets in timeit and this shows a ~33% slowdown.

6

Of course, yes. I was aiming for conciseness, but it’s not that big an improvement.

7

I don’t think this is common enough to warrant adding a method or extending the abilities of an existing method of dict.

I can recommend the funcy module for good, well-tested implementations of many such basic operations (not only for dicts). For this case, funcy.omit() does exactly what you want:

funcy.omit(old_dict, 'key1', 'key2')
8

How odd.

I would have taken the existence of funcy.omit as evidence that this
requested feature is common enough to warrant adding a method, rather
than the opposite.

9

This is the way I see it: The funcy library has over a hundred such helpers. If we added many or all of them to Python’s built-ins and stdlib, IMO we’d be doing the language a disservice by increasing the number of basics anyone has to learn to read and maintain Python code. On the other hand, for a specific codebase, deciding to use funcy consistently can be beneficial; I’ve seen this done with good results.

4 Likes
10

I have considered this possibility at multiple occasions, with the slight difference that i wished for the existence of a subdict(keys) method (keeping items instead of dropping them). We basically could represent different operations on dicts like difference (the case proposed by OP), intersection (my case), and also union… this could be effectively represented by some operations on sets of keys (respectively -, *, +), but the ideal syntax is unclear as you want to create/modify dicts while operating on sets of keys…

Whatever, there is another fact to take into account : there is no speed up gained from creating a subset of a dict “all-at-once” compared to iteratively creating it. For example, the update method is no (significantly?) faster than updating the elements one-by-one. This is linked to the hashmap data structure properties. (Please correct me if I’m wrong.)

Therefore any additional method would be performing under-the-hood the creation of a list of keys and, subsequently, the creation a new dict or the modification of an existing one by iterative addition or removal of items.

11

I know i am unearthing this topic, yet I noticed the fromkeys method is a static one, and I think it could take an original dict as an argument, and create an output dict with the provided keys associated to values taken from the original one. This could help clean bits of user code without adding methods. It could be used in conjunction with the operations on sets of keys. It also fits well with the intuition coming from the name of the method, I think.

12

For me this is a frequent use case. Also, Pydantic supports both include= and exclude= args for this purpose (and I like the naming).

1 Like