Query multiple dictionary keys

Ideas
1

In python today, you can query a dictionary like so, d['some_key'] but you can only query one key at a time. If you attempt to query the dictionary using d[['key_a','key_b']] it returns with
TypeError: unhashable type: 'list'

I propose that we add in so d[['key_a','key_b']] works the same way {key:value for key,value in d.items() if key in ['key_a','key_b']} works now, allowing users to quickly get multiple keys from a dictionary.

I believe this is an intuitive way to query multiple keys, it matches the behavior in Pandas DataFrames for querying multiple columns, so many python users already think in this way, making me think it is a natural way to do this operation with less typing than a dictionary comprehension.

2

Could you please clarify the following: the comprehension {key:value for key,value in d.items() if key in ['key_a','key_b']} creates a new dict (in an ineffective way, but let’s stay focused). Is this what you mean by “quering multiple keys”?

3

I don’t like this. I frequently use dictionaries with tuple keys, a legitimate and useful construct in Python today. Having easily confused syntax (just a typo away!) that provides completely different semantics sounds very frustrating.

Consider:

my_dict = {
    "foo": 0,
    "bar": 1,
    ("foo", "bar"): 2
}

print(my_dict[["foo", "bar"]]) # {"foo": 0, "bar": 1}
print(my_dict["foo", "bar"]) # 2

Sure, you may not be mixing key types like this in a single dict, but it’s going to be very easy to use the wrong access syntax and get confusing errors.

7 Likes
4

I would prefer to use a slice with a tuple step to get a sub-dict from a dict. Something like:

In [371]: class SubDict(dict):
     ...:     def __getitem__(self, what):
     ...:         if not isinstance(what, slice):
     ...:             return super().__getitem__(what)
     ...:         if what.start is not None or what.stop is not None:
     ...:             raise ValueError('SubDict slices must not have start or stop')
     ...:         return type(self)(filter(lambda i: i[0] in what.step, self.items()))
     ...: 

In [372]: d = SubDict({'a': 1, 'b': 2, 'c': 3})

In [373]: d[::('a', 'b')]
Out[373]: {'a': 1, 'b': 2}
5

What’s wrong with just writing a helper?

def get_multiple(mapping, keys):
    # Make keys a set for quicker membership checks
    keyset = set(keys)
    return {k: v for k, v in mapping.items() if k in keyset}

I don’t think this is a common enough need to justify built in syntax. But writing a helper in your project is easy, so the lack of built in syntax isn’t a major issue.

Personally, I’d prefer to return a list of values, rather than a sub-dictionary, which is another reason why picking one behaviour to “bless” as having built in syntax is questionable, but that’s a separate matter.

10 Likes
6

I’m not sure if you’re aware, but there’s an unfortunate quirk whereby:

d[a, b]

is equivalent to

d[(a, b)]

Therefore if your dict had tuple keys (which is perfectly reasonable), it would be impossible know if: d[a, b] means return {a: d[a], b: d[b]} or d[(a, b)].

You should add a helper function.

(I think your helper should iterate like {key: d[key] for key in ['key_a', 'key_b']} for efficiency and probably just return a tuple as Paul suggested.)

1 Like
7

@NeilGirdhar But they’re suggesting d[[a, b]].

1 Like
8

Oh, my mistake!

9

This could certainly be done, but you’d have to subclass dict to do it. That means you won’t be able to use this on someone else’s dictionaries, but OTOH, it’s something you can do in your own code without needing anyone else’s approval :slight_smile:

class multidict(dict):
    def __getitem__(self, key):
        if isinstance(key, list):
            return {k:self[k] for k in key}
        return super().__getitem__(key)


>>> stuff = multidict({"a": 1, "b": 2, "c": 3})
>>> stuff[["a", "b"]]
{'a': 1, 'b': 2}

Note that I’ve gone for a slightly different implementation than you had. The way I’ve done it here, the results will be returned in the order you requested them; you had it return them in the same order they’re in the original dictionary. More importantly, this version will report errors like stuff[["a", "d"]] instead of just returning the ones that are present. But if you specifically want the behaviour of ignoring unknown keys, then go with the original version - this is your code and you can do whatever makes sense for you.

Incidentally, what you’re asking for here could perhaps be better described as a set-like intersection operation. This was discussed a little while back as part of PEP 584 - the conclusion at the time was that the union operator was definitely worth adding, but the others offered less value and were not part of the proposal, without outright rejecting them. So perhaps the right way to proceed would be to seek this syntax:

>>> d ^ {'key_a', 'key_b'}

This is a notation supported by some other languages, but not - so far - Python. I think if you want to have this as a part of the core language, you’ll get more support for this than with a variant of subscripting, and it’ll achieve the same goal.

5 Likes
10

@Rosuav Another difference is that your result has keys as they were requested, not as they appear in the original dictionary. For example, for multidict({1: 2, 3: 4})[[True]] you return {True: 2} instead of {1: 2}.

1 Like
11

Ah that’s a good point. Again, it could be something that you might want either way.

12

Lots of really good points here. So for doing tuple keys,
d = {'a': 1, 'b': 2, ('a','b'): [1,2]}

is valid, while

d = {'a': 1, 'b': 2, ['a','b']: [1,2]}

raises a TypeError.

So when we query the valid dictionary,

d[('a','b')] == [1,2]

and the proposal would create the behavior:

d[['a',('a','b')]] == {'a':1, ('a','b'): [1,2]}

Python already does not allow lists as keys, and this behavior should be maintained, because
d = {'a': 1, 'b': 2, ['a','b']: [1,2]}

raises TypeError: unhashable type: 'list' which will be maintained with this proposal.

So the user still needs to stay aware of the types they are using when constructing a dictionary, but that’s true in all programming. However, it is easy to write a helper function so that’s probably good enough for most use cases.

13

FWIW, there was a similar discussion in 2022: User-defined slice-like objects, and slicing mappings

14

d[[*keys]] will become unclear and difficult to read in nested expressions, it is a no-go to me.

I would prefer something like d1 & d2 to return a subdict by intersection (with values taken from d2), and/or d @ keys to return a list of values.

15

I would prefer d1 & s1 to return a subdict by intersection with a set - that way, there’s no way to be confused as to where the values come from.

4 Likes
16

if we would go that way it should also work if one of the two is a dict.keys() instance, so that we can also write d1.keys() & d2 and d1 & d2.keys()

2 Likes
17

That’s right, furthermore, d.keys() returns a specific keys instance, we could have d1 & d2.keys() to return a subdict while d & l to return a list, d & s to return a set, maybe d & t to return a tuple, etc…

18

This looks like a X, Y problem. -
You may want a “deconstructor” - a shortcut to retrieve multiple values from a dictionary into local names

update\] - sorry, if you want a subset of the original dictionary, not a deconstructor, then the language has a construct for it: dict comprehensions\[/update

For one, the feature that was added to the language to do this kind of thing was the match/case constrcut. Unfortunatelly, it ended up being too verbose and bloated for this use case - it works mostly as a normalizer for a single item.

Now, most important - why this is a backwards-compatibility “no go”:

Then, passing a non-hashable object, like a list, as the index operator in the `a[b]` syntax errors in dictionaries, but only because default dictionaries don’t take lists as keys - it is perfectly legal and valid syntax, and works for any object that implements `_getitem_`.
While it on the surface looks like “ok, but this would be a feature for dictionaries only, other mappings still do as they please”, a “dict” is what “de facto” ditctates the expected interface for a mapping in Python. Changing the way dicts handle one specific kind of object as key would break the expectation that anything currently implementing `collactions.abc.Mapping` can be used as a drop-in replacement for a dict - that is hundreds, more likely thousands, of implementations of classes in popular existing packages.

Workaround for personal/in house projects:

For one, if the proposed syntax would make sense in your projects, it is a matter of creating a dictionary wrapper to do that, and you are good:

from collections import UserDict

class DeconstructorDict(UserDict):
    def __getitem__(self, key):
       if isinstance(key, list):
          return super().__getitem__(item_key) for item_keyin key
       return super().__getitem__(key, list)

And you are good.

-
Or just use the existing operator.itemgetter:


from operator import itemgetter as ig



key_a, key_b = ig("key_a", "key_b")(mydict)
4 Likes
19

Why special syntax instead of a named method?