Proposed additions to `operator`

Ideas
1

a) There are operator.is_ and operator.is_in. I propose also adding:

def in_(a, b):
    return a in b

def not_in(a, b):
    return a not in b

b) There are operator.call and it’s object operator.caller. There is operator.methodcaller. I propose adding:

def callmethod(obj, name, /, *args, **kwds):
    return getattr(obj, name)(*args, **kwds)

c) To complete functionals I also propose adding:

def map_(obj, func, /, *args, **kwds):
    return func(obj, *args, **kwds)

class mapper:
   ...

d) And finally, not sure if this could be a place for it, but I found it useful. To add to dictionary operations. Common operation, but not operator:

def get(obj, idx, default=None):
    try:
        return obj[idx]
    except (KeyError, IndexError):
        return default
2

a) You are aware of operator.contains, right? The reverse operation better mirrors the data model of python (the __contains__ method), which is one of the reasons for this entire module to exists in the first place. Why is it necessary, or even all that helpful, to add in_ and not_in?

b) operator.caller doesn’t exists. What are the potential use-cases for callmethod you are seeing? it seems quite hard to use with regard to use with map or a similar interface in contrast to other available functions

c) Having map and map_ both existing is IMO a non-starter. I am not really sure what your goal with this operation is, but it should have a different name. This seems to just be an alias for call with a slightly different (and more confusing) argument order? And mapper is just a weaker version of functools.partial?

d) IMO, this falls under “not every 3 line function needs to be in the stdlib”. While much of operator is just a 3-line function, part of the point is that they directly reveal underlying operations that CPython implements on a deeper level (for example, with a single opcode and/or operation). This isn’t one of them. It might be better in functools or maybe collections.

2 Likes
3

I came from a bit different perspective here.

I am making proxy objects, that defer evaluation. So I make various operations first and evaluate the whole structure later.

Thus, I have a node node and I want to be able to define all possible operations with it.

Now contains doesn’t work, because if I have node in obj, it will call containment operator of obj, but in_ could be used to signify the deference of the operation until the node evaluates.

Yeah, it doesn’t… But it could.

In my case, the use case for callmethod is the same as in a)

I named it apply at first, but then named it back to map. Does functools.partial allow for positional argument skipping? I did introduce sentinel VOID in my own partial implementation to allow for this. Nevertheless, I did write mapper to perform this for a node in a more lightweight manner.

Nah, yeah, as I said, this was a very weak proposal.

All in all, as I said, my angle is a bit different I guess.

It is more along the lines of “emulating operator behaviour from perspective of the object”. Or in other words “… from the perspective of a first term of the operation”

While it seems that current rationale of operator module (or the POV from which many of your arguments come from) is more like “emulating operator behaviour from perspective of the operator”

4

Just as a datapoint, a quick GitHub search for lambda a, b: a in b and operator.contains(b, a) does bring up a few cases where an operator.in_ was missed. Some examples:

Jinja2

asteroid

numba

pyanalyze

1 Like
5

Not sure what you mean? The magic method you need to implement for node in obj is __contains__. So it is actually impossible to overload this without a change in syntax, or modifying obj as well.

The point is you said it does. So you appear to not have paid attention to what is available. I am also not sure what operator.caller would do.

No, but that is a completely different topic. I don’t know how this relates to your proposal for map or mapper.

Well, yes, that’s why the module is called operator :stuck_out_tongue: While I agree that this philosophical view could be changed, I still see no real benefit for suggestions b-d

Hm, these example do suggest to me that operator.in_ and operator.not_in are probably a good idea. Trying to generically do stuff like map AST nodes to actions is made slightly easier by this. But I am sure this has been discussed before, maybe it makes sense to search through the history of the operator module.

6

You’re right. I can not emulate behaviour of the operator, but I use operator mixin (similar to what dask did dask/dask/utils.py at b663dca0fa4ca4686b8c08f7cb30d11320012901 · dask/dask · GitHub), where I automatically create operators and methods for such objects (proxies, nodes, array types, etc). So having in, not_in, I can make specifications with method names. So my object will not emulate operator behaviour, but the best of what I can have is a method created for a node object, so I can emulate it via node.is_in(obj).

I did not deny that I said it and implicitly agreed with your observation that I made a mistake. Whether I did or did not pay the attention is subject to investigation. In this case, I have my own caller and just forgot that I use my own implementation and not the operator. My caller implementation looks like:

class caller:
    __slots__ = ('_args', '_kwds')

    def __init__(self, *args, **kwds):
        self._args = args
        self._kwds = kwds

    def __call__(self, obj):
        return obj(*self._args, **self._kwds)

    def __repr__(self):
        args = list(map(repr, self._args))
        args.extend('%s=%r' % (k, v) for k, v in self._kwds.items())
        return '%s.%s(%s)' % (self.__class__.__module__,
                              self.__class__.__name__,
                              ', '.join(args))

    def __reduce__(self):
        if not self._kwds:
            return self.__class__, self._args
        from functools import partial
        return partial(self.__class__, **self._kwds), self._args

Also, these are equivalent:
a) methodcaller('name', *args, **kwds)(obj)
b) caller(*args, **kwds)(attrgetter('name')(obj)).

So one way to look at is A + B = C, where A = attrgetter, B = caller, C = methodcaller. Now A and C exist in operator. If subtraction C - A was possible, then there would be a very weak case to implement B, however it is not, so having caller could be useful for functional completeness.

This is in response to:

So my point is that it is not a weaker version of functools.partial, because functools.partial can not do what mapper can.

All in all, these are just proposals of things that I personally had to implement myself, while trying to achieve a complete functionality of object abstractions (Another example would be an array class which calls operators and methods over every element of it).

And yes, intuitively I felt that in_ and not_in could be most useful outside of what I do.