Although this was my initial goal, I have first explored idea of functools.argorder, to be used with functools.partial. Addition: `functools.argorder` It offers some benefits, but is not suitable for standard library due to various reasons.
So the only thing that was left is to actually propose a change to functools.partial.
In short, it is a functionality extension of functools.partial, which allows positional arguments to be placeholders, so that postitional arguments of a call fill those places first.
There are number of libraries implementing this in pure python, but none of them are efficient. And efficiency in this case is one of the main drivers. Some of those packages:
Also, this was already proposed. Functools.partial extension to support specific positional arguments The main oppositions were:
- Just use lambda for this
What led me here is the fact that functional toolkit is inefficient for small sizes of iterables. So part of the problem this is solves is that it improves upon lambda performance so that functional toolkit such as map in combination with any can be used for any size iterables being sure that it is among the most performant options. However, it has been proven that for short size iterables loops greatly outperform any other method and there are cases where it can be of significant importance.
This has been discussed and laid out in: Builtins.any performance - #29 by dgrigonis
- Another argument against was that “Extension would lead to performance decrease”
However, the performance decrease is negligible as can be seen below:
from functools import partial
from partial2 import partial as partial2
import unittest.mock as utm
_ = VOID = utm.sentinel.VOID
p1 = partial(opr.sub, 1)
p2 = partial2(opr.sub, 1)
p3 = partial2(opr.sub, _, 1)
p4 = partial2(opr.sub, VOID, 1)
print(p1(2)) # -1
print(p2(2)) # -1
print(p3(2)) # 1
print(p4(2)) # 1
%timeit p1(2) # 48 ns
%timeit p2(2) # 52 ns
%timeit p3(2) # 54 ns
%timeit p4(2) # 54 ns
So performance has not suffered much and there are still couple of places for optimisation.
Implementation
- Implementation is straightforward and I have not found any issues with it.
- There is a restriction for number of positional arguments sourced to new callable to be higher or equal than the number of placeholders. This ensures that there is no ambiguity regarding creation of joint argument tuple.
Use case
from functools import partial
from partial2 import partial as partial2
from hello import ilen2 as ilen
import unittest.mock as utm
_ = VOID = utm.sentinel.VOID
p1 = partial(opr.sub, 1)
p2 = partial2(opr.sub, 1)
p3 = partial2(opr.sub, _, 1)
p4 = partial2(opr.sub, VOID, 1)
print(p1(2)) # -1
print(p2(2)) # -1
print(p3(2)) # 1
print(p4(2)) # 1
%timeit p1(2) # 48 ns
%timeit p2(2) # 52 ns
%timeit p3(2) # 54 ns
%timeit p4(2) # 54 ns
from operator import contains
pred = lambda d: contains(d, 9)
pred2 = partial2(contains, _, 9)
a = [{i: i} for i in range(10)]
ilen(filter(pred, a)) # [{9: 9}]
ilen(filter(pred2, a)) # [{9: 9}]
%timeit ilen(filter(pred, a)) # 784 ns +86%
%timeit ilen(filter(pred2, a)) # 421 ns
# -----
b = [{i: i} for i in range(50)]
%timeit ilen(filter(pred, b)) # 3.76 µs +135%
%timeit ilen(filter(pred2, b)) # 1.59 µs
# -----
b = [{i: i} for i in range(100_000)]
%timeit ilen(filter(pred, b)) # 3.43 ms +100%
%timeit ilen(filter(pred2, b)) # 1.64 ms
And use case which led me here:
def any_loop_(maps, key):
for m in maps:
if key in m:
return True
return False
# ------------------------------------
k = 0
maps = [{}] * k + [{k: k}]
# LOOPS N 1 5 50 100 100K
%timeit any(k in el for el in maps) # 610ns 800ns 3.6µs 6.1µs 5.5ms
%timeit any(True for el in maps if k in el) # 610ns 790ns 1.9µs 3.4µs 3.0ms
%timeit any_loop_(maps, k) # 160ns 265ns 1.6µs 3.0µs 2.8ms
# FUNCTIONALS
pred = lambda m: k in m
pred2 = partial2(contains, _, k)
%timeit any(filter(pred, maps)) # 190ns 465ns 2.9µs 5.5µs 5.4ms
%timeit any(filter(pred2, maps)) # 160ns 300ns 1.5µs 2.9µs 2.7ms
%timeit any(map(contains, maps, repeat(k))) # 360ns 400ns 1.4µs 2.6µs 2.2ms
For the problem above, without this addition there was only 1 performant functional approach - map + repeat. Using lambda was ok for iterable size up to 5, but that is all.
This proposal adds 1 more performant solution for this problem. Although it is still not as fast as loop, for short sizes, but its performance is competitive across all sizes.
However, while this is a problem that I was concentrating on, functools.partial is a general utility and this functionality would be applicable to many other curry cases.
