`functools.pipe` - Function Composition Utility

Ideas
36

Although this does not change proposed concept, this was useful to clarify definitions.

So to summarise. Definitions:

a) function composition:
    * f ⚬ g ⚬ h == h(g(f(x)))
b) function piping: function composition in reverse order
    * h >> g >> f == f ⚬ g ⚬ h == h(g(f(x)))
c) iterator piping: function piping, where payload is an iterator
                    and functions are iterator processors
                    (consumers and most of the time producers too)
d) Function application: https://en.wikipedia.org/wiki/Apply
    * g @@ f @@ x == g(f(x)) == (f ⚬ g)(x)      # Forward
    * x |> f |> g == g(f(x)) == (f ⚬ g)(x)      # Inverse

Now, if this was to be integral part of Python language, there are various options:
a) FunctionType (and all callables) implement operators.
b) Operators are implemented at parser level
c) Object oriented approach such as pipelines - Rust
d) Custom statement implementation

In case of (a) or (b), one could do something similar to:

x = range(100)
result = (
    x
    |> partial(map, lambda x: x*2)
    |> list
)
# OR
result = x |> (
    partial(map, lambda x: x*2)
    >> list
)

In case of (c) it would be similar to:

x = range(100)
result = Pipeline(x).map(lambda x: x*2).to_list()

The drawback of this approach is that at least with this design function composition can not be re-used for other inputs.

In case of (d) it is hard to say. But some options were explored in Introduce funnel operator i,e '|>' to allow for generator pipelines.

Now…

All of the above options are heavy and I don’t think the cost of those can be justified.

In contrast, this proposal allows user to have very similar convenience with significantly smaller addition to CPython: pipe object.

Then, user customise pipe to have similar syntax to the (a) and (b) examples above:

x = range(100)
result = (
    x
    | pipe(partial(map, lambda x: x*2))
    | pipe(list)
)
# OR
result = x | (pipe()
    >> partial(map, lambda x: x*2)
    >> list
)

Also, it can achieve (c) by implementing extra methods to pipe, so that the user can do:

result = x | pipe.from_map(lambda x: x*2).to_list()

So this proposal is not aiming to implement specific approach, but rather aims to implement smallest possible component that would allow user to build on top of it. This component benefits from being in stdlib, because:
a) User reaps performance benefits - performant __call__
b) Does not need to write boilerplate and can build on top of class, which is well integrated to the rest of Python’s stdlib. E.g. has correct signature & __get__ method.

Use Cases

  1. function piping needs:
  2. These are some libraries that are explicitly aimed at pipeline implementation:
    1. Client Challenge
    2. Client Challenge
    3. Client Challenge
    4. GitHub - JulienPalard/Pipe: A Python library to use infix notation in Python
    5. Client Challenge
  3. More performant identity function:
    • github search: /lambda x: x( )?(\)|,)/ - 128k files
  4. Predicate composition:
    • github search (only a small fraction): /lambda x: x\[.*\](\.[^\d\W]\w*)/ - 57k files
    • Using pipe is arguably less readable, but would provide better performance in certain cases where it results in faster code and is needed.