# \`functools.pipe\` - Function Composition Utility

**URL:** <https://discuss.python.org/t/functools-pipe-function-composition-utility/69744>\
**Category:** Ideas\
**Created:** [October 31, 2024, 4:46am UTC](https://discuss.python.org/t/functools-pipe-function-composition-utility/69744 "2024-10-31T04:46:57Z")\
**Posts on this page:** 1\
**Showing post:** 36

<div class="post-metadata">

**Author:** ![dg-pb](https://sea2.discourse-cdn.com/flex002/user_avatar/discuss.python.org/dg-pb/32/16248_2.png) [@dg-pb](https://discuss.python.org/u/dg-pb)\
**Post date:** [November 20, 2024, 8:35pm UTC](https://discuss.python.org/t/functools-pipe-function-composition-utility/69744/36 "2024-11-20T20:35:28Z")

</div>

Although this does not change proposed concept, this was useful to clarify definitions.

So to summarise. Definitions:

```bash
a) function composition:
    * f ⚬ g ⚬ h == h(g(f(x)))
b) function piping: function composition in reverse order
    * h >> g >> f == f ⚬ g ⚬ h == h(g(f(x)))
c) iterator piping: function piping, where payload is an iterator
                    and functions are iterator processors
                    (consumers and most of the time producers too)
d) Function application: https://en.wikipedia.org/wiki/Apply
    * g @@ f @@ x == g(f(x)) == (f ⚬ g)(x) # Forward
    * x |> f |> g == g(f(x)) == (f ⚬ g)(x) # Inverse

```

Now, if this was to be integral part of `Python` language, there are various options:  
a) `FunctionType` (and all callables) implement operators.  
b) Operators are implemented at parser level  
c) Object oriented approach such as [pipelines - Rust](https://docs.rs/pipelines/latest/pipelines/)  
d) Custom statement implementation

In case of (a) or (b), one could do something similar to:

```python
x = range(100)
result = (
    x
    |> partial(map, lambda x: x*2)
    |> list
)
# OR
result = x |> (
    partial(map, lambda x: x*2)
    >> list
)

```

In case of (c) it would be similar to:

```python
x = range(100)
result = Pipeline(x).map(lambda x: x*2).to_list()

```

The drawback of this approach is that at least with this design function composition can not be re-used for other inputs.

In case of (d) it is hard to say. But some options were explored in [Introduce funnel operator i,e '|\>' to allow for generator pipelines](https://discuss.python.org/t/introduce-funnel-operator-i-e-to-allow-for-generator-pipelines/67701).

### Now…

All of the above options are heavy and I don’t think the cost of those can be justified.

In contrast, this proposal allows user to have very similar convenience with significantly smaller addition to CPython: `pipe` object.

Then, user customise `pipe` to have similar syntax to the (a) and (b) examples above:

```python
x = range(100)
result = (
    x
    | pipe(partial(map, lambda x: x*2))
    | pipe(list)
)
# OR
result = x | (pipe()
    >> partial(map, lambda x: x*2)
    >> list
)

```

Also, it can achieve (c) by implementing extra methods to `pipe`, so that the user can do:

```python
result = x | pipe.from_map(lambda x: x*2).to_list()

```

So this proposal is not aiming to implement specific approach, but rather aims to implement smallest possible component that would allow user to build on top of it. This component benefits from being in stdlib, because:  
a) User reaps performance benefits - performant ` __call__ `  
b) Does not need to write boilerplate and can build on top of `class`, which is well integrated to the rest of Python’s stdlib. E.g. has correct signature & ` __get__ ` method.

### Use Cases

1. function piping needs:
  - @sayandipdutta’s reply only scratches the surface of various bigger frameworks having their own implementation: [`functools.pipe` - Function Composition Utility - #28 by sayandipdutta](https://discuss.python.org/t/functools-pipe/69744/28)

2. These are some libraries that are explicitly aimed at pipeline implementation:
  1. [Client Challenge](https://pypi.org/project/function-pipes)
  2. [Client Challenge](https://pypi.org/project/function-pipe/)
  3. [Client Challenge](https://pypi.org/project/functional-pipeline/)
  4. [GitHub - JulienPalard/Pipe: A Python library to use infix notation in Python](https://github.com/JulienPalard/Pipe)
  5. [Client Challenge](https://pypi.org/project/funcpipe/)

3. More performant identity function:
  - github search: `/lambda x: x( )?(\)|,)/` - 128k files

4. Predicate composition:
  - github search (only a small fraction): `/lambda x: x\[.*\](\.[^\d\W]\w*)/` - 57k files
  - Using `pipe` is arguably less readable, but would provide better performance in certain cases where it results in faster code and is needed.

---

_[View the full topic](https://discuss.python.org/t/functools-pipe-function-composition-utility/69744)._