Chain like Ruby

Ideas
1

Hi there :wave:

I’m a wxPython user and I’m using my own customized wx.py.shell.
There, I added one quite simple magic rule that is,

x @ y => y(x)

Hence, x @y @z => z(y(x)). For example,

>>> 5 @range @list @p # interpreted as print(list(range(5)))
[0, 1, 2, 3, 4]

Compared to Ruby,

irb> p 5.times.to_a
[0, 1, 2, 3, 4]

we can chain the methods in the same way. I think this would make typing joy and comfortable (as Rubists proudly say). For example, when you typed >>> range(5), you don’t have to go back, type list(, go to the end, and type ). Just type @list.

Most importantly, you can chain any callable object across any modules, e.g.,

# input:
>>> buf @io.BytesIO @Image.open @np.asarray @plt.imshow
# interpreted as:
==> plt.imshow(np.asarray(Image.open(io.BytesIO(buf))))

I think this point is more flexible than Ruby.
Please let me know what you think.

1 Like
2

@ already means matrix multiplication, so redefining it to act as the |> operator from Elixir or some other call chain syntax won’t really work.

3

I think the idea is neat, not sure whether the specific notation is crucial.
The pipe operator is very common in UNIX systems, and it would be nice to have it also in Python. It doesn’t have to use @, and it probably can’t use the UNIX notation | (unless the contextual parser can do some magic), but it can probably use |> for example.
The primary objection I can think of is that it would add yet another way to do something (going against the Zen), and the syntax might get messy for anything that isn’t a callable that takes exactly one argument (needing to wrap things in lambdas or create named functions just for this).

1 Like
4

Wouldn’t have to do anything to the parser, just add an implementation for __rmatmul__ on the function type

class Function:
    ...

    def __rmatmul__(self, value):
         return self(value)

The problem is that almost always the @ (at-sign) operator is considered to have the left operand act on the right operand, so you might want to use the | (pipe) operator instead with __ror__

5

Thank you very much for your replies.

Silly me… I didn’t know the __matmal__ ops! :worried:
I have been wondering why my scintilla in the shell stopped highlighting the decorator from PY35.
Now I got it.

Please let me continue the discussion.

The reason of using @

Let x @f @g => g(f(x)) and regard @ as an operator for f and g (not as a binary operator here).
Then, a product of @f and @g corresponds to the composition of g and f, but in reverse order.
This is the feature of ‘pullback’ that is a fundamental idea in the mapping theory.
The ‘pullback’ is like a mathematically defined synonym of ‘substitution’.
Since @decorator is also a kind of substitution, using the same symbol @ was natural to me.

Pros and cons

@brettcannon, @itamaro,
Yes, the symbol is not crucial. maybe |>, $<, $@, $(…), and whatever. However, we must take care that many symbols can significantly reduce readability. (in this point of view, Ruby is failing, I think).

@EpicWink,
It is Interesting to customize the right-hand-side operator. It really works! But here I don’t want to limit the specific class. I want to extend this feature to the parser of the shell.

Well, I forgot to mention that this topic is intended only for REPL time, not coding time. One may dislike that the syntax of REPL and actual python code is inconsistent. I agree too, but I can also show a few reasonable reasons for accepting this.

  1. The main purpose of REPL is to try it fast and learn what will happen.
  2. We shouldn’t feel stressed about typing. A good REPL will make the user’s learning curve steep.
  3. The history that the interpreter parsed as the actual python code can be recorded and used in the coding time (copy and paste).

P.S.
Thanks for the efforts of all Python core developers and contributors.

6

Do something like this?

class do(partial):
    def __rmatmul__(self, iterables):
        self.results = tuple(map(self, iterables))

>>> range(5) @do(print, end=',')
0,1,2,3,4,

EDIT *iterables → iterables

7

That could work, though at least for me it takes come mental gymnastics to parse :slight_smile:

I kinda like the way that Hack handles output binding across pipes (Expressions And Operators: Pipe)

$x = vec[2,1,3]
  |> Vec\map($$, $a ==> $a * $a)
  |> Vec\sort($$);

(with the enhancement that we can optionally drop the $$ to treat it as a callable that takes a single arg, so:

$x = vec[2,1,3]
  |> Vec\map($$, $a ==> $a * $a)
  |> Vec\sort;

)

1 Like
8

I think it can be simplified more to mimic the Hack :wink:

from functools import partial

class apply(partial):
    def __rmatmul__(self, argv):
        return self(argv)

p = apply(print)

>>> range(5) @apply(list) @p
[0, 1, 2, 3, 4]

>>> [2, 1, 3] @apply(map, lambda x: x*x) @apply(sorted) @p
[1, 4, 9]

Unfortunately, the following doesn’t work: :worried:

>>> [2, 1, 3] @apply(np.array) @p
ValueError: matmul: Input operand 1 does not have enough dimensions ... (snip) ...

because (x @ y) calls x.__matmul__ prior to y.__rmatmul__.

1 Like
9

Just an idea:

  • x @ y would be equiv. to y.__call__(x) unless __rmatmal__ is explicitly defined.
  • Thus, it should have lower precedence than __rmatmul__.
  • But should have higher precedence than __matmul__.

I haven’t had solved this contradiction yet, but I achieved @ magic syntax by directly overwriting the interpreter of wx.py.shell:

I also think the good point about this feature of the shell is that evaluating expressions step by step allows the user to check for correct typing and results step by step.

10

I was always fond of an idea of a pipe operator in Python, in the very least because it doesn’t put me at a mercy of a package developer. Say, Pandas users can turn hard-to-read code like this:

df = pd.DataFrame(company_sales)
df = df.drop(columns="Company1")
df = df.dropna(subset=['Company2', 'Company3'])
df = df.rename(
    columns={
        'Company2': 'Amazon',
        'Company3': 'Facebook',
    }
)
df['Google'] = [450.0, 550.0, 800.0]

into something very clean like this:

df = (
    pd.DataFrame(company_sales)
    .drop(columns="Company1")
    .dropna(subset=["Company2", "Company3"])
    .rename(columns={"Company2": "Amazon", "Company3": "Facebook"})
    .assign(Google=[450.0, 550.0, 800.0])
)

But NumPy users have no such luxury, as developers have decided against implementing method chaining for whatever reason. So I either write NumPy code in an extremely verbose manner like in example #1, or I deeply nest it, which is an even worse option readability-wise. And if your source array isn’t named something concise like arr but, let’s say, array_after_calculations_sales, the code just gets exponentially uglier, as you have to repeat that specific variable name over and over. Having a pipe would free the user from tastes of package maintainers for your particular class. Moreover, it would liberate the user from being forced to name temporary variables to which they are assigning, which always is kind of a challenge.

I guess the biggest point against it would be that it might cause some discomfort for beginners who learn Python as their first language and have not had any experience with a language that supports a pipe operator, like Elixir or R.