Possibilities for improve pipelining syntax with new PEG parser

Ideas
13

What about interactive REPL usage?
The built-in REPL reads lines one by one uses essentially two heuristics to know when a statement/block is complete and ready to run:

  1. if a physical line can parse as a complete logical line, it is;
  2. a blank line closes all indented blocks

(1) is motivated by making simple use cases Just Work:

>>> print("Hello")Enter
Hello
>>> 2+2Enter
4

I believe (1) has so far informed the design of the grammar?
There are no present situation when a line looks like a complete statement but you need lookahead to figure it isn’t.
IMHO it’s also a nice property for humans reading the code.

(2) is actually a deviation from the grammar of .py files! It was not an obstacle to make the full grammar better (blank lines allowed in functions and other blocks, by lookahead for next non-blank non-comment line, or EOF).

I realize people who write code like this are likely to use a REPL with multi-line editing support like ipython or Jupyter.
Still, the proposal needs to say how the builtin REPL will work.
What happens when you type:

>>> last_status = casesEnter

Is it immediately executed? (so the new style without parentheses won’t work in REPL)
Do you get a continuation prompt:

...    # per case: sort by some_timestampEnter
...    .withColumn("rank", ...)Enter
...Enter

But in the latter case, do you need EnterEnter after every single statement?

Personally, I’d be happy for the answer to be “add separate keys for exectution vs continuation to builtin REPL” and/or “add real multi-line editing to builtin REPL”.
What about dumb terminal support? Is it a requirement?
Consider this:

$ echo 'if 1:
    print(1)
  
    print(2)' | python3.9 -i
1
2
$ echo 'if 1:
    print(1)
  
    print(2)' | python3.9 -i
Python 3.9.0a6 (default, May  5 2020, 18:42:59) 
[GCC 9.3.1 20200408 (Red Hat 9.3.1-2)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> ... ... 1
>>>   File "<stdin>", line 1
    print(2)
IndentationError: unexpected indent
>>>

So when input is from a pipe, python uses non-interactive grammar with lookahead! But you can force a REPL with -i, which uses the heuristics (1) (2).

2 Likes