I am not very sure about anything at this point really (except that this is interesting and could be worthwhile to continue). I need to digest PEP638 properly and put everything into perspective - a lot of stuff to consider by now.
However, the way I see it now:
PEP 638:
CODE → PARSE → AST → MACRO → NEW_AST → EVAL
While this proposal does not need the tail (mostly due to performance penalty and generally not doing unnecessary things) so all what is left:
CODE → PARSE → EVAL
As I said, maybe it is possible to to split it into 2 steps:
- CODE → PARSE(ENGINE)
- ENGINE → RESULT
In case of macros, then engine could do AST transforms as per PEP638, and in case of more straight forward DSLs, it can just evaluate straight away.
I have similar insight here.
I think there is one more thread worth linking to: Make the PEG parser available to all Python users
Of course, parser would not be made available, but it would be possible to utilise it for DSLs that can make different use of Python syntax.