1. Motivation
This is to provide convenient Pythonic-DSL parsing.
I believe (hopefully) upcoming t-strings in conjunction with such extension could be efficiently used for Pythonic-DSLs with interpolation.
See my thought process how I arrived to this point at DSL Operator – A different approach to DSLs - #73 by dg-pb
2. Proposal
The idea is simple:
print(ast.dump(ast.parse('a + 1'), indent=4))
Expression(
body=BinOp(
left=Name(id='a', ctx=Load()),
op=Add(),
right=Constant(value=1)))
class Custom:
def Expression(body, *args, **kwds):
return body
def BinOp(left, op, right):
sop = {ast.Add: '+'}
return [sop[type(op)], left, right]
def Name(id, ctx):
return id
def Constant(value):
return value
print(ast.parse('a + 1', overloads=Custom.__dict__))
# ['+', 'a', 1]
3. Performance
This does not offer anything new from perspective of functionality.
Same can be done by:
ast.parse→ AST- Do everything with AST
The main point of this is to be able to avoid unnecessary steps allowing optimal performance.
E.g.:
def two_step(s):
bin_op = ast.parse(s, mode='eval').body
return bin_op.left.value + bin_op.right.value
%timeit two_step('1 + 1') # 5.31 μs
# By how much could it be faster?
%timeit ast.parse('1 + 1', mode='eval') # 4.86 μs
from ast import Expression, BinOp, Name, Add, Constant, Load
%%timeit
Expression(
body=BinOp(
left=Name(id='a', ctx=Load()),
op=Add(),
right=Constant(value=1)))
# 2.41 μs
4.86 - 2.41 = 2.45 µs
So constricting AST takes more than half of the time. Furthermore, a lot of things can be done straight away, which would otherwise need to be done using ast.Visitor, which is much more expensive.
From optimization point of view this is similar to: JSON parser - Tutorial — Lark documentation
4. Implementation
Either:
ast.parse(..., overloads=None)- Separate function
a)ast.parse_dsl
b)builtins.eval_dsl
I have no strong preference really. But would obviously take time to find the best place for it given this is something that is deemed to be useful.
5. To Sum Up
This would significantly speed up ast.parse functionality for variety of applications.
