DSL Operator – A different approach to DSLs

Ideas
73 
from _string import formatter_parser
import re
import operator as opr


CMP_REPL_ID = re.compile(r'\{([^\d\W]\w*)\}')


class ns_dict(dict):
    def __init__(self, d, symbols, f_symbol):
        dict.__init__(self, d)
        self.symbols = symbols
        if f_symbol is None:
            f_symbol = lambda x: x
        self.f_symbol = f_symbol

    def __getitem__(self, item):
        if item in self.symbols:
            if (f_symbol := self.f_symbol) is None:
                return item
            return self.f_symbol(item)
        return dict.__getitem__(self, item)


def parse(string, namespace, f_symbol=None):
    parse_it = formatter_parser(string)
    symbols = set(filter(None, map(opr.itemgetter(1), parse_it)))
    assert not symbols & set(namespace)
    debrack = CMP_REPL_ID.sub(r'\1', string)
    namespace = ns_dict(namespace, symbols, f_symbol)
    return eval(debrack, namespace)


print(parse('{x}**2 + 3*{x} - 5', {}, lambda x: 1))
# x**2 + 3*x - 5
# -1


class cas:
    i = 'i'
    def mul(a, b): return ['*', a, b]
    def add(a, b): return ['+', a, b]
    def cos(x): return ['cos', x]
    def sin(x): return ['sin', x]


print(parse('mul({r}, add(cos({theta}), mul(i, sin({theta}))))', cas.__dict__))
# mul(r, add(cos(theta), mul(i, sin(theta))))
# ['*', 1, ['+', ['cos', 1], ['*', 'i', ['sin', 1]]]]

Issues with this:

1. Flexibility (Inability to override operators)

Solution ast.parse-like parse_dsl:

class cas:
    i = 'i'
    def name(name): return ...
    def bin_op(op, a, b): return ['op', a, b]
    def args(...): return ...
    def call(name, args, kwds): return ['call', name, args, kwds]
ast.parse_dsl(string, namespace=cas.__dict__)

So that one can override anything pythonic (e.g. a[1] or b[2])

2. Performance

import ast
from math import cos, sin
i, r, theta = -1, 1, 6
%timeit parse('mul({r}, add(cos({theta}), mul(i, sin({theta}))))', cas.__dict__)    # 51 µs
%timeit eval('r * (cos(theta) + i*sin(theta))')                                     # 25 µs
%timeit ast.parse('r * (cos(theta) + i*sin(theta))')                                # 23 µs

So parse_dsl could be as fast as 23 µs and extra optimizaion efforts could be developed to bring it down further.
e.g.:

ast.parse_dsl(string, namespace=cas.__dict__, caching=MyCacheStrategy())

etc…

So with t-strings solution would look look similar to:

def parse(string, namespace):
    interpolations = t-string-stuff # == {'dummy_r': 1, 'dummy_theta': 2}
    namespace.name = lambda x: interpolations[x] if x in interpolations else namespace.name(x)
    debrack = ... # place dummy_names into places of `{theta}, {r}, etc`
    return parse_dsl(debrack, namespace.__dict__)

cas = functools.partial(parse, namespace=cas)

cas(t'{r} * (cos({theta}) + i*sin({theta}))')

So ast.parse-like function with overridable node construction methods
would pretty much be best that can be done.

Note, in Julia need for $ is inevitable. {} is not that much worse.

So I don’t think it is worth to go any further for runtime solution.
As no runtime solution exists that would make implementing the following sensible anyways:

a = maybe('{a}[1] or {b}[2]')

which if done in Pure Python without dsl is 50 ns, while runtime solution will not be faster than 10 µs. So 200 slower.

The above is a rough draft.
Things could be done more elegantly for sure.
But I think it is enough to paint high level picture of what is possible with t-strings and a bit of non-syntax-changing utility development.

So my proposal is the following:

  1. for runtime solutions, aim re-using Python’s existing ast.parse logic and other existing good stuff to devise tools along the lines of parse_dsl above to make things as good as possible and combine that with t-strings.
  2. Concentrate on devising perfect pythonic solution of macros

So:

  • t-strings + parse_dsl = Pythonic DSL with interpolation @ Run-time
  • t-strings + parse_custom = Non-Pythonic DSL with interpolation @ Run-time
  • MACROS = Pythonic DSL @ Compile-time
  • Other programming languages = Non-Pythonic DSL @ Compile-time