New syntax `Trailing Block` for constructing objects with complex structure

Ideas
21

For me the problem with the dominate example is not so much the extra with keywords, but the magic action at a distance for what looks like regular function calls (e.g. in the first with link(...) not only creates a link object, but also adds it to doc.head. As with so much new functionality that’s something we could get use to, but right now it feels wrong.

Not to pick on the dominate example, but it introduces spooky action at a distance, the following code adds li items to the ol without any visible connection between the two. That’s not good for understandably and maintainability of code.

def magic():
    for i in ['home', 'about', 'contact']:
        li(a(i.title(), href='/%s.html' % i))


with doc:
    with div(id='header').add(ol()):
        magic()

Your proposal doesn’t have this particular problem, but there’s still a lot of opportunities to introduce unexpected behaviour in your proposal, including side effects of regular refactoring (e.g. splitting of a calculation that’s getting too complex introduces new variables and that affects what’s getting passed to __trailing__. SwiftUI appears to get away with this, but they have static typing at compile time and did have serious usability problems when the feature was first introduced (in particular hard to understand compiler errors when you made a mistake).

2 Likes
22

Yes, I highly agree with your opinions on the approach of dominate. The implicit actions hide deeply within the classmethod (that is what I found when reading the source), and there is threading-related things that the developer handled it carefully.

I also aknowledge that the trailing method approach also brings potentially unexcepted behaviors. What I plan to avoid this issue is to use strict typing on the arguments of trailing method.

In the above examples I’ve been trying to code with this kept in mind, such as

The positional arguments & key word arguments can all be typing with static-like types. Or, furthermore, if the proposal is accepted & widely used, libraries like pydantic (Welcome to Pydantic - Pydantic) can also be possible to offering a more strict constraint checker.

23

Because this:

return <table>
    <tr><th>Tada!</th><th>Table stuffs</th></tr>
    <tr><td>Whatever</td><td>Hmm</td></tr>
</table>;

looks better, in many people’s opinions, to this:

return TABLE([
    TR([TH("Tada!"), TH("Table stuffs")]),
    TR([TD("Whatever"), TD("Hmm")]),
]);

Which, by the way, are two actual real ways of building elements in JavaScript.

There is a time and a place for this sort of structure, and UIs are the most of it.

1 Like
24

These defects does exists. But we cannot deny its convenience on decribing complex data structure.

My way to avoid bringing the defects is to using strict typing when define __trailing__. Like what TSX, the strict typing version of JSX does.

25

Yes, I came up with this idea when I was doing frontend UI works. But later I found this idea might have a more board way.

The __trailing__ method actually offer a way to customize the individual behavior of each indent block, which with present python only few keywords like if, with, for can do.

With the new feature, normal classes can define the indent block behaviors following them.

26

To be honest, that sounds like an invitation for every single project to become a DSL that raises the barrier of entry for modifying the code base.

Have you looked into whether Syntactic macros would scratch your itch? I don’t know if that is going anywhere, but there’s at least a core dev who was interested enough to propose it.

2 Likes
27

Yes, my big plan is to make native python a DSL producer; in case of sounding too radical, at least, application of developing UI & complex data structure is a solid start.

Thank you for the references about macro PEP! :blush: I’ve not gone so far to macro programming. When I was studying C++, the most difficult part for me is that. I thought I could hardly handle this pattern of programming.

28

There’s a fairly strong and well-established principle that Python syntax shouldn’t be mutable in ways that encourage writing DSLs in Python itself (as opposed to parsing a DSL from a string or text file, for example). So you should expect to get a lot of resistance to this idea.

Personally, I think that DSLs can be useful when used with restraint. The problem is that people very often don’t have the same perception of what “with restraint” means as I do :slightly_smiling_face: Therefore, I think the current situation in Python is a reasonable compromise, and I’m not in favour of something as radical as you seem to be imagining.

3 Likes
29

Of course, nothing prevents sufficiently motivated people from, say, using the ast module to embed a DSL within Python (effectively creating an extended version of the langauge); nor from creating a DSL that is extended with Python. (For the latter, in particular, I’m thinking of Ren’Py. I don’t like its design, but it’s been successfully used by many people.)

1 Like
30

Excuse me for going too distant in the last post. :sob:

I am strongly agree with the principle that DSL should be used with restraint. I am not a fan of magic codes, metaprogramming or macro programming (or ruby language), too.

In this proposal I still focus on strengthen the language expressiveness on UI & complex data development. The gap between current proposal and a metaprogramming version of python is too large, like that between gravity law & spaceship.

In the past years, UI development has changed a lot, which turns from pure DSL (HTML/XML) to GPL with specialized syntax, like jsx & swiftui. I have the feeling that Python might also need such a new syntax. In the domain of UI development, Python takes little share than some other programming languages, which is a sharp contrast to the share in the domain of AI , Data Science & web development.

I am considering that maybe this new syntax can be caged in a specialized subset, for example just processing complex data & design UI. It can remain as a potential weapon for further usage or a new domain. Think about the blooming history of ruby (again, I am not a fan of it): it is a long time of wait till web framework rails comes out, after which python followers django, flask comes as well.

31

Using ast module is a so large work that I can not imagine currrently.

Actually, I think this proposal is reasonably pythonic . It only combines existed, definitely pythonic syntax: dunder method and indent block, without involving new keywords. With just a first glance of the codes I post above, people will most likely consider them as native python codes, instead of a modified version.

32

What is most concerning to me in this approach is that individual expressions Node() are implicitly treated as statements of type "add Node() to smth else" which does not seem to happen anywhere else in Python syntax.

Combining on ideas of @Rosuav and @kfdf above, maybe it could be something like this?:

gather root = Node(...):
    red_or_black = True # attribute
    yield gather Node(...): # collects inner nodes and resends the result upwards
        def on_walk(...): # attribute
            ...
        yield Node(...)
        yield Node(...)

gather x = Window():
    yield MenuBar(...)
    for lbl in field_names:
        yield Label(lbl)
        yield Input(name=lbl)
    yield Button("Save")
1 Like
33

The above codes can be interpreted in this way: the instantiation of Node , which is placed alone at the leading line of a indent block, should be treated as a keyword just like with, for, try or while; and thus, statements & expressions created within such block will certainly have specialized effects (such as, within try block exceptions will be caught, and within with block resources will be closed). The effects within the indent block may not be limited to gather, but any customized behavior that you define in __trailing__.

I am trying not to bring new keywords, and since this syntax is designed for describing complex, large, nested data structure, too many repeated keywords may not be an ideal idea. The above codes are likely similar to the approach of dominate, which uses with keyword at the leading of every level.

34

You don’t need new syntax to describe complex, large, nested data structures. Context managers can do that fine.

For example, I expanded your little Window/MenuBar/Label/Input example into a wxWize program:

import wx
from pprint import pp
import wize as iz

app = wx.App()
field_names = ["names", "of", "fields"]
inputs = dict()
with iz.Frame() as frame:
    with iz.Panel(orient=wx.VERTICAL):
        with iz.MenuBar() as mb:
            with iz.Menu("File"):
                iz.MenuItem("Save")
                iz.MenuItem("Quit", callback=lambda event:frame.Close())
        frame.SetMenuBar(mb)
        with iz.GridBagSizer():
            for lbl in field_names:
                iz.StaticText(lbl, x=0)
                inputs[lbl] = iz.TextCtrl().wx
            iz.Button("Save", x=1, EVT_BUTTON=lambda event: pp([(lbl,ct.GetValue()) for lbl,ct in inputs.items()]))
frame.Show()
app.MainLoop()

This is a complete, runnable program, by the way, provided wxPython and wxWize are installed.

wxWize uses global state to keep track of what to append to, which is not entirely clean design, but works well enough for its purposes.

Here’s another example that doesn’t, an little internal library to generate XML/HTML called xmlgen, which let’s you do this:

def create_a_simple_html_page():
    xml = xmlgen.XmlString()
    with xml.html:
        with xml.head:
            xml.title("A Very Simple Page")
        with xml.body:
            xml.p("The first paragraph of text.")
    return xml.getvalue()

You can probably guess what it does. This is not runnable, I’m just showing you to give you an idea of the kind of structure you can use: There’s a container object (here, XmlString), that keeps track of the current element, so that elements get added in the right place. __enter__ and __exit__ create and finalise objects, and push and pop them to an element stack.

35

This doesn’t seem like something which needs to be added to python, context managers solve this very well, there are also other ways you can achieve this.

I solve a similar issue in my own gui framework by using callables and __getitem__, this has been enough for me to describe my ui.

36

To way of using with statement is the approach of dominate (dominate · PyPI) , which is disscussed by me & Ronald in above posts.

This approach brings too many with keywords to code. I am reminded of the history that XML bought too many tags with angular brackets to data and then be replaced by JSON in the domain of web data transmission. Moreover, it is not the key problem.

As Ronald mentioned above

When you digging into the source code, it will be found that such approach with with statement puts the code of gathering nodes & adding children in class members. It is not a local scope but shared by class instances, which may leads to similar issues of non-recommended global statement. Besides, class members do not consists among threads. To handle this appoach robustly, much more extra codes should be filled in class memeber than those of __trailing__ method needs.

37

The proposal can handle UI design efficiently, but that is not all. It can be used widely on other domains.

Actually I am trying to bring a tiny declarative programming pattern carefully by this new syntax. Declarative programming is practically proved to be efficient in UI design, but not limited to this domain. We will find much more scenarios for the new syntax to show its power.

38

The magic action at a distance is a design choice in dominate, not something forced by the use of context managers. It could just as easily have been made so that you have to write doc.div, doc.li etc. instead of just div and li, and then the references to doc would tie it together.

39

I had been aware of it.

Actually, every framework / library / dialect / syntax sugar of declarative programming is a shortcut of imperative programming (creating every nodes & attributes by flatten assignment). There always be, and should be the corresponding way of flatten codes. Regarding the first example I took:

What the proposal focus on is to make this procedure more clear & easy. The finally goal to construct a nested, compelx object remains unchanged.

40

I agree that the proposed feature could be beneficial for specific circumstances. However, I think the situation you’ve illustrated could be enhanced without the introduction of fresh syntax. Have you considered employing an appropriate operator overloading?

# without with statement
class Node:
	def __init__(self, ...):
		...

	def add_child(self, other: Node):
		...

	__iadd__ = add_child

node = Node(...)
node += Node(...)

child = Node(...)
child += Node(...)
child += Node(...)
node += child

# with with statement
class Node:
	def __init__(self, ...):
		...

	def add_child(self, other: Node):
		...

	__enter__ = lambda self: self
	__exit__ = lambda self, exc_type, exc_value, exc_traceback: None
	__iadd__ = add_child

with Node(...) as node:
    node += Node(...)
    node += Node(...)

    with Node(...) as child:
        child += Node(...)
        child += Node(...)
    node += child