Pyxl, tagged templates, html builders, oh my!

The python community has taken several swings at “HTML in python” in the past decade, including:

  • PyXL, worked on by several folks including Guido, which allows JSX-like syntax in Python. (Also: michaeljones/packed)
  • Tagged templates proposals, most notably the tagged strings draft PEP by Jim Baker & Guido
  • Direct HTML builders, like (to name one of many examples) the nascent htpy.dev library

These are quite different approaches to solving related but not identical problems!

Tagged templates are obviously the most versatile of the bunch — the PEP has examples for sql strings, shell strings, and others — but they have potential tooling drawbacks relative to a native JSX-like syntax (how should my_html"""...""" be formatted, linted, etc. if at all?).

Libraries like htpy seem to be stop-gap solutions by folks who recognize the value of using Python in its full generality to build HTML after having encountered limitations of templating languages like Jinja & friends.

Personally, I’d love to see something happen here. My instinct is that tagged templates make sense as a future Python language feature, whereas PyXL/JSX-style syntax, while useful, is more special-purpose, and probably belongs in separate tooling. That said, maybe there’s appetite for a native language extension; I’m not sure.

I’m curious what others want (if anything!) in this direction…

Personally, I have little or no use for HTML builders, but there are other forms of tagged templates (SQL and maybe shell-like ideas) that might be of interest. But honestly, I think it’s something where existing solutions (which don’t need language-level support) are good enough.

2 Likes

What’s wrong with htpy.dev as a solution? Syntax extensions has huge costs. What’s the benefit?

To me, htpy looks like many javascript libraries that were in regular use just before JSX took over. With the perspective of time, my instinct is that the JS community considers JSX to be an unambiguous win. That this would be the case wasn’t obvious up-front and, yes, the overall complexity/expense was (and to some extent remains) high.

i use something very close to what htpy does in my own framework and haven’t had really much issue with it, ive experimented with using custom encodings for “html syntax in python” but ide support is the main issue with it.

I don’t think this should be part of the language but something that could definitely be a community thing which is supported by ides and LSPs, the main issue is that python is not commonly preprocessed unlike the js ecosystem with its build systems, ts and minification, so something like this in python would feel very foreign.

I stumbled across this myself as I was looking for an alternative to the typical “Python in HTML” approaches like jinja. I threw together an HTML builder myself before finding ones with better design considerations. A few quick points:

  • PyXL (and packed, and mixt) are really cool, if a bit of a hack. The lack of IDE and type checker support for this sort of thing is a turn-off. Each would require bespoke plugins.
  • Tagged templates are a nice next step, though I’m really not sure how much more that would achieve over just an f-string.
  • Direct HTML builders will continue to get the job done, though the syntax definitely takes some getting used to.

With the rise of tools like htmx, I’m ultimately hoping for a revival of interest in an approach like PyXL/packed/mixt - syntactic sugar for HTML builders. I don’t know that tagged templates would actually make a meaningful difference in workflows for generating HTML, though I could be wrong.

I couldn’t stop thinking about this. I ended up hacking together a revival of PyXL/mixt, though with a different approach. I’ve pushed it here: pyxy-org/pyxy. It uses parso (the backend for jedi) with a modified grammar for the tokenization, which is then processed and passed to htpy for the markup generation. The latter step might not be necessary.

The nice thing about doing it this way is that it works off-the-shelf with existing tools which can read Python’s grammar (sort of, there are a few new things I’ve declared in there). The bad thing is this.

I haven’t cleaned up the code at all, and I haven’t tried it yet with declaring the encoding at the top of the file. With a few more hours of work I’ll have the value interpolation working in the markup.

But in all, it was enough to make this possible:

import pyxy  # noqa

if __name__ == "__main__":
    exec(b"print(<custom-tag disabled custom-attr=\"blue\">asdf 2378409 asdf</custom-tag>)".decode(encoding="pyxy"))

I’d highly recommend checking out the WIP tag-string PEP: GitHub - jimbaker/tagstr: This repo contains an issue tracker, examples, and early work related to PEP 999: Tag Strings

Tagged templates are a use case it explicitly attempts to address.

Cool! I didn’t give it a full read-through before, but I’m liking this more now.

I implemented some of the ideas from there so I won’t have to wait on the changes to land in a later Python version. This just uses the ast module, much less of a hack.

from pyxy.tagstr import tagstr, html

def is_logged_in() -> bool:
    return False

animal_images = ["cat.png", "dog.png", "cow.png"]
status_image = "logged-in.png" if is_logged_in() else "logged-out.png"

@tagstr(html)
def demo():
    return F'''
        <div>
            <img src={status_image}>
            <ul>
                {(F'<li><img src={image_file}></li>' for image_file in animal_images)}
            </ul>
        </div>
    '''

print(demo())
<div>
    <img src='logged-out.png'>
    <ul>
        <li><img src='cat.png'></li><li><img src='dog.png'></li><li><img src='cow.png'></li>
    </ul>
</div>

Under the hood, it replaces all instances of an uppercase f-string (F"...") with a custom string handler. This ensures that the more common syntax with a lowercase f f"..." will not be touched.

The actual implementation is not dissimilar to what’s detailed in the PEP, making use of deferred evaluation via lambdas.

In the example, the handler is the html function passed in to @tagstr(). Inside the html handler, it automatically joins any iterables and quotes any values if they’re preceded by an equals sign (though the latter is a little bit of a hack).

For reference, here’s the pyxy.coding version:

# coding: pyxy

def is_logged_in() -> bool:
    return False

animal_images = ["cat.png", "dog.png", "cow.png"]
status_image = "logged-in.png" if is_logged_in() else "logged-out.png"

def demo():
    return (
        <div>
            <img src={status_image} />
            <ul>
                { <li><img src={image_file} /></li> for image_file in animal_images }
            </ul>
        </div>
    )

print(demo())