Recursive .format, PEP-3101 formatting

What if we enabled a recursive string formatting method ?
See the following in-python implementation and example of use :

def fullformat(st, *args, **kwargs):
    """
    Recursively formats st (a string) using the PEP-3101 syntax.
    """
    original = st
    stor = [st]
    for k in range(sys.getrecursionlimit()):
        st = st.format(*args, **kwargs)
        if st in stor:
            if st == stor[-1]:
                return st
            raise ValueError(f"String {original!r} leads to a formatting loop.")
        stor.append(st)
    raise ValueError(f"String {original!r} takes too many iterations to fully format.")
>>> a = "{origin}, to {destination}"

>>> a.format(origin="{f} going {g}", destination="DEST", f="FF", g="GG")
... "{f} going {g}, to DEST"

>>> a.format(origin="{f} going {g}", destination="DEST", f="FF", g="GG").format(... (same parameters))
... "FF going GG, to DEST"

>>> fullformat(a, origin="{f} going {g}", destination="DEST", f="FF", g="GG")
... "FF going GG, to DEST"

>>> fullformat(a, origin="{origin} for {origin}", destination="DEST")
ValueError : String "{origin}, to {destination}" takes too many iterations to fully format.

>>> fullformat(a, origin="{destination}", destination="{origin}")
ValueError : String "{origin}, to {destination}" leads to a formatting loop.

I would propose .rec_format as a method name for str.
The messages for the exceptions should probably be changed and include the parameters but that’s a detail.
The concept is so simple, yet so easily prone to infinite loops when implemented hastily, I think it may deserve a builtin implementation.

Interesting idea.

What value does it add? What kind of programs could be written (at all, or, more easily) by adopting this construct as a language feature?

If it is just a curiosity, it is probably unwise to add it to the language. Adding features increases cost of implementation, testing, and documentation. It is not worth it unless the feature also adds value.

Little Bobby Tables

1 Like

@JDLH It would be quite long and complicated to explain the context, really. In a nutshell, it’s about a system for displaying a character from asset files, and there’s dependencies between the assets - there’s different kinds of hats, and for a single hat you want to use the good model depending on the kind of hair the character has…
Anyway, when writing it, I ended up writing this function, and as I said the easy mistakes I first made could be avoided for future devs by implementing this natively.

@storchaka in what situation would the use of fullformat be more risky than str.format in that regard ?
Fwiw, I can hear the “it’s no use” argument, although I’m not sure I would agree with it, but I don’t see a security risk here.

It’s a sufficiently rare requirement, and easy enough to implement sufficiently well for a given application, that writing it as a local function in your application is sufficient, IMO.

Yes, avoiding infinite loops is fiddly to get right. But most use cases won’t need to worry about that. Similarly, most uses won’t need to worry about what other failure modes might exist, and get them right. Whereas a stdlib implementation has to get everything right - so the implementation cost is substantially higher. Also, I’m not sure where this would go in the stdlib, but there’s a significant chance it would need to be in C, unless it gets a brand new module of its own. So that’s yet more implementation complexity.

And if you really expect to need the version you posted in a lot of places, publish it on PyPI. Then other users (should you get any) will give you a good idea of how significant the need is, as well as what other edge cases need to be addressed and what the maintenance burden is like, giving a much better basis for discussion about stdlib inclusion.

4 Likes
fullformat("Hello, {user.name}!", user=User(name="{user.__class__.__init__.__globals__._secret}"))
2 Likes

Can someone who understands the proposal explain in in plain language?

@storchaka Accepting the second string in your system would already be a security failure in itself, if you were only using str.format. fullformat doesn’t change anything.

@guido You take a string, apply .format to it, and apply it again to the result until nothing moves anymore.
In other words, fullformat allows .format to interpolate things which can themselves contain (or generate) string interpolation.

str.format is not vulnerable, your fullformat is. If you don’t see this, I have nothing to add.

2 Likes

Sounds like an interesting algorithm but not of sufficient use to be placed in the standard library.

There’s another way to help others with what you’ve learned though: a PyPI package.

2 Likes