`shlex.quote` formatters, and how is interpolation usually handled when quoting is required

Python Help
1

Fs there a “formatter” for shlex.quote? Something that would quote the stringified object before interpolating it into the final string? For example, doing: f"echo {a} {b}" instead of f"echo {shlex.quote(str(a))} {shlex.quote(str(b))}"

2

I came up with this hack, but it only accepts positionals

from typing import Protocol, NamedTuple
import shlex

class Formattable(Protocol):
    def __format__(self, format_spec: str) -> str: ...

class ShlexQuoted(NamedTuple):
    nested_object: Formattable
    def __format__(self, format_spec: str) -> str:
        return shlex.quote(self.nested_object.__format__(format_spec))

def shlex_qformat(format_str: str, *objs) -> str:
    return format_str.format(*map(ShlexQuoted, objs))

In [3]: shlex_qformat("{} {}", 1, 2)
Out[3]: '1 2'

In [4]: shlex_qformat("{} {}", "1 2", 2)
Out[4]: "'1 2' 2"
3

What’s your use case?

Are you trying to construct a shell command to run a subprocess? If so, you should prefer skipping the middle man (the shell) and pass the command line arguments directly to the subprocess. That way quoting never enters the picture. The simplest way to that that is to use subprocess.run with a list argument:

subprocess.run(["echo", a, b])

If you just want an unambiguously quoted representation of the string, you could consider using the string’s repr representation. That comes with the convient !r conversion specifier for f-strings: f"{str(a)!r} {str(b)!r}".

Otherwise, if you want to implement some custom string formatting syntax, you can try subclassing string.Formatter. Here’s a basic example that adds a conversion specifier !q that calls shlex.quote. You can override other methods to further customize the behavior.

import shlex
import string
from typing import Any


class MyFormatter(string.Formatter):
    def convert_field(self, value: Any, conversion: str) -> str:
        if conversion == "q":
            return shlex.quote(str(value))
        return super().convert_field(value, conversion)


>>> formatter = MyFormatter()
>>> s = """Aren\'t  "quotes" great?"""
>>> formatter.format("{}", s)
'Aren\'t  "quotes" great?'
>>> formatter.format("{!q}", s)
'\'Aren\'"\'"\'t  "quotes" great?\''
4

thanks for the recommendation, but this is for sending commands to an embedded device via telnet, I need to quote the arguments.

This seems to be exactly what I need. Thanks!

5

If you want to leverage f-strings (or, for that matter, the format method of the string rather than of a separate string.Formatter), you won’t be able to replace the field-conversion logic (so as to support !q as far as I know. You can still make format specifiers like :q work if you control the type of the passed-in values, by implementing their __format__, like in your existing example. Modifying a bit:

class Quotable:
    # Suppose actual logic is implemented here,
    # with sensible __init__ parameters and everything,
    # for a type that actually is useful for your program logic.

    def __format__(self, format_spec):
        result = str(self)
        if format_spec == 'q':
            result = shlex.quote(result)
        return self

a, b = Quotable(), Quotable()

f'echo {a:q} {b:q}'

But if the values aren’t already that type, then you won’t gain anything with such an approach - you’d still have to add calls in the f-string to make the conversions, so it just adds complexity. The simplest thing would be to just make a wrapper:

def sqs(x): # shlex-quoted string
    return shlex.quote(str(x))

f'echo {sqs(a)} {sqs(b)}'

In the long run, I think this will be simpler than using a custom string.Formatter.

1 Like