Thanks, I can tell a lot of good thought went into this PEP. Overall, I like the design and think it would be a good addition to the language.
However, I have one significant concern. It relates to the deferred evaluation of interpolation expressions. I understand the motivation for this design choice, but I think it has many downsides that are not recognized or addressed in the PEP. The good news is that there are reasonable ways to accommodate lazy evaluation use cases without any of these downsides (more on that later).
Let me start by trying to convince you that deferring the evaluation of interpolation expressions in the common case is a bad idea.
First, it complicates the mental model for users of tag strings. They have to assume that all of their interpolation expressions may not be evaluated immediately. Deferred evaluation requires special considerations, so doing it implicitly leads to surprises. It places a higher burden on users to make their code correct in all circumstances. If they are the author of both the tag function and the tag string, this is less of an issue, but in many cases these will be different developers.
Users of tag strings cannot reassign or modify any of the values used in interpolation expressions after the tag string definition because the final evaluated string may be affected.
name = "Bob"
my_str = greet"hello {name}"
name = "Ellen"
print(my_str) # What would you expect here?
del name
print(my_str) # Will this crash?
There is no guarantee that evaluations will be performed by the tag function exactly once and in a left-to-right order.
id = 1
def get_next_id():
global id
id += 1
return id
my_str = tag_fn"{get_next_id()}: a, {get_next_id()}: b"
print(my_str) # Is this guaranteed to print "1: a, 2: b"?
# What if the tag function evaluates them multiple times?
# What if the tag function evaluates them in a different order?
# What if the tag function skips evaluating some of the?
print(my_str) # Will this print "3: a, 4: b"?
Second, when users encounter problems like this, the issue will be difficult to debug. Debugging async code is challenging in general, and this is even more challenging because debuggers will typically lack the context to show users what is executing at the time the bug occurs.
Third, static analysis tools like mypy and pyright will need to conservatively assume that all interpolation expressions are evaluated in a deferred manner. This means they’ll inevitably produce false positives in cases where evaluation isn’t deferred (the common case). Here’s an example:
def func(name_or_id: str | int):
if isinstance(name_or_id, str):
# The type of name_or_id is narrowed to `str` here,
# but the narrowed type cannot be used if `name_or_id`
# is evaluated in a deferred manner. This will result in
# a static type error when calling the `upper` method.
print(greet"Hello {name_or_id.upper()}!")
I see two potential ways to avoid some or all of the above problems.
Fix 1 (my recommendation): Make deferred execution explicit by having the user provide a callable in the interpolation expression. This fix addresses all of the problems I mentioned above.
With this fix, all interpolation expressions are evaluated immediately by the runtime, as they are with f-strings. This guarantees that they are all evaluated exactly once in left-to-right order, preserving the common-sense mental model of f-strings.
In the (relatively rare) case where the user wants their interpolation expression to be evaluated lazily (e.g. because it’s an expensive call or the information is not yet available) and this functionality is supported by the tag function, they can provide a lambda or other callable in their interpolation expression. If a tag function supports deferred (lazy) evaluation, it can look at the evaluated value of the interpolation expression and determine whether it’s callable. If it’s callable, it should call it to retrieve the value. If the value is not callable, the tag function should assume that the value can be used directly in a non-deferred manner.
name = "Ralph"
# `name` is evaluated immediately
greet"Hello {name}"
# The expression `lambda: name` is evaluated immediately.
# It is callable, so the tag function calls it in a deferred
# manner to retrieve the final value.
greet"Hello {lambda: name}!"
# Here "tag" is evaluated immediately but `fetch_body_deferred`
# is callable, so it is called in a deferred manner.
tag = "body"
def fetch_body_deferred() -> str: ...
html"<{tag}>{fetch_body_deferred}<{/tag}>"
The nice thing about this approach is that the common case (where deferred evaluation isn’t needed or desired) is much simpler. It puts the user of the tag string in control. In the less-common situation where deferred evaluation is desired, it’s clear to the user — and to static analysis tools — that deferred evaluation is being used. Users can be more cautious when deferred evaluation is intended, and static analysis tools can detect potential programming errors that result from deferred evaluation without generating false positives in the common immediate-evaluation case.
(Partial) Fix 2: Document clear expectations for tag functions. This mitigates some, but not all, of the problems. I don’t recommend this fix unless there are objections to fix 1.
This fix involves a clear, documented contract for tag functions. They should be expected to evaluate every interpolation expression exactly once and in order. If they fail to honor this contract, users of tag strings may see unexpected behaviors.
This approach is still problematic because it addresses only some of the problems. There’s also no way for the runtime or static analysis tools to enforce this contract, so there’s still potential for bugs.