Introduce 'expr' keyword

Ideas
22

I think the missing bit is “getting expression string without evaluating it, while letting IDE know that it is expression”. Don’t think it is possible with f-strings. Is this possible with t-strings at any chance?

1 Like
23

But for what purpose would you ever want a debugging message to show an expression without evaluating it? Even the OP lists this use case with the expression explicitly evaluated.

24

Regarding debugging:

print(f'{foo()[0].name!e} evaluates to {foo()[0].name!r}')

I don’t want to evaluate it 2 times. The first one should just paste an input string, while my IDE recognises it as syntax and parser does few checks on its validity.

I had some use cases over time. Apart from debugging, one that I remember is synchronization of dict keys with variable names, so that I can do:

def foo(a):
    d = dict()
    d[f'{a!e}')] = a   # Or whatever solution

And my IDE allows me to refactor nicely.

Maybe I am missing something.

So this would implicitly evaluate A and B expressions and discard the results without any application?

25

I was talking about a helper debug function that takes an f-string:

def debug(expr_value):
    expr, _, value = expr_value.partition('=')
    print(f'{expr} evaluates to {value}')

debug(f'{1 + 1=}') # 1 + 1 evaluates to 2

But what kind of a real-world application were you creating this dict for?

By friendlier debugging messages I was referring to the use case #3 that the OP listed:

26

Ah ok, thanks.
This does seem to cut it for many cases nicely, but f'{expr!e}' would make it much more convenient and adaptive to situation. E.g.

d = something[0].attr    # type(dict)
...
...
...
print(f'{something[0].attr["key"]!e} has value {d["key"]!r}')

Found mail logs from that time. So 3 use cases from back then:

  1. logging
  2. dict with keys synced to variable names: [Python-ideas] Extract variable name from itself
  3. Things like: https://www.mail-archive.com/python-ideas@python.org/msg33011.html
27

Sorry, I was gone for a while. Others have already replied with some other use cases, thank you all for that!

Here is another example: A simplified version of code in my current codebase. This class is a node of a tree structure that is immutable (as you can see by the frozen and tuple children):

@dataclass(frozen=True)
class CollectionNode[T: SlotTypeT]:
    slot: T
    parent: 'CollectionNode[T] | None'
    children: tuple[CollectionNode[T], ...]

    def _post_set_children(self, children: tuple[CollectionNode[T], ...]) -> None:
        """Set the children field of the node.
        
        It is logically impossible to set both parent and children at
        construction time for a whole tree. Therefore we allow the 
        exception to postpone setting the children field.
        """
        object.__setattr__(self, 'children', children)

Instead of 'children', I could use expr('CollectionNode.children') or expr CollectionNode.children (whatever way we will implement it) here. If I ever change the attribute name, it would then give an error, or better yet, auto-rename the given expression.
Here this is especially meaningful that the expression is not evaluated, because it is technically just a type annotation on the class and only an actual attribute on an object. So if this is evaluated, it will cause an error.

1 Like
28

So this would implicitly evaluate A and B expressions and discard the results without any application?

No, if we talk about the original idea, using an expr keyword, it would return a tuple[str, T] if an equal sign (=) is at the end, otherwise it would simply return the exact expression as a string.
At the beginning I had some wrong assumptions that we could just have the interpreter check whether the expression would mention valid identifers if the expression is not also evaluated, which is not really possible and something I would not want add anymore. Checking if it is a valid expression and that all parts of the expression are statically available identifiers should be done by the linter/type checker/ide.

But I would step away from the expr keyword recommendation and instead opt for the previously mentioned expr function, that takes in a string, which can optionally be evaluted, but the string should be special cased by linters/type checkers/the ide and treated like it was an expression outside of a string.

Example:

class Foo:
    bar: str

expr('Foo.bar')  # returns 'Foo.bar'
expr('Foo.bar', true)  # raises AttributeError, because it is just a type hint on the class

Assuming we are in VSCode, the string 'Foo.bar' should be highlighted as if it was outside of a string and therefore should also be affected by renaming, similar to type hints in the ForwardRef format (wrapped in a string).

1 Like
Proposal: Implement builtin way to stringify expression nicely (#2)
29

I actually had some similar ideas once and came up with the approach here.

from uneval import expr, var, evaluate

my_expression = var.stars[0]
my_expression = expr("stars[0]")  # alt
print(my_expression)  # stars[0]

evaluate(my_expression, stars=range(2, 5))  # 2

# or
stars = range(2, 5)
evaluate(scoped(my_expression))  # 2

Maybe not a language feature, but it comes close or can inspire new ideas. I think if this or a similar package became more popular it has more chance to be adopted. There is also sympy, but it uses a custom ast representation and is more math-oriented. I know that in other languages like R and clojure, these kinds of features are used more often.

30

This comes pretty close to what you want, I think:

import inspect
import executing

def expr(arg, with_value=False):
call_frame = inspect.currentframe().f_back
call_node = executing.Source.executing(call_frame).node
source = executing.Source.for_frame(call_frame)
left = source.asttokens().get_text(call_node.args[0])
if with_value:
return left, arg
else:
return left

stars = [1, 2, 3, 4, 5]

print(expr(stars[0]))
print(expr(stars[0], True))

Output
stars[0]
('stars[0]', 1)

My peek and vardict packages use this technique. See www.salabim.org/peek and www.salabim.org/vardict for details.

31

So I think there are 2 variations in demand here.

  1. Getting expression string
  2. Getting expression string with evaluated result

1. Getting expression string

So the main question is “why not just type string?”.
Or in other word “what value does this have compared to just typing the string?”

I think this is valuable for programmatic usage, where the expression needs to be validated (but not evaluated). What would the validation look like? I think 2 main things that can be done:

  1. Make sure expression can be successfully parsed in eval mode
  2. Make sure all reference names are present in current scope

(1) can be done easily.

(2) is the dealbreaker for many solutions.
Any “black magic” method will fail to capture some variables in some cases.
I.e. when the variable is part of expression but closure is not making use of it, it is not captured in co_cellvars.

What could be the solution for this?

f'{expr!e}'
t'{expr!e}'

At least I think so. As:

  1. No new keywords
  2. Syntax highlighting already implemented.
  3. Being part of string formatting… well, can be conveniently used as part of string formatting.

In short, I just think this is a good place to put it.
f'{a=}' already does this (+additional eval+repr), thus it would be an apple to the box of apples.

What is the best that can be done now?
Not sure, but the best that I have is:

def exprof(expr: str, /, *args):
    ast.parse(expr, mode='eval')
    return expr

exprof('_a[0]["b"](')         # SyntaxError
exprof('_a[0]["b"]()', _a)    # NameError

_a = 1
exprof('_a[0]["b"]()', _a)    # '_a[0]["b"]()'

ast.parse takes care of (1).
As for (2) - source all variables present in expression to args to validate their existence.

Not ideal though, I would still like to see !e conversion in the long run.

2. Getting expression string with evaluated result

What doesn’t work?

f'{expr=}'.split('=')
  1. Split can not be robustly determined for all cases
  2. 2nd part is not the value, but repr of value - if the value is needed, then this is wasteful evaluation + repr(val)

Good news is that t-strings have covered this one already:

def exprofval(template):
    intplns = template.interpolations
    assert len(intplns) == 1
    i0 = intplns[0]
    return i0.expression, i0.value

_a = [{'b': lambda: 999}]
expr, value = exprofval(t'{_a[0]["b"]()}')
# ('_a[0]["b"]()', 999)

Given “high” demand for this, I would not expect anything more convenient that that.
This is already pretty sweet.
Does it all. Including highlighting. Well, not yet in discourse.

So from where I stand, !e conversion would complete this.