A "with" variant for use in expressions

Ideas
1

I feel I can pretty much express this suggestion in one line:

process_in_some_way(json.load(f) with open('foo.json') as f)

To me this looks like a natural, obvious and useful counterpart for the true_exp if pred else false_exp use of if in expression context.

(This feels so obvious it’s surely been suggested before — and likely, therefore, already rejected before — but it’s fiendishly difficult to search for “with” specifically as a Python keyword rather than a general part of English. Apologies if I’ve duplicated suggestions I’ve been unable to find.)

I’ll note there’s one potential enhancement to such a scheme: allowing the context manager to control the result of the expression in the same way as it already controls the value assigned to the target on entry.

For example, in:

my_strs.append(my_obj.dump(file=f) with io.StringIO() as f)

…the with expression might reasonably return f.getvalue() rather than the (probably None) return from my_obj.dump().

Proposed implementation of that enhancement: an __exit_expr__ function:
object.__exit_expr__(self , value , exc_type , exc_value , traceback )

There could be a complementary pair of fallbacks:

  • When a context manager is used in an expression and lacks __exit_expr__, __exit__ is called instead and the value is passed through unmodified.
  • When a context manager is used in a statement and lacks __exit__, __exit_expr__ is called instead with None as value, and its return value is discarded.
6 Likes
2

I like this.

with open("file.toml", "rb") as f:
    config = tomllib.load(f)

I’ve written code like the example above countless times, and it’s a minor annoyance each time. It would be great if I could write it like a lambda:

config = tomllib.load(with open("file.toml", "rb") as f: f)

data = with open("file.txt", "r") as f: f.read()

Generalizing this a bit, the syntax foo = with EXPRESSION as TARGET: ANOTHER_EXPRESSION should be syntactic sugar for:

with EXPRESSION as TARGET:
    foo = ANOTHER_EXPRESSION

I would be perfectly happy if, like lambdas, this were limited to single-line expressions. Any other case probably deserves its own block anyway.

There’s also async with, but I’m not knowledgeable enough about async to make a suggestion for it.

1 Like
3

You can define your own small helper function if you like:

def using(mgr, fn):
    with mgr as res:
        return fn(res)

process_in_some_way(json.load(f) with open('foo.json') as f)
process_in_some_way(using(open('foo.json'), json.load))

my_strs.append(my_obj.dump(file=f) with io.StringIO() as f)
my_strs.append(using(io.StringIO(), lambda f: my_obj.dump(file=f))

config = tomllib.load(with open("file.toml", "rb") as f: f)
config = using(open("file.toml", "rb"), tomllib.load)

data = with open("file.txt", "r") as f: f.read()
data = using(open("file.txt", "r"), lambda f: f.read())
9 Likes
4

Why use open() as a context manager if it’s only going to be piped into a config parser? Why not json.loads(pathlib.Path(“/path/string/”).read_text())?

I get for more complicated file object usages this isn’t applicable, but if only doing a simple config parser, holding it as a string isn’t too bad is it?

3 Likes
5

Why, what’s the problem?

6

In an ideal world, json.load() would stream the data in, rather than loading it all into memory then calling json.loads().

Unfortunately, looking at the source code, it appears this is not an ideal world. :disappointed_face:

This proposal seems preferable because it leaves the door open for alternative implementations of json.load()to handle reading the file differently. And, besides, I offered it as a simple example; in general there may not be such alternatives as Path.read_text().

1 Like
7

I strongly detest the idea that a code block that controlls resource lifetimes is valid for code Golf inlining

8

Only having some __exit_expr__ would not be enough to control the expression.

Two options for controlling it would be:

  • Having the expression be used in the same way as normal with / async with context managers.
  • Adding __aenter_expr__, __enter_expr__, …, __aexit_expr__, __exit_expr__.

Personally though, I’m -1 on this, the same can be done in Python already, and it’s more readable right now.

I’d be happy to hear about other possible use cases though, perhaps that could change my mind (and that of others against this).

1 Like
9

I don’t think calling it code golf is fair, it looks readable to me and resource management in expressions exists in other languages, especially those without statements.

1 Like
10

Proponents of this idea should think about what whitespace and indentation, specifically in Python, are good for.

Each block scope, indented, gives you a visual which matches some structural element of the code. I think context managers were a brilliant addition to the language because they play synergistically with the value and meaning of indentation.

-1 from me.

9 Likes
11

When will the file f be closed?

3 Likes
12

open() was just the first example of this pattern I could come up with.

There’s no problem hence the “minor annoyance”. I just find it a bit cumbersome/unergonomic to use a with block for a single line expression.

I don’t think it needs additional methods. I imagine something like:

config = with open("file.toml") as f: f.read()

would just be syntactic sugar for:

def _read_config():
    with open("file.toml"):
       return f.read()

config = _read_config()

You can already write a single line with statements and I’ve seen them in the wild:

def read(file: str | Path) -> str:
    match file:
        case str():
            with open(file, encoding="utf-8") as f: return f.read()
        case Path():
            return file.read_text(encoding="utf-8")

In fact, you can write the following right now[1]:

with open("pyproject.toml") as f: config = f.read()

print(config)

  1. But the order makes it incredibly hard to read, which is probably why it’s not a popular pattern (I’ve never seen it in the wild). ↩︎

1 Like
13

IMO it’s important to remember that every language change is a relatively significant disruption, and is not something that we undertake lightly. If a proposal does nothing more than address a “minor annoyance”, then it falls a long way below the bar for being a realistic proposal for a language change.

I’m not trying to pick on anyone in particular here (and specifically @Monarch it was just your choice of words that triggered this thought for me) but I think it’s something that gets forgotten very often, in the enthusiasm for an interesting new idea.

I’m -1 on this proposal, and in particular I think that code using it is less readable and maintainable than code that uses a normal with statement.

9 Likes
14

What’s that normal way for the OP’s example?

process_in_some_way(json.load(f) with open('foo.json') as f)
15

What would __enter_expr__ do that __enter__ did not, though?

16

I do wonder how the true_exp if pred else false_exp syntax entered the language, if it had to pass that test?

17

I suppose the normal way involves an intermeidate single-use variable like:

with open('foo.json') as f:
    data = json.load(f)
process_in_some_way(data)

While saving an intermediate variable is nice, I don’t really like a syntax where I have to read backwards to follow the flow of execution. Same mixed feelings towards comprehensions and the ternary operator, but at least they save a few more lines of boilerplate to make them more justifiable.

18

And avoiding a second reference. Could be beneficial for performance or clarity if process_in_some_way has the only reference. With the additional variable, the reader might wonder whether that variable also gets used in subsequent code. Especially if process_in_some_way modifies it. The proposed syntax could solve all this.

1 Like
19

You could also define a helper function for reading json files:

process_in_some_way(jsonyx.read('foo.json'))

This opens the file in binary mode, handling utf8, 16 and 32 automatically.

NOTE: I’m not requesting this helper to be added to the json library.

1 Like
20

Or just

with open('foo.json') as f:
    process_in_some_way(data)

I really don’t see why saving one line is important enough to add a whole new language construct, with its own set of edge cases, potential enhancements, arguments over what’s “better form” that end up getting enshrined in linter rules, etc.

4 Likes