Try With Resources — Elegant Handling of Exceptions In Context Managers

Ideas
1

A common task in programming is accessing some resource that might or might not be available. If the resource is available, processing can happen as usual — but the resource should be safely released when it is no longer needed; however, if the resource is not available when requested, usually some exception is raised and should be handled or propagated. Exceptions can also occur during the processing of an open resource — in which case the resource should be closed.

This task is common enough that some languages like Java have implemented try-with-resources constructs that allow for this pattern. A resource can be accessed — but if any exceptions occur while acquiring the resource or during processing of the resource, they can be caught.

Python currently does not have a single construct to achieve this — resources can be acquired and closed with context managers, and exceptions can be caught with try statements. Combining these constructs, while a common enough task, is clunky.

The pattern of resource acquisition/processing with error handling is common enough that I feel Python should allow with statements to include except, else, and finally blocks to handle any errors that are raised during a context manager’s lifetime.

For example:

with open(file, "rb") as f:
    config = json.load(f)
except FileNotFoundError as e:
    panic_with_message(f"Cannot find file: `{file}`", e)
except JSONDecodeError as e:
    panic_with_message(f"Invalid JSON in: `{file}`", e)
else:
    run(config=config)
finally:
    global_cache.clear()

Context managers do have the ability to handle exceptions as they are being exited (For example, as Suppress does); however, there is currently no elegant way of providing custom handling logic to context managers.

3 Likes
2

This has been discussed before: `try with` Syntactic Sugar - #8 by oscarbenjamin — but the conversation there seemed to revolve mostly around aesthetics and code length rather than providing a codified implementation of this common resource handling pattern.

1 Like
3

While it could be made less verbose, I don’t see any technical issues with

try:
    with ...:
except:
    ...
else:
    ...
finally:
    ...

So I’m neutral on the idea atm.

1 Like
4

The example block of code is quite appealing. But doesn’t this need to decide (or at least it should be documented) whether the context manager’s __exit__ method is called first, or the finally: block is executed first? A clue is in the name, but a combination of try: and with makes that unambiguous and explicit.

1 Like
5

I guess that would depend on the expected behaviour of examples such as:

class Echo:
    def __enter__(self):
        print("Entered Echo")

    def __exit__(self, exc_type, exc_val, exc_tb):
        print(f"Exiting Echo with: {exc_type}")
        return False


with Echo():
    print("Raising ValueError")
    raise ValueError()
except ValueError:
    print("Except ValueError")
finally:
    print("Finally")

Would this print:

Entered Echo
Raising ValueError
Exiting Echo with: <class 'ValueError'>
Except ValueError
Finally

Or:

Entered Echo
Raising ValueError
Except ValueError
Exiting Echo with: None
Finally

I’m in favour of the former — I feel the __exit__ code should run before any exceptions or other blocks are handled — which would close any resources as soon as the with block exits. This is also how Java’s try with resources behaves:

public class Main {
    public static void main(String[] args) {
        try (Closing closing = new Closing()) {
            System.out.println("In try block");
            throw new Exception();
        } catch (Exception e) {
            System.out.println("Catching Exception");
        } finally {
            System.out.println("Finally");
        }
    }
}

class Closing implements AutoCloseable {
    public Closing() {
        System.out.println("Creating Closing");
    }

    @Override
    public void close() {
        System.out.println("Closing now");
    }
}

prints:

Creating Closing
In try block
Closing now
Catching Exception
Finally

__exit__ running before any except blocks also removes confusion around the order of events for exceptions that happen in __enter__ or the context manager constructor (e.g. FileNotFoundErrors) which are not followed by __exit__ and exceptions that happen in the with block which are followed by __exit__

6 
class Echo:
    def __enter__(self):
        print("Entered Echo")

    def __exit__(self, exc_type, exc_val, exc_tb):
        print(f"Exiting Echo with: {exc_type}")
        return False

try:
    with Echo():
        print("Raising ValueError")
        raise ValueError()
except ValueError:
    print("Except ValueError")
finally:
    print("Finally")

Produces exactly the output that you say you prefer:

Entered Echo
Raising ValueError
Exiting Echo with: <class 'ValueError'>
Except ValueError
Finally

What’s the problem?

1 Like
7

You are conflating two different sources of errors: with calling open and with using the object that a successful call to open returns. They can and should be separated. The with statement is already a sort of try statement; any exceptions are passed to the __exit__ method of the context manager. In this case, you want to handle an exception that occurs before the file-like object even exists.

What you’ve shown should be written like this

try:
    f = open(file, "rb")
except FileNotFoundError:
    panic_with_message(f"Cannot find file: `{file}`", e)
else:
    # *If* the file was opened successfully, ensure it is closed
    # no matter what happens while we are using it
    with f:
        try:
            config = json.load(f)
        except JSONDecodeError as e:
            panic_with_message(f"Invalid JSON in: `{file}`", e)
finally:
    global_cache.clear()

It’s only 2 lines longer, and it’s more clear about when a file actually needs to be closed. It’s not important that the call to open appear at the start of the with statement, only that the value it can produce does.

9 Likes
8

While it is true that you can write all that. But it is not as elegant.

Although I understand the reluctance in considering this as a suggestion since the alternatives exist.

1 Like
9

Elegance is in the eye of the beholder. I find it elegant that with blocks and try blocks are orthogonal components that can be easily composed. I would find it less elegant to combine the two.

If the only benefit is saving 9 characters ("try:\n ") and a level of indentation, it seems like a clear loss to me.

1 Like