Request for comment PEP: Make the Exception class a context manager

I’m ready to submit my first PEP proposal. Googled how to go about it, and it seems I should float the idea here first.

Abstract
Make the Exception class a context manager so any builtin Exception can be caught using a “with clause”

Note descendants of BaseException but not Exception will not receive this functionality, and it is rare or unusual to catch these “exit exceptions”.

SystemExit
KeyboardInterrupt
GeneratorExit

Motivation

Pythonic code asks for forgiveness instead of permission.

Consider a function “index” that returns the index of a key in a list/tuple. If the key is not present it returns -1

    def index0(indexable, key):
        if key in indexable:                # Ask permission!
            return indexable.index(key)
        return -1

The above is unpythonic and inefficient. A linear search through an unsorted list is slow, and this search must be performed twice. Once to determine if key is in indexable, and again to determine its index.

The code below is preferred, but rather verbose and ugly

    def index1(indexable, key):
        try:
            return indexable.index(key)
        except ValueError:                  # Ask forgivness
            return -1

Specification

A common design pattern is to use a context manager as an alternative to try/except. Generally the code inside the with block will exit the function if successful and continue after the with block on failure

    def index2(indexable, key):
        with Catcher(ValueError):           # More forgiveness
            return indexable.index(key)
        return -1

The proposal is to add context manager functionality to all exceptions inheriting from Exception. The Exception class, not the Exception object has the context manager methods.

    def index3(indexable, key):
        with ValueError:
            return indexable.index(key)
        return -1       

While it would have to be implemented in CPython as Exception is not a Python declared type; the implementation is conceptually as follows:

    class ExceptionMeta(type):
        """Metaclass to add context manager functionality to Exception"""
        def __enter__(cls):
            """Do nothing and return None to as. Although never expect as clause to be used"""
            
        def __exit__(cls, extype, ex, tb):
            """Return True to swallow exception if ex type matches"""
            if isinstance(ex, cls):
                return True
            

    class Exception(BaseException, metaclass=ExceptionMeta):
        """Common base class for all non-exit exceptions."""

Since context manager functionality is defined in the metaclass; only Exception classes can be used as context managers. Exception instances as caught in except: statements cannot be used as context managers.

    try:
        (1, 2).index(3)
    except ValueError as ex:
        print(ex)               # ValueError: tuple.index(x): x not in tuple
        with ex:                # AttributeError: __enter__
            pass

I don’t see why the Catcher function that you used in one of your examples isn’t good enough. That example is identical you your proposal except that you say Catcher(ValueError) rather than just ValueError.

There’s no way I can see that it’s worth a language change for such a minimal benefit.

In any case, your Catcher already exists as contextlib.suppress:

>>> from contextlib import suppress
>>>
>>> def f(n):
...     with suppress(ZeroDivisionError):
...         return 1/n
...     return -1
...
>>> f(2)
0.5
>>> f(0)
-1
2 Likes

It’s an interesting idea and I think you’re in the right place to discuss it. While I’m not sure it’s worth it (as @pf_moore says, maybe Catcher() is good enough), I’ll just quibble with this part. It’s not that uncommon to catch SystemExit and KeyboardInterrupt.

1 Like

I am not convinced that this example:

def index1(indexable, key):
    try:
        return indexable.index(key)
    except ValueError:                  # Ask forgivness
        return -1

is “rather verbose and ugly”, or that the proposed alternative saving one line:

def index3(indexable, key):
    with ValueError:
        return indexable.index(key)
    return -1       

is an improvement over the contextlib.suppress.

Actually, I would say the opposite: suppress is self-documenting code (at least to English readers). It suppresses its arguments. Whereas using an exception as context manager leaves the question open,

“Yes, I can see that you are using an exception as context manager. But what does it actually do???”

The other issue with this proposal is that this is no replacement for try...except. You can only catch a single exception at a time, and the only thing you can do with it is ignore the exception. There’s no else or finally block.

So the application is very narrow:

  • you want to catch a single exception;
  • and ignore it;
  • without importing contextlib.

Definitely not a replacement for general try...except...else...finally blocks, and the only advantage over contextlib.suppress is that it would be built-in. The disadvantage is that you would need multiple blocks to suppress multiple exceptions:

# instead of `with suppress(ValueError, TypeError, LookupError)` we need
with ValueError:
    with TypeError:
        with LookupError:
            block

I think the minor benefit of having it built in is outweighed by all the other disadvantages, especially the loss of self-documenting code.

1 Like

I defintely also feel any very minor niche benefit of this feature is worth the loss of readibility and one (obvious) way to do it. But I just want to clear up one thing—regarding

Is there a reason you couldn’t just do

with ValueError, TypeError, LookupError:
    ...

as you can with any other context manager?

Well I can see this idea is not popular.

My defense of the idea is that one of the reasons I really like Python is its terse syntax. Any C programmer know the age old argument about braces on single line conditional statements:

for (i=0; i<count; i++)
    if (thing[i] > 0)
        mogrify(i)

against

for (i=0; i<count; i++) {
    if (thing[i] > 0) {
        mogrify(i)
        }
    }

I find those not strictly necessary braces: line noise; and am in the camp that they are not needed.

Perl my least favourite language makes them compulsory along with lots of other syntactic sugar.
Python completely short circuits the issue by using indentation instead.

Likewise with: tuple declarations:

tuple1 = 'spam', 'eggs'
tuple2 = ('spam', 'eggs')

I prefer the first syntax.

So my proposal is to strip out redundant syntax. A whole word suppress and 2 brackets! Gone! Marvelous

Regarding the issue of catching multiple exceptions in Python 3.10 at least it’s possible in one line:

def index4(indexable, key):
    with ValueError_, TypeError_, AttributeError_:
        return indexable.index(key)
    return -1
        
atuple = 'zero', 'one', 'two'

print(index4(atuple, 'spam'))
print(index4(None, 'eggs'))

As a matter of interest once one of the __exit__ returns True all subsequent get no exception.

In the example above the first call all __exit__ get the exception because ValueError_ is the outer:
In the second example AttributeError_ is the inner, swallows it, and the other two see no exception.

I would still say you are doing something unusual if you catch SystemExit, KeyboardInterrupt (and something bizarre if you catch GeneratorExit)

Terminal emulators obviously catch KeyboardInterrupt. Maybe unit test frameworks would want to catch SystemExit

But these are fringe cases and probably quite advanced programmers.

If you have an application with open connections it’s common to catch KeyboardInterrupt so that the connections can be gracefully closed. Generally, catching KeyboardInterrupt is very useful to make sure code is run upon terminating a program. I assume you know all this and mostly disagree with the usefulness.

2 Likes

with ValueError, TypeError, LookupError:

Today I learned.

1 Like

That’s an interesting interpretation. Python is very definitely not designed to be “terse” - one of the statements in the Zen of Python is “explicit is better than implicit”, which generally conflicts with over-terse constructions. You won’t find much sympathy here for arguments that assume that being terse is a good thing (terseness can be good when it improves readability, but it’s the readability that is the goal here, not the terseness).

7 Likes

It reminds me an idea of making int iterable. To write

for i in 10:
    ...

instead of current

for i in range(10):
    ...
1 Like

I expressed myself badly. I have actually written a terminal emulator that did catch KeyboardInterrupt
Amazingly I’ve never written a real communications protocol in Python.

But were I to:

I’d wrap the whole thing in a try finally. No except just use the finally to cleanup before exiting.
Or use a with clause where the __exit__ did the cleanup. Both of those use cases are exception agnostic. You’re really looking for some cleanup code to always run whether an exception happened or not.

In fact I sometimes do:

try:
    return stuff
finally:
    fix(stuff)

to have a post return cleanup or operation.

1 Like

I wouldn’t agree with this. It’s giving an awful lot of functionality: __iter__ and __next__ to the lowliest of objects namely: int

Now not to get sidetracked but I think there’s an argument that len() could return a rich integer that is iterable like range(), but otherwise behaves like an int.

Then again why not just use:

def rlen(thing):
    return range(len(thing))

Don’t worry I won’t be submitting my PEP on this topic. It’s fairly clear it does not find find favour, and I must admit the quote above has given me food for thought.

I like terseness but am forced to agree a keyword like Catcher or suppress does improve readability. Sadly I must admit I didn’t know about suppress in contextlib and has spun my own Catcher doing exactly the same.

Also in a interview when asked what [::-1] does I actually said in a code review I would say change it to to reverse() or reversed() because then it’s implicit what it does. (Also of course reversed() only creates an iterator instead of replicating the string or list)

1 Like

Actually, it’s explicit: you are spelling out what it does quite literally – r e v e r s e.

1 Like

Just to note, Catcher and suppress are not language keywords, they are (context manager) objects like any other.

1 Like

Good to know grammar and linguistic sins (by me) will not be tolerated in this thread. (post, discussion?)

Cry havoc and unleash the hounds of syntax.

Well, this is not exactly a matter of mere “punctuation”; in terms of the Python language, which this proposal is about (or in any programming language, generally), there is a pretty fundamental difference, in both behavior and impact, between a keyword and a mere standard library class object. Clearly understanding and articulating which you mean is particularly helpful to remember in this context of requesting a change to said language, as it aids both the intelligibility and credibility of your proposal.

3 Likes

Sorry if I came across as complaining. I was actually trying to be funny.

Ethan Furman and C.A.M. Gerlach are correct.
My terminology was sloppy and you’re right to expect better from those who want to contribute to the language.