Extend type hints to cover exceptions

Typing
41

I understand the arguments of both sides.
I would like to see the ability to indicate a recommendation to handle certain exceptions.

As most simple I perceive a new keyword like „propagate“ that I can use to indicate that a certain condition might raise an exception that the caller should handle. A typical use case would be a connection error on a http request. So I could write sth like

def f():
try:
request_data()
except ConnectionError as ex:
propagate ex

So that a type checker can give me a hint to handle ConnectionError when I use f()

42

How would the type checker know who should handle the exception? It’s not necessarily the caller.

43

How is this different from just raise?

3 Likes
44

With apology on reviving a pretty-dead thread, I would like to note a completely different use case for exception annotations.

Specifically, I may want to specify an interface with a function signature that indicates which exceptions will be thrown under certain edge cases. As a basic example, with the proposed syntax:

class MyObjectHandler(ABC):
    ....
    @abstractmethod
    def get_by_id(id_: str) -> None ^ NoSuchObjectException:
        ...

A type checker will not check that you handle or raise all exceptions that might be raised in your code, or even that you explicitly hint all exceptions that you raise; instead, it may check only that you do raise those exceptions that you explicitly hinted.

This is very useful for two reasons:

One, that when you implement MyObjectInterface, you’ll know to raise the proper exception if a user tried to get a nonexisting object. Today this info will sit in the docstring of the method in the base class, but this is much less concise and much easier to miss — IDEs will generate the function signature in implementation stub, but not the docstring.

That Interface implementations handle edge cases in a consistent manner is extremely important, because the functions that use these interfaces must be able to handle them themselves. The code that uses MyObjectHandler wants to know when the get operation fails for this reason. Will it return None? Will it raise an exception? If so, which type? Being able to look at the type hinting both at implementation and at use has an immense benefit.

Two, at runtime, frameworks that generate UI out of pythonic functions, like fastapi and typer, will be able to automatically generate documentation for error responses.

As an example, fastapi could provide some way to specify the status code of exception types, and then if it knows that an api function raises NoSuchObjectException, the generated openapi.json could detail a possible response of 404 if the requested object does not exist. This would save a great amount of time writing down your possible responses by hand, which is what you have to do currently — as fastapisimply has no way to know otherwise.

2 Likes
45

I think that wording is an issue. Throughout this thread, I’ve seen a lot of talk about checked exceptions. But, the word checked is problematic as that’s not what hints are about (note that this is the last use of the word “checked”). They are hints that can be ignored in some of my codebases. Just like I choose to not ignore type hints in some codebases, it would be nice to not ignore exception hints in some codebases too.

Custom forms of Exception Hints are already possible via Annotations.

import typing

Exceptions = typing.Union

class MyNonFatalException1(Exception): ...
class MyNonFatalException2(Exception): ...

def foo() -> typing.Annotated[int, Exceptions[MyNonFatalException1, MyNonFatalException2]]:
    ...

With this, I can continue to ignore exception hints just as I already ignore exception hints written out in docstrings.
However, this is just one custom way of doing it. You could write this in many other ways. It would be nice to have standardized syntactic sugar for hinting exceptions so I can ignore them in a standardized way.

3 Likes
46

Ignoring them in a standardized way is already possible, just don’t write them (or write them as comments) :slight_smile: The hard part is when you want to NOT ignore them, and you’re trying to do that in a standardized way. Because at that point, we’re right back to the question of “what does this annotation even mean?”. It’s easy to say “let’s have an annotation about exceptions” and much much harder to find a single semantic meaning for that annotation that everyone can agree on.

When I write a function like this:

def spam(msg: str, when: int | datetime.datetime): ...

we can all agree that this function is supposed to receive a string as its first parameter, and an integer or datetime as its second. That is well-defined semantics. So what are the semantics for this exception annotation? Does the function have to strictly include the statement raise MyNonFatalException1 in it? What if it calls a different function that raises that? Should it be annotated or not? People do not all agree on this which is why we still haven’t gotten anywhere.

When you are NOT ignoring these annotations, how do you interpret them?

1 Like
47

What exactly are the implications of this form of hint for a tool like mypy?

1 Like
48

Two more links for completeness:

3 Likes
49

I really like the idea of a exception type check, and I think this can still be done in a pythonic way.

I think for the most part their is merit in just ignoring the type of exceptions entirely, then in an except block, the ide can list all the expected errors that might happen in the try block, and then the developer can either handle one or more of the errors as shown by the ide, or handle whatever error they want to. This will narrow down for the developer what errors they could expect, and reduce the overhead of reading documentation.

Often there are exception that can explicitly be handled, but go unnoticed, either because of poor documentation or because of it’s obscurity in development situation or even because you forgot.

Then the unhandled errors can just go up the chain. So you can just write however you want, but when handling error you’ll get a little extra help.

Also for things like __next__ and contextlib’s contextmanager, it can help a little, by giving you a consice description of what error you should raise.

It’s almost always easier for me to code if I don’t need to read documentation to do what I want to do and if the ide can give me appropriate suggestions depending on context, but not restrict me too much in terms of how I write my code. Which is why I like python so much. And this can help without really adding much in terms of restriction.

As I understand, this wouldn’t be the traditional checked exception, but a very loose variant of the same.

One last note, which isn’t as important, I often just write down a bunch of code and document it only after I’m done writing all of it, so having an extra help on handling error that was thrown in another file, can very much help me.

3 Likes
51

I think that Java gave checked exceptions a bad name. They had a new toy and they got carried away and made things like IOErrors checked, which is a real pain - what are you going to do about it?

However if done well and importantly optionally, the feature is very handy. I notice that language designers are coming back round to this way of thinking, e.g. Swift has added them (swift-evolution/proposals/0413-typed-throws.md at main · apple/swift-evolution · GitHub).

Their syntax is:

func callCat() throws(CatError) -> Cat {

In Python something similar might be:

def callCat() raise(CatError) -> Cat:

Or perhaps with a decorator to save any new syntax.

@raises(CatError) 
def callCat() -> Cat:
8 Likes
52

I actually found this really helpful as a flag for code that does I/O. I wouldn’t want it added to Python just for that, of course! But interestingly, a lot of the pain points with checked exceptions like IOException are also there with asyncio – you can’t easily add in I/O to code that didn’t do it before, and that makes some things like controlled rollout of new code paths separately from release cycles harder.

3 Likes
53

Well I don’t really think that an exception goes directly behind the → because it’s not really returning anything

54

Comparing your NoReturn[ZeroDivisionError] syntax with the above proposed @raises(ZeroDivisionError) syntax, I’d prefer the latter. Though this is another option:

from typing import Raises
def callCat() -> Raises[Cat, CatError]: ...

The decorator syntax is a bit more pythonic, but we already have TypeGuard and TypeIs, which use the return-type-annotation instead of a decorator (the rationale for that can be found in the original PEP: PEP 647 – User-Defined Type Guards | peps.python.org ), so for consistency, I’d prefer the return-type syntax. In addition to the argument written in the PEP, I’d also add that:

  • Raises[Cat, CatError] can be used when using Callable: Callable[[], Raises[Cat, CatError], avoiding the need to always define a callable protocol for this
  • the Raises[Cat, CatError] syntax would also allow the use of generic exception types; this might be useful when forwarding exceptions from callbacks

Regarding the semantics, I guess the main question is whether type checkers would enforce that all uncaught exceptions by other function calls within the function body must be forwarded. I think that’s what most comments against the idea are worried about: that you end up having to declare a huge number of exceptions for any function, just because somewhere in the call stack there was a function that can raise.

First, it seems like a good idea to exclude KeyboardInterrupt and MemoryError from this mechanism because they can be raised anywhere. (Are there other exceptions like this? Maybe IOError should be excluded as well?)

Another thing that would help, I think, is if Raises[Cat] would be short-hand for Raises[Cat, Any], which conveys the information that some error may be raised, without specifying which one in particular. You can use this any time it becomes too tedious to track specific errors.

Alternatively, one could remove any kind of claim of totality from the mechanism (i.e., the declared exceptions are explicitly allowed to only represent a subset of the possible exceptions). In this case, the job of the static type checker is just to check that raising the declared exception is possible at all. So:

def f() -> Raises[float, ZeroDivisionError]:  # error
    return 1.1

would be a type-check error, because the type checker can’t see a way for this exception to be raised.

However, this would be allowed:

def f(flag: bool) -> Raises[float, ZeroDivisionError]:
    if flag:
        g()
    return 1.1

def g() -> Raises[float, ZeroDivisionError]:
    return 1.0 / 0
55

Conceptually this is very similar to warnings.deprecated (spec, docs), which also statically describes a side-effect, not a type. Especially if you consider the fact that you can override the default DeprecationWarning, for example @deprecated("don't", category=PendingDeprecationWarning) [1]. So in that sense, the deprecated name only describes its most common use-case, as it can be used to statically describe the effect of any Python warning: it might as well have been called something like emits_warning.

So I don’t think it would be strange at all to also have something similar to describe the effect of raising an error. The proposed ^ syntax would be a nice way of doing so IMO. But if we want to be conservative about it (which would likely be the easier path here), we could also also go with an API that’s syntactically closer to the familiar warnings.deprecated decorator, e.g. @typing.raises(exc: type[BaseException], msg: str, /).

  1. this unfortunately isn’t supported by most modern static type-checkers, even though it seems very like a useful feature to me ↩︎

56

I don’t think this belongs in the type system unless it’s going to be proposed as something actually checked by typecheckers. The proposal of Never[ExceptionType] is also itself problematic, because there are non-exception ways to have a function which doesn’t return (infinite loop for a heartbeat in a daemon thread, for example). It would be more useful to differentiate between the various effects than it would be to differentiate between the various exceptions for the specific effect of raising an exception.

Pretty much every python function is capable of raising as well.

1 Like
57

This actually reminds me of the 1 situation where I want to annotate errors, which is with @overload to annotate ‘forbidden’ arg and kwarg combinations.
Currently I believe @deprecated is the only way to tell type-checkers that specifying a or b is allowed but specifying both isn’t. @typing.raises(NotImplementedError) or something in that direction could be slightly tidier? I’m not entirely sure.

1 Like
58

You can use overloads for that and don’t need a way to “annotate an error”, the absence of an overload that accepts both covers it.

1 Like