PEP 484: When will we have syntax for Exceptions?

Typing
21

Besides the standard library there is a library annotated types, GitHub - annotated-types/annotated-types: Reusable constraint types to use with typing.Annotated, that tries to standardize some annotations. This could live there. I’d recommend issue for that library and if maintainers are receptive to the idea having it there. If it grows in usage as documentation/other tool it can be re-explored in a few years.

I’d be fine with exceptions in annotations primarily intended for documentation and similar tools. As long as type checkers don’t enforce some notion of checking exceptions putting exceptions somewhere in return annotation seems fine. You could add sphinx plugin (explore sphinx-autodoc-type hints) to use this raises annotation.

3 Likes
22

Would you only annotate exceptions explicitly raised in this function? That can be statically analyzed by finding raise statements. Are you trying to annotate those exceptions that might be raised by other functions? Then we’re right back to “everything could raise BaseException”.

What would this information be used for?

1 Like
23

What do you think of the idea of using this for abstract methods? I’m not sure if I can get a good look at what the implementation is calling for. docstring? Yes, but you can do it nicely without nonPythonic text, right?

Again, this is just self-documenting code. I don’t want to read the library code, the creator can just give me the API and documentation for it.

24

I’m imagining this used in similar way docstrings sometimes have raises clause that lists some notable exception types that library method may trigger. It’s not intended to be exhaustive and is more meant as advice for user to consider them. Considering them may mean doing nothing and thinking that error should be propagated.

Numpy is a good example, Style guide — numpydoc v1.7.0rc0.dev0 Manual

An optional section detailing which errors get raised and under what conditions:

Raises

LinAlgException
If the matrix is not numerically invertible.

This section should be used judiciously, i.e., only for errors that are non-obvious or have a large chance of getting raised.

1 Like
25

Okay, so what’s the advantage of them over docstrings? Tools that currently show docstrings to users would have to be changed to reflect this annotation, and what would they do with it other than show it as if it had been part of the docstring?

If it’s just for the human, how is the docstring not sufficient?

1 Like
26

The main advantage is it’s more easy to manipulate annotations at runtime than parse and check docstrings. I think it is more style preference as with strict docstring formatted/linter you could entirely do it in docstrings and rely on them to follow some rules that make it easy to work with.

The other aspect is I prefer to read type hints from method signature vs from doc string. Before type hints and even now some libraries included types as part of docstring. I could see a similar argument for exceptions being more readable to some in annotation near the return type.

I have weak view on actually doing this but think it’s reasonable if some library authors want to use Annotated to include exception metadata. As long as type checkers/typical linters don’t try to enforce it.

I don’t use it for exception metadata but do frequently use Annotated to attach other metadata. I have a lot of dataclasses intended to define config files where I include in Annotated documentation strings per field that could be used to generate information of config’s schema.

Edit: I also do not think argument for this is strong enough to warrant inclusion in standard library today. If people explore using more metadata in annotated for signatures and exceptions become common after a few years for some known libraries maybe it can be revisited. Now it feels too early.

2 Likes
27

One of the advantages of rolling your own is that you get to try it out
and find the pain points or shortcomings. You can also offer it on PyPI
for others to experiment with or to built features around (eg to
generate docs for the exceptions expected).

I’m not dissing your idea: I myself have yet to read the previous
threads.

But a purely leading design discussion (like this or the preceeding
threads) only gets you so far. Trying out a working example such as
yours lets your and others explore the suggested design.

3 Likes
28

Could you please stop characterizing proposals for using the annotation system to document potential exception types that could be raised, as “checked exceptions”? It is not the same thing at all, any more than the existing use of annotations is “static typing”.

3 Likes
29

Okay, sure! Show me how they’re different. If function f() raises BadThingieException and function g() calls f(), there are two possibilities:

  1. g() does not declare any exceptions. In this case, your exception annotations have practically no value, as they can be trivially viewed by static analysis of raise statements
  2. Or g() must declare that it raises BadThingieException (or catch it in that function). In this case, it is checked exceptions.

So I’ll stop calling them checked exceptions once they stop being checked exceptions. What is your use-case for these annotations, if it isn’t to show every exception that this or what it calls could raise?

3 Likes
30

Is numpy’s recommendation of exceptions in documentation checked exceptions? The current non-enforced annotation proposal is closer to that. There is not an intent to enforce g must specify exception and no tool to actually check for it. This is closer to a documentation recommendation where documentation does not live in docstring.

Normally checked exceptions implies some enforcement of exceptions being propagated in signatures. With no enforcement the current proposal is focused mainly on optional documenting what users way want to be aware of.

For you specific example, the answer would be is g under normal usage of your expected audience likely to trigger FooException? Then consider mentioning it similar to documentation style rule. There is no hard requirement to include and I’d guess some libraries if they followed this practice would include it only for major ones (mentioning list_files function may return non-existent directory but not other exceptions) while some libraries may mention it more commonly.

Direct answer to your example is what g actually calls mostly doesn’t matter and only exceptions maintainer thinks are worth mentioning should be listed. Python standard library similarly sometimes calls out certain methods raise certain errors.

31

I’m guessing you mean this?

https://numpydoc.readthedocs.io/en/latest/format.html#raises

That’s exactly the sort of thing that I’m saying that this is equivalent to, and as such, is no better than. There’s no advantage to creating a new system, then trying to get it added to documentation parsers etc, when you could just use the docstring. Unless you want something to be checked by an automated tool, the docstring is perfectly acceptable.

32

I thought I did, repeatedly, last time we discussed this; but again: they don’t cause a fatal compilation error when they’re wrong. Checked exceptions require a static compilation type-checking pass to exist in the first place; Python doesn’t have that, and the existing use of annotations simply does not count.

Can I not say the same about type annotations? If function f declares that it returns a str, there is nothing (first-party) that requires g to declare a return type, nor a type for anywhere that the result of f() is assigned, etc. Yet, if I publish f as a library, making that declaration does have value, whether the client code wants to use type annotations for g or not.

I don’t think that static exception analysis is any more trivial than static type analysis, certainly not in a language as flexible as Python. (For example, nothing prevents me from writing code that, for example, deserializes and raises a valid exception instance from a pickle file.) Aside from that, a third-party tool that is already built to examine function signatures statically (but not to examine bytecode statically) will have a much easier time reasoning about exception declarations in annotations than it would about raise statements.

To document (i.e., not enforce checking, while offering an option to people who want to use third-party tools) exceptions that are salient to the normal operation of the code, i.e., which don’t indicate either a programming error or an unplanned-for condition in the environment (ctrl-C is fundamentally different from a malformed input file). Even languages that actually do have checked exceptions (like Java) have no difficulty making distinctions like that, btw, and programmers can follow simple rules to choose whether to check their own.

By documenting the potential for exceptions in this way, the information becomes more readily available to tools that can warn about potential problems.

I don’t even like type annotations, but there are people who want them, and their reasons are understandable and well considered. The reasoning about exception annotations works the same way.

1 Like
33

The entire point of exceptions is that any function you call can raise an exception, and if you don’t handle it, it simply bubbles up. That is why they are of value. That is also why, unlike return values, they are not inherently part of the API.

So they should go in the docstring. So far, you haven’t given a good reason for them to be machine readable.

So, just the ones that this function raises, or also those that bubble up? You still haven’t answered that question, and it’s a rather important question.

The distinction between “unpreventable” and “recoverable”, as given in the SO answer you linked to, is a false dichotomy. Only the function’s caller can determine this.

(Also, really not a fan of Stack Overflow links. I strongly dislike its method of encouraging people to spam their links everywhere in order to boost their own reputation on the site, which benefits the site by increasing ad impressions. It seems like a dirty way to boost revenues. Can’t you provide your arguments here instead of forcing us to go look at your preferred site?)

2 Likes
34

A nice part about docstrings that follow specific formats is that they become machine readable!

35

And the docstrings won’t be bubble up?

I think the code with doc string looks much more unpleasant than with normal annotations. Going from the opposite, why do you use annotations? Use doc strings to describe method signatures \_(._.)_/

Your argument revolves around the fact that exceptions will bubble up. But you can simply ignore them or handle, for example. And the API developer can show you not all exceptions, for example ignore such ones as KeyboardInterrupt.

36

I don’t think we’re going to get anywhere with this discussion, so I’m going to bow out of it. Thing is, I don’t have to prove that your argument is meritless; the onus is on you to prove that there is value in this. And once again, that hasn’t happened.

37

I agree. I will delete this topic. I think the idea of annotating exceptions is a good one, but not now. At the moment, it is useless in most programs.

38

I don’t recommend deleting topics. Leave the record there for the next person to come along with the same thought.

1 Like
39

Ok, well, maybe it will give someone a better idea.

40

Yes; with my mod hat on please don’t go around deleting ideas just because they were rejected, for the reasons @Rosuav mentioned. Doing so would be disruptive to the community and not very respectful of both your and others’ time in proposing and debating the idea.

Furthermore, it also defeats much of the purpose of the #ideas category, so that people can look at previous proposals and discussions to see what’s been discussed already; otherwise we’d just get the same idea over and over and have to rehash the arguments endlessly, which doesn’t really help anyone, nor does it give people a basis to introduce new followup ideas to address objections to previous ones.

Thanks!

5 Likes