Accepting PEP 654 (Exception Groups and except*)

PEPs
21

@Tinche, Would PEP 678 notes solve this problem?

CC @Zac-HD

22

Hm, probably yeah actually. Why do the notes have to be strings though? The root exception group will probably need to be parsed by tools, and having richer data in the notes would be better. @Zac-HD

Why not just have the note be anything, and when printing it out just repr whatever is in there.

23

Mostly because __note__ is intended for display to users, and requiring that notes are strings makes it easy to concatenate them (etc).

If you want programmatic handling of structured data, that’s not really what __note__ is for. Can you say more about the problem you’re trying to solve / API you want to offer?

24

I have a similar use case: Issue 46431: Trouble subclassing ExceptionGroup - Python tracker

25

That repr() call happens while you are rendering an exception. So you get into “what should the interpreter do if repr raises an exception?”, and there is no simple answer for that.

26

Yes, I’d be glad to.

Imagine you have a class that looks like this:

from dataclasses import dataclass

@dataclass
class Test:
    a: int
    b: float

You want to receive it via JSON (an extremely common use case). The JSON libs all produce a dictionary, you use cattrs to structure it into an instance of Test. But what if the payload is malformed (the values for ‘a’ and ‘b’ cannot be converted to int and float, respectively)?

Right now, I have cattrs set up so that the first exception encountered flies out (this would be a ValueError: invalid literal for int() with base 10: 'a_string'). That works, but what if:

  • you want more context during debugging
  • you actually want a better error message for your caller

You need to know which field caused the error, and ideally you’d get all the errors at once instead of one by one. Hence, ExceptionGroups.

So in this case, I suppose, I’d subclass ExceptionGroup:

class ClassStructuringError(ExceptionGroup):
    cls: type  # Useful to have, right?

And the code in cattrs would essentially do this (ignoring a bunch of edge cases now):

def structure_Test(o):
    errors = []
    try:
        a = int(o['a'])
    except Exception as e:
        e.__note__ = "Attribute a error"
        errors.append(e)
    try:
        b = int(o['b'])
    except Exception as e:
        e.__note__ = "Attribute b error"
        errors.append(e)
    if errors:
        raise ClassStructuringError("Structuring Test", errors, cls=Test)
    return Test(a, b)

And for the purposes of displaying a nice traceback what would be adequate.

But what if I want to catch this exceptiongroup, walk its tree and return this via my HTTP API:

{
  "errors": {
    "data": {
      "a": "invalid literal for int() with base 10: 'a_string'",
      "b": "could not convert string to float: 'a_string'"
    }
  }
}

My issue is that with notes being strings, I essentially need to parse my own note. Which is not the end of the world but I feel like I’m doing a formatting/parsing dance for no reason. Maybe it’s OK.

27

Can it bubble out like any other exception?

28

We could be printing to the screen an unhandled MemoryError that has some complicated note. What should happen with the exception from repr(note)?

29

@Tinche I don’t think your problem is actually a use case for notes. The purpose of notes is to enrich the traceback display. You want to associate a value with each contained exception for a different purpose. You could assign it to any field on the exception: e._cattr_context_info = ‘the variable name’.

30

Ah I see. Ok, I can live with that. The base Exception doesn’t have slots defined so this should essentially always work, right?

31

Correct. Exceptions pre-date __slots__ by so much there’s no way we could introduce them on fundamental classes like Exception w/o breaking a ton of code out there.

2 Likes
32

I see the backport doesn’t support __note__ printing, would it be difficult to add?

33

I explicitly excluded __note__ since it’s an unrelated feature from a PEP currently in draft stage. Was I wrong in doing that?

34

TBH I was also confused because the PEP isn’t marked as accepted, but apparently it’s implemented in 3.11?

35

So it seems. Still, this is not part of PEP 654 which the backport implements. I’m not sure it makes sense to have __note__ in just (Base)ExceptionGroup and not other exceptions. I could be convinced otherwise though.

36

__note__ was added accidentally to main w/o a PEP or a discussion with the SC about bringing it into PEP 654 post-acceptance. We have asked for a PEP to discuss the API and make sure it should stay as-is, be tweaked, or come out entirely.

1 Like
37

Ah I see. Since this is out in 3.11a4 I can try around playing with it in a library of mine, would a description of that be a useful piece of data for the steering council to consider?

38

I think just the renderer needs to be tweaked? But if it got merged accidentally and the steering council needs to discuss it further better wait, yes.

39

Would it make sense for the backport to support __note__ even if it ends up being officially adopted in Python 3.11?

40

Sure! If you can provide it to the PEP authors to include in the PEP then it can be part of our consideration.

1 Like