Add `with_notes` method to exception classes

Ideas
1

Currently, it is non-trivial to attach a large number of notes to an exception from an iterable. Calling add_note for each item looks ugly and many method calls incur overhead. Fortunately, the notes are stored in the __notes__ attribute of the exception, which is a list of strings. But that attribute doesn’t exist when there were no notes; it is only added on the first add_note call! Therefore, I propose to add a with_notes method to BaseException that takes an iterable and returns the original exception after the addition of the notes, just like with_traceback sets the traceback and returns the same exception object. Something like:

def with_notes(self, notes, _=object()):
    if (first := next(notes := iter(notes), _)) is _: # check against a sentinel
        return # no notes to add
    self.add_note(first) # create the __notes__ attribute if it doesn't exist
    self.__notes__.extend(notes) # works because __notes__ is always a list
2

Isn’t this a bit over engineered? How many notes are you adding for this to become a performance issue? Are your exceptions not exceptional enough? Is this something you’ve profiled?

for note in notes:
    exc.add_note(note)

Alternatively, maybe combine all notes into a single large note. Can probably be cached with functools.cache but it’s hard to imagine a cache being much faster than a join:

exc.add_note("\n".join(notes))
6 Likes
3

I’m used to add a note (just one - unlike the OP) quite often:

try:
    ...
except Exception as err:
    err.add_note(f"This error occurred while processing job: {job_id}")
    raise err

and would welcome a method similar to the suggested one:

try:
    ...
except Exception as err:
    raise err.with_note(f"This error occurred while processing job: {job_id}")
5 Likes
4

It may just so happen that one would like to report a discrepancy between the keys of a dictionary and a container, where the container should be a subset/superset of the keys. In that case, I personally use map to format each missing/extra key in a certain way and add them as notes to the exception. Something like:

raise ValueError('unknown keys:').with_notes(map('- key %d: %r'.__mod__, enumerate(extra_keys)))

would greatly simplify my code. This may be a niche pattern though, and Xitop’s suggestion makes sense.

5

That would be the same as some

raise ValueError('unknown keys:').with_note(
    "\n".join(string for string in map(
        '- key %d: %r'.__mod__,
        enumerate(extra_keys)
     ))
)

In my opinion only a with_note method should be needed, but I agree it would be a useful utility either way. The pattern of adding a note, then raising the exception should be common enough, so I don’t think a lot more reasoning should be required.

6

This approach makes sense, but then it would be harder to parse the notes. I’m open to either though, because I have conceded that my use case is very specific.

7

Are you trying to use notes to pass information around via exceptions?

If this is used for flow control and it happens often enough for there to be performance considerations then have you considered creating a custom exception type? Then there’s no need to “parse” the notes because the unknown keys will be stored in a format you have full control over.

class UnknownKeys(Exception):
    def __init__(self, extra_keys):
        self.extra_keys = extra_keys
        super().__init__(*extra_keys)  # Unpack into exc.args
8

I just want them to format like notes, and this is the most straightforward way to me.

9

Note that if you don’t need the frame that adds a note to be included in the traceback you can simply re-raise:

try:
    ...
except Exception as err:
    err.add_note(f"This error occurred while processing job: {job_id}")
    raise

This is unlikely to be accepted because it goes against the design philosophy of Python’s built-ins and stdlib that any method that modifies an instance in-place should not return the instance.

By the way, instead of adding a note you may also consider raising a custom exception with the original exception chained:

try:
    ...
except Exception as err:
    raise JobProcessingError(f"Error processing job #{job_id}") from err
3 Likes
10

Maybe there is a chance. I think the primary cause for that design decision was a deep dislike of fluent interfaces, i.e. chaining of several such methods. But chaining is unlikely to happen here. And if need be, .with_note could accept variadic *notes.

I like the notes and I’m glad they were added, but their support feels like unfinished. This [re-raising with a note added] is one of the points. The other is that str and repr do not print them, only the traceback functions do.

2 Likes
12

I only thought of this suggestion because with_traceback set a precedent.

3 Likes
13

Ah true that. Having a precedent certainly helps the case, though in the case of notes adding a with_note method when there’s already an add_note method seems a little comparatively redundant.

1 Like
14

The reason is that it makes a clear distinction between mutating and non-mutating methods and helps to catch errors that result from mixing them up. It does preclude fluent interfaces, but I don’t think that’s a problem. I see fluent interfaces as a workaround for languages not having certain features, such as keyword arguments. Python does have those features, so it doesn’t need fluent interfaces.

1 Like
15

That would be good, but it is not the case. On my python 3.13, with_traceback does mutate the object

>>> x = ValueError()
>>> x.__traceback__  # default constructor sets __traceback__ to None
>>> y = x.with_traceback(tb)
>>> y.__traceback__
<traceback object at 0x7f0411211880>
>>> x.__traceback__
<traceback object at 0x7f0411211880> # x was mutated
>>> x is y
True    # actually, with_traceback does not return a copy

The (only) difference so far is that

>>> x.add_note("abc")
>>>

add_note returns None, not x.

3 Likes