PEP 678: Enriching Exceptions with Notes

Thank you for explaining all this.

Is the info that is returned from the note function in your example supposed to be displayed by python’s built in traceback? If not, can you design your mechanism to use any other field name on the exception?

Yes, I would think that the information in the note function should definitely be shown by Python’s built-in traceback. Forgetting about friendly-traceback for a moment, I think that this addition of a note field is an excellent idea.

As for friendly-traceback, it could make use of any other field name added to the exception. However, I doubt that there would be very much support to add yet more “approved” attributes to the standard Python exception.

I don’t think you need the new field to be approved in any way, you just define it as the friendly traceback api.

The only reason we need the note to be approved as part of the language is because we want to change the interpreter’s built in traceback display code to display it.

A couple of other comments about translations. flufl.i18n is a library I’ve maintained for many years, and which was born from a refactoring of the original GNU Mailman code. There are basically two ways to think about translations, a “simple” API which supports translating simple applications like CLIs, where there’s only one language context in effect at a time. There’s another API which is a little more complicated (but I’d argue still elegant) for situations where you can have multiple translation contexts in play at any one time. In the case Mailman, imagine that you need to translate a page for someone taking an action on a web page, and that action generates a notification to two list owners and a list member. Now, imagine that all four actors of that action have different preferred languages. The service is crafting the notification in one place and then needs to send or display that notice in four different languages. flufl.i18n can and does handle this just fine.

Remember too that translations often have placeholders where runtime data needs to get interpolated into the post-translation string and that languages can and do reorder those placeholders. Maybe that data is only available at the point at which the translated string is being used to notify the user. So in general you have to delay both the translation and the substitution as late as possible to the point where you are actually going to use the final translated string.

This is a real-world example. I have no idea whether __note__ as it’s currently defined, or friendly-traceback can or even needs to handle cases like this.

@encukou: Once libraries start depending on __note__ being either str or None , I’m afraid that loosening won’t be much easier than tightening. See the add_exc_note function in the PEP: projects that adopt it would start failing if the constraint is loosened.

You’re correct of course, Hyrum’s Law is a harsh master :sweat:


@aroberge, thank you for your detailed comments on translation. I really appreciate your (and @barry’s) input, since I know relatively little about this.

Would it make sense to attach the __note__ as a string with the then-current language settings, and an additional e.g. _friendly_note attribute with the translatable string? This would show the best-guess translated note, but on set_lang() you could clear the current note and either re-translate it, or wait for the user to call why(). I don’t have much sense of whether this convenience-vs-rigidity would be an improvement overall though.

You also mention that “third-party projects to effectively add translatable information to exceptions, much more effectively than the proposed __note__ field. However, very few people know about it.”. I’d love to hear more! If you mean Registering custom error types - friendly-traceback 0.5.19 , I don’t think this can replace __note__ without an equivalent way to attach extra information beyond the exception type.

However, since friendly-traceback was mentioned in the PEP, I couldn’t leave the impression that having the note field be a simple string was really the best option for friendly-traceback.

If you would prefer, I’m happy to remove the mention of friendly-traceback or to add a caveat such as “(e.g. friendly-traceback, albeit without translations)”.


Overall, I’d prefer not to incur the complexity of deferring note evaluation to support translations, on the basis that this matches the status quo for exception messages and __note__ can be translated in the same way.

If the steering council would prefer a PEP which proposes built-in support for translation of error messages I’d be happy to support this, but do not have the expertise to lead such an effort.

I can’t speak for friendly-traceback, but it seems that with it would currently have trouble identifying where one note starts and another ends, so it wouldn’t be able to translate individual pieces (and leave unknown ones alone).
Or a GUI would have trouble hiding/collapsing individual notes.

Was a tuple of strings considered? The way to add a note could become something like:

@contextlib.contextmanager
def add_exc_note(note):
    try:
        yield
    except Exception as err:
        err.__notes__ += (note, )
        raise

That would have an additional benefit: writing a composable note-adding library (one that plays well with others) would be easier to do than one that removes other notes.

IMO, the expectation of what libraries should do when adding notes is pretty important to get right, even if the PEP says it’s not “a core part of the proposal”. Where else should it be specified? I don’t think interoperation with other libraries should be left to each individual library. Today’s libraries that want to set notes might be “top-level” – you probably won’t combine asyncio, trio and Hypothesis in unpredictable ways – but that’ll change if exception groups are successful.

I think adding such a caveat might be appropriate for this PEP. From my point of view, something like the following might be the best:

  • programming environments for novices can provide more detailed descriptions of various errors, and tips for resolving them (e.g. friendly-traceback, albeit without translation; see [3]).

And, at the bottom:

[3] While outside the scope of this PEP, if desired by a particular project, support for translation could be added by an additional custom field that would not be shown by the standard Python traceback mechanism; perhaps something like

_translatable_note = lambda: _("Some translatable text.")

= = =
Granted, this would not permit to compose notes (as Petr Viktorin pointed out). However, this would allow the user to focus on solving one problem at a time, without being overwhelmed by too much information.

@aroberge
I’m still confused about the translation problem. I understand that the note added to the standard traceback would not be translated, only the outputs of various friendly-traceback functions would be (is this correct?). In that case, why does the text need to be repeated in a lambda attached to another attribute on the exception? Could the friendly traceback code not invoke translation on e.__note__, by calling _(e.__note__) or something along those lines?

@encukou
I think a tuple of strings might be a good idea, to keep notes separated until it comes time to display them (I think you’re right that people will add more notes than we expect, and not only with exception groups).
Though, it would be just as easy to wipe out the existing notes with err.__notes__ = (note, ) . Should we consider an add_note() function on BaseException? Then __note__ can be a read-only attribute that returns a tuple.

To me, that doesn’t sound like something to guard against.
add_note might be easier to implement if strict type checking for str is needed at the C level… but then again, that check might be deferred to when the note is displayed.
But that’s getting too deep into the implementation, you’re the expert there.

My initial response was to write “No, and here’s why…” … but as I wrote and edited the detailed answer below, I realized that, in a realistic scenario, it’s more like “Actually, something like this might work, but in a very slightly different way.

And perhaps, this is enough to justify not adding the end note I mentioned in a previous response to Zac. This is likely the main point of this reply, and most people can stop reading here.

Apologies to those that were inundated by my previous not-as-well-thought-out replies as I’d hope to provide.

===

I was blinded by the fact that, in friendly-traceback, I keep adding more and more cases (more than 600 translated strings so far, with many more to come), and different translations are updated at irregular rate: so, I consider it important to give multilingual users the possibility to change their preferred language at any time so that they can find the translation that is the most helpful to them.

In this more realistic scenario, for a third-party library, the number of strings to translate would likely be much smaller, and would not have translators struggle in trying to keep up with a constantly changing number of strings to translate. In such a scenario, e.__note__ could be a string and everything would work fine.

However, your suggestion of having friendly_traceback invoke _(e.__note__) would definitely not work. To understand it all, I need to describe how gettext works, probably going into way too much details.

Imagine that you create a simplified Turtle library for an international audience. You write custom error messages and you and your team intend to add translations of these error messages. [Suppose one error message is "A red turtle cannot turn left."]

You surround every string to be translated by a function call _(). You then use pygettext to scan your files so that every translatable string can be added to a template file (extension .pot). For gettext, a translatable string is one that is an argument of _(). So, e.__note__ in the source code is not a string that is translatable. However, e.__note__ = _("A red turtle cannot turn left.") does contain a string that is translatable within your library.

From the template file, you and your team create a corresponding translation file (extension .po) for each language that you support. In addition to being used by gettext to identify the strings to be translated, you define the _() function within your own package so that it can find your translation files.
You package everything together (python files and language files [.po and also .mo]) and upload it to pypi.

In the description of your Library you would likely say “our package includes English, French, and Italian versions”. You also give your users the possibility of choosing their preferred language for your own module/package. This is something that they have to set themselves, most likely as the very first instruction. A Spanish speaker might decide they prefer Italian over the English default.

friendly-traceback defines its own _() function. When it is invoked, it looks in its own collection of translations to find the appropriate string. I can assure you that its own collection definitely does not and will not include the string "A red turtle cannot turn left.". So, it cannot use its own _() function to take the content of an untranslated e.__note__ and provide a translation.
However, hopefully your library would already have create the appropriate translation to e.__note__ by this point.

1 Like

I don’t think it’s harder to type-check with/without add_note, it’s really a matter of which API works better for users of notes.

1 Like

This is actually an important point. If you’re building an i18n’d application, you must only include full sentences as translatable strings. There are two reasons: some languages change the order of placeholders, and some languages essentially cannot translate sentence fragments. So unless you were really careful about how you concatenate notes, you might just end up with untranslatable strings there.

1 Like

This is why I suggested that an object implementing __str__() should/could/would be allowed. If you’re going to complicate things enough to handle the case of anything more than just a str, then I think it makes sense to thing of a better mechanism.

That said, if the PEP accepts not just concrete str objects, but also subclasses of str, then maybe that would be enough to implement more complicated applications?

I would rather revert the feature than make it as complicated as some folks propose. lambda returning str, , something with __str__ method, tuple of strings… Horror!

3 Likes

At least in my experience, that would most likely be something like:

def set_note(e: Exception, color: str, direction: str):
     e.__note__ = _('A $color turtle cannot turn $direction')

The string that translators would have to translate is "A $color turtle cannot turn $direction" and translators know that $color and $direction are placeholders and they are free to change the order of those placeholders based on their language’s grammar etc.

FWIW, the flufl.i18n language grabs its substitutions at runtime from the surrounding scopes, globals then locals, so you don’t have to repeat yourself. Yay for sys._getframe()!

1 Like

Is that the scope where the note is created or the scope where the traceback is being rendered?

Remember that notes have the sole purpose of adding something to the interpreter’s builtin traceback display. They are not intended to be something that applications use for other purposes. Perhaps friendly-traceback should be removed from the use cases for notes altogether, since it is replacing the default traceback by something else.

I don’t think that allowing any-object-with-str is a solution re Petr’s concerns. The issue he raised is how different libraries would coordinate the use of note between them. So this calls for a very prescribed scheme, rather than a very permissive one.

Sorry, friend-traceback adds information to the default traceback.

Edit: one of its goals is to enable users to learn how to understand all the information contained in a standard traceback.

It’s generally the scope in which the _() function is called. There are some corner cases where that needs to be deferred to other scopes, but those aren’t common (e.g. when you want to mark a module global as translatable string for gettext but you need to translate and expand it later in some other call).

Personally I think so, but as I’ve said I’m wary of translatable exceptions for anything other than educational purposes.

Right, but you build your own representation of the augmented traceback. You’re probably not going to replace all of this with notes, whether we make them strings or tuples of strings or lambdas.