PEP 387: including exception messages in the backward compatibility policy

Committers
1

There’s a rule that release managers use when considering backports that is based on a reading of PEP 387 but isn’t explicitly worded out in the PEP.

That rule is effectively “don’t backport changes to exception message strings because user code often parses such messages”.

PEP 387 currently says the following falls under the backward compatibility policy:

  • Documented exceptions and the semantics which lead to their raising; and
  • Function, class, module, attribute, and method names and types.

I would prefer to list “exception message strings” explicitly in the PEP to avoid ambiguity in the future.

Is there anything I should consider before making that edit?

4 Likes
2

I was under the impression that the messages are not supposed to be stable. I’ve seen many PRs changing exception messages.

2 Likes
3

We definitely don’t use any deprecation policy against exception messages but I remember being told to avoid changing message strings in a 3.x branch after 3.x.0 was released. No problem changing the message in 3.x+1.

4

IIRC We’ve changed exception message strings when the existing ones aren’t good enough and are actively confusing users before - but this seems rare and only likely to come up for a new exception that code isn’t likely to have had time to depend on the message of. I consider an exception message that is just plain wrong to be a Bug to be fixed in bug fix patch releases. Adding more detail to an otherwise correct existing message is Feature territory. Otherwise that general policy of avoiding error message, str, and repr, changes makes sense to me.

1 Like
5

In that case the messages belong to the unstable api.

1 Like
6

technically correct… but I suggest we avoid that wording just in case it gives anyone the idea that they should not feel dirty when they depend on the content of an error message. :stuck_out_tongue:

7

I agree that in a stable version such as Python 3.13.x, error messages must NOT change. In a new major version, we can change it, even it should be discouraged if it’s for dubious reasons :grin: For example, just add or remove a trailing dot sounds that it’s not worth it. Fixing a typo or grammar sounds likr a better reason to me.

3 Likes
8

A few months ago I improved a dataclasses exception by adding more info at the end of it. I considered it an enhancement and did not backport. I think Victor’s description is a reasonable one (assuming “major version” describes all 3.13 versions, for example).

2 Likes
9

Error messages can be changed as a side effect of a bug fix which do not change any error message directly. Just the code now uses different function which produces different error message, or handles arguments in different order (both produce errors, but with different messages). I hope that the new policy will not become an obstacle in the way of bugfixes.

5 Likes
10

My understanding has been that we do not change message classes in bugfix releases unless it is a bugfix that really needs backporting, which has been very rare, and even rarely in new releases. But messages are more often changed and are better to not be depended on.

11

Perhaps you meant “minor version”? :smile: Or did you think about Python 4?

1 Like
12

By major, I mean 3.x.0 version, obviously :wink:

1 Like
13

That “rule” is too tight and doesn’t leave room for judgment calls.

In issue [3.12] gh-110050: Adjust the newline position in the TypeError message of the random.seed call. (GH-110051) by miss-islington · Pull Request #110625 · python/cpython · GitHub the OP had a readability concern. The exception message looked weird enough to be confusing. I agreed to the edit and don’t think a new strict rule should block us from a sensible edit.

In general, if the users are net better-off, we should do it. In this case, the likelihood by perhaps a 100 to 1 is that a user will be reading the exception message with human eyes rather than making code that parses it and depends on the exact wording and spacing.

Also, we already have an established practice from doctest to add a ... in tracebacks specifically to avoid a dependency on exact wording.

Lastly, we should be reminded that every backport changes something for someone. I don’t see any reason to single out exception messages as being off limits while actual behavioral changes (bug fixes) are routinely backported. Readability fixes are bugfixes,

2 Likes
14

For me, it’s the same problem than adding new warnings after beta1: projects rely on the exact error message, especially tests. I already saw code checking for the same exact full error message to look for a specific error. doctests is a good example but not the only one. Yeah, it’s a bad pattern, but people use it.

1 Like
15

OK, thanks for your feedback! Based on what was said here, it’s clear there isn’t enough consensus for a black & white rule. Therefore, instead of including exception message strings in PEP 387, I’ll amend the dev guide to say that exception types and exception message strings are often considered API by user code, and as such, changes to them should be carefully considered.

4 Likes
16

Exception types are definitely API. The lack of consensus here is about the messages. I don’t think we should lump the two together like that.

5 Likes
17

What would you suggest in terms of wording then?

18

I liked your wording, I’d just edit out --“exception types and”–.

4 Likes
19

I prefer relied upon instead of considered API.

Or even: While exception message strings are not part of the API, user code does rely on them, so changes to them should be carefully considered.

6 Likes
20

Perhaps clarify that “changes to them should be carefully considered” really only applies to readability edits. If the error message is flat-out wrong, misspelled, or misleading, we can and will fix it.

Perhaps also add a note that this is primarily about reduction of harm and minimizing negative effects on users. We still do emphatically recommend that users treat docstrings, exception messages, and line numbers as implementation details that SHOULD NOT be relied upon.

We know that some users don’t follow best practices and so we will try to reduce the impact, but that doesn’t make it a recommended practice. You really don’t want the PEP to become a license for people to believe that they should rely on message wording.

“If you jaywalk, we will stop and let you cross, but really, don’t jaywalk, it’s not safe.”

9 Likes