Different behavior of `BaseException.with_traceback` in Python 3.11 vs 3.10?

Python Help
1

Here is a short code example. I’m chaining function calls f, g, and eventually h where an exception is being raised. The traceback, as expected, shows the call to f (in <module>), then the call to g (in f), then the call to h (in g), and eventually the raise (in h):

>>> def f(): g()
... 
>>> def g(): h()
... 
>>> def h(): raise Exception()
... 
>>> f()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in f
  File "<stdin>", line 1, in g
  File "<stdin>", line 1, in h
Exception
>>>

As far as I understand, I can use BaseException.with_traceback to modify __traceback__ of an exception. In my case, I want only the most recent step of the traceback to remain, i.e. the actual line (raise Exception in h) that caused the exception to show up. (Similar to sys.tracebacklimit = 1.) So I’m using tb_next to “walk” to the most recent step of the traceback, set that with with_traceback as the traceback of the exception and then raise. This does exactly what I expected and what I need:

Python 3.11.3 (main, May  3 2023, 08:13:29) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(): g()
... 
>>> def g(): h()
... 
>>> def h(): raise Exception
... 
>>> try:
...     f()
... except Exception as e:
...     tb = e.__traceback__
...     while tb.tb_next is not None:
...         tb = tb.tb_next
...     e.with_traceback(tb)
...     raise
... 
Exception()
Traceback (most recent call last):
  File "<stdin>", line 1, in h
Exception
>>>

However, unfortunately, this is only the case for Python 3.11. If I run above code in Python 3.10, it’s like with_traceback has no effect:

Python 3.10.11 (main, May  4 2023, 06:08:16) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> def f(): g()
... 
>>> def g(): h()
... 
>>> def h(): raise Exception
... 
>>> try:
...     f()
... except Exception as e:
...     tb = e.__traceback__
...     while tb.tb_next is not None:
...         tb = tb.tb_next
...     e.with_traceback(tb)
...     raise
... 
Exception()
Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "<stdin>", line 1, in f
  File "<stdin>", line 1, in g
  File "<stdin>", line 1, in h
Exception
>>>

How can this different behavior be explained? Isn’t this a bug in Python 3.10 that has been fixed in 3.11 but not backported? I had a look at the changelog, but couldn’t find anything that would match my issue. Same for the commit history in Objects/exceptions.c. So I’m probably just overlooking it?

2

To me, this sounds like it should work in Python 3.10 like it currently does work in 3.11:

with_traceback(*tb* )

This method sets tb as the new traceback for the exception and returns the exception object (…)

Source: https://docs.python.org/3/library/exceptions.html#BaseException.with_traceback

I couldn’t find any recent changes (to explain the difference in Python 3.11 vs 3.10) mentioned in the above linked docs for with_traceback. Also, there’s nothing mentioned at https://docs.python.org/3/reference/datamodel.html#traceback-objects either.

3

In What’s New In Python 3.11 — Python 3.12.1 documentation, it says:

  • When an active exception is re-raised by a raise statement with no parameters, the traceback attached to this exception is now always sys.exc_info()[1].__traceback__. This means that changes made to the traceback in the current except clause are reflected in the re-raised exception. (Contributed by Irit Katriel in bpo-45711.)

I take that to mean that this difference in behavior is intentional. As for why it’s been changed, I think it likely has to do with the next point on that list:

  • The interpreter state’s representation of handled exceptions (aka exc_info or _PyErr_StackItem) now only has the exc_value field; exc_type and exc_traceback have been removed, as they can be derived from exc_value. (Contributed by Irit Katriel in bpo-45711.)
1 Like
4

Oh… I did not realize that besides https://docs.python.org/3/whatsnew/changelog.html (which is where I was looking for mentions of __traceback__), there’s also a much more detailed changelog for each release, https://docs.python.org/3/whatsnew/3.11.html in this case (which does mention the changed behavior of __traceback__ modifications).

I take that to mean that this difference in behavior is intentional.

Sure! Maybe I misunderstand but it sounds like you’re defending the change from 3.10 to 3.11? However, I meant it exactly the other way around: I was looking at the 3.11 feature as a bugfix and was wondering why it hadn’t been backported to 3.10. That has been cleared up by the link you provided.

Thank you very much for your help.

5

Oh, no, I just meant that it looks like this is a new feature for Python 3.11.