Exceptions some time can be lost

Python Help
1

Main problem: exceptions some time can be lost.
Assume E1 and E2 are two completely unrelated exceptions.
(For example E1 is FileNotFoundError and E2 is AttributeError)

class E1(Exception): pass
class E2(Exception): pass

def raise_double_error():
	try:
		raise E1(1)
	except E1:
		raise E2(2)

def main1():
	try:
		raise_double_error()
	except E2 as e:
		pass
		# comletely handle E2 but not E1

main1() # E1 lost

Let’s try to fix it: (already I need some extra code)

def main2():
	try:
		raise_double_error()
	except E2 as e:
		# comletely handle E2 but not E1
		if e.__context__:
			raise e.__context__ from None
	if orig:
		raise orig

try:
	main2()
except E1 as e:
	print(repr(e.__context__)) 
	# During handling of the E2, another exception E1 occurred
	# what???
	# but this message will be supressed by e.__suppress_context__ because it raised from None

I want to propose behavior of next code to make automatic by default when people write code as main1

def main3():
	orig = None
	try:
		raise_double_error()
	except E2 as e:
		orig = e.__context__
		# comletely handle E2 but not E1
	if orig:
		raise orig
		# if main3() called from another except E3 clause
		# E3 should be appended to tail of orig.__context__ chain

try:
	main3()
except E1 as e:
	print(repr(e.__context__)) # None
2

E1 wasn’t lost; it was handled inside raise_double_error by catching it and raising E2.

Do you want the caller of raise_double_error to see both exceptions as separate entities, or do you want to see that E2 is raised while attempting to handle E1? You may want to look at exception groups (8. Errors and Exceptions — Python 3.14.7 documentation)

3 
def foo():
	raise E2
def bar():
	foo()
def baz():
	bar()
def raise_double_error():
	try:
		raise E1(1)
	except E1:
		1/0
		do_some(obj.missing_attribute)
		baz() # raise another unexpected exception so E1 remain not handled
		# handle E1

How to use exception groups in such cases?
I want handle E2 and don’t miss E1.

4

Also to distinguish unexpected raising as above and explicit raising as:

except E1 as e1:
	# partially handle E1 and replace it to E2
	raise E2() from e2

I want to propose compare e1 and e2 and if e2 is e1 then don’t E2().__context__ = e1 and to deprecate E2().__suppress_context__

5

Your examples are a bit hard to understand. Here you have code in the except block which apparently will raise three more exceptions (ZeroDivisonError, AttributeError from the “missing attribute”, and E2). There isn’t any way to “handle E1” while simultaneously “automatically” handling or ignoring other exceptions that may happen during that handling.

If I saw that code my response would be that the handling of E1 should be done first in the except block, and then after that (maybe in a finally) do the other stuff that might raise other exceptions. Or, if the possibly-exception-raising code is necessary for handling E1, then you have to bite the bullet and include nested try/excepts to deal with each piece.

Aside from that, this kind of thing is very strange:

	try:
		raise E1(1)
	except E1:
		1/0

Why Raise E1 just to catch it immediately? Can you come up with a more realistic example?

6 
errors = []
def raise_double_error():
	try:
		with open("non_existent_file.txt", "r") as f:
			return f.read()
	except FileNotFoundError:
		text = "FileNotFoundError caught"
		# unintended bug:
		print(text.uppercase())  # AttributeError: 'str' object has no attribute 'uppercase'
		errors.append(text) # FileNotFoundError doesn't completely handled
def main(result_container):
	try:
		result_container.result = raise_double_error()
	except AttributeError:
		text = "AttributeError caught"
		errors.append(text)
		# AttributeError completely handled
	# I expect here any other exceptions to fly right through.
class AnswerContainer:
	__slots__ = ('answer',)
	def __init__(self):
		self.answer = None
answer = AnswerContainer()
main(answer)
print('answer:',answer.answer)
print('=== caught errors: ===')
for e in errors:
	print(e)

I get

answer: None
=== caught errors: ===
AttributeError caught

FileNotFoundError lost

7

Again, the except clause doesn’t somehow magically ensure that everything inside it will happen regardless of exceptions raised within that block. If you have code in the except block that might raise another exception, but you want to make sure that later code in the except block still runs, you’ll need an additional nested try/except.

	except FileNotFoundError:
		text = "FileNotFoundError caught"
		# unintended bug:
		print(text.uppercase())  # AttributeError: 'str' object has no attribute 'uppercase'
		errors.append(text) # FileNotFoundError doesn't completely handled

The situation here is just what you say in the comment: it is a bug. If some of your code doesn’t run because an exception occurred that you didn’t plan for, and it causes your code to behave in a way you didn’t intend, that’s just a bug in your code.

8

And the best way to avoid having bugs in your code is to have less code. Simply allowing an exception to bubble requires no effort, introduces no bugs, and in many MANY cases, is the best way to see what errors are happening.

If you’re bothered that exceptions are getting lost, stop catching them.

9

I used to think that any function could throw any exception, and if it wasn’t caught (and handled), it would break the entire program, fly to the root, and I’d be guaranteed to see it.
If I would expect only certain errors from each function, I will build all error handling on if/else.
Aren’t exceptions designed to catch errors/bugs in code, among other things?

I don’t think anyone assumes that catching one exception can lead to the loss of another exception.
Are you suggesting not catching exceptions at all? But then why is try/except needed?
Or are you suggesting that I keep in mind about each function, which exceptions it can throw and which ones it can’t, and try not to put unnecessary try/except blocks?
Won’t it increase the code if I do a lot of small try/except around only the necessary parts? Is it a bad practice to put try/except in the root of the program?
You should keep in mind that some of the code may be a library of some kind.

I think the language is poorly designed if it stealthily swallows errors or forces me to write a lot of additional boiler plate code.
And I wanted to suggest/discuss ways to remedy this situation.

10

It won’t break the entire program, but the rest is correct: you are (generally) guaranteed to see it. So yes, you should let them escape unless you can actually HANDLE them. Catch exceptions only when you can fix the underlying problem. Otherwise, let them through.

And yes, exceptions report on bugs in your code, among other things. But the best way to detect bugs is to see those exceptions, not to try to catch them. Don’t reinvent error reporting. It’s there waiting for you at zero effort.

Well, yes, that WOULD be poor design. Fortunately, it doesn’t. Errors won’t be swallowed unless YOU swallow them. You don’t have to write boilerplate to report errors, that happens by default.

11

In other words, you are saying that it is extremely unlikely
that another exception will occur while an exception is being processed in one code,
and in another code (calling the first one) this other exception will be successfully handled?

12

It is (usually) a symptom of poor engineering if there’s a possibility of an unintended exception being raised inside an except block. Keep your except blocks minimal, wherever possible.

Try/except are needed in cases where you expect an exception to happen. Using an except block is you explicitly handling an exception. Your default should usually be to let exceptions bubble up to the top level. Error messages are your ally.

13

None of the examples you gave shows that. In all of your examples, the exception is “lost” because you caught that exception. Not another exception. The same exception. If you have code like this (which is basically what you have in all your examples):

try:
    raise E1
except E1:
    # stuff here

. . . the exception E1 is “lost” because you caught E1. You didn’t catch some other exception, you caught E1.

Imagine that an exception is like an alarm that starts ringing when a certain thing happens, and continues to ring until it is turned off. You seem to be conceptualizing it like this:

try:
    raise E1 # alarm starts ringing!
except E1:
    do_stuff()
    # alarm is turned off and stops ringing

And so you think that if something goes wrong inside do_stuff(), the alarm should still be ringing, because it hasn’t been turned off yet. But it’s actually like this:

try:
    raise E1 # alarm starts ringing!
except E1: 
    # alarm is turned off and stops ringing
    do_stuff()

The alarm is turned off right at the beginning of the except block, and so if something goes wrong in do_stuff, no other code ever knows about the E1 alarm, because it was already turned off.

This is why, as others have said, you should keep the except block as minimal as possible, to avoid something else going wrong that could interrupt the handling of E1.

14

Keep your except blocks minimal, wherever possible.

Is there a way in the language to make sure that the function I call in the except block doesn’t raise an exception?
Or do you suggest making an additional boilerplate in each except block

	... 
except E1: 
	try:
		handle_E1()
	except BaseException: 
		crash_program()

?
What if this happens in code that I don’t control (in some library)?

The alarm is turned off right at the beginning of the except block

Yes, the behavior is like that now, but it shouldn’t be like that.
People understand this and have already thought of something:
Adding a turned off exception to the .__context__ of the new exception.
Enable .__suppress_context__ if the new exception is raised like raise E2() from ...
Now it is necessary to at least do a warning to stderr when it has E2 (as e2) processing is completed and e2.__context__ is not None and not e2.__suppress_context__.

By the way, in __exit__(self, e_type, e, traceback) this situation is handled correctly: alarm is turned off only after it returns True.

15

There is no need for your boilerplate, and if there is, it is a sign that you should refactor your code. If an error occurs in your third-party error handling function, it is indicative of a bug on their end, so you should simply allow it to propagate. Still, it is unlikely that you’d have to use a third-party error handler that is so complex that it is potentially faulty. Moreover, it would be very disruptive if a warning was emitted every time. The bottom line is that you should not catch errors just to reformat the information within unless it indicates malformed input by the user or inconsistent external state pertinent to your code, since the traceback is usually enough to locate a bug.

16

it would be very disruptive if a warning was emitted

it is unlikely

17

Your idea of the error handling is a little bit off. I try to give you some hints:

  • Exception handling is not omnipotent, you need to respect its limits.
  • You should only handle what you expect. Do not try to handle anything else.
  • Do not try to handle programming bugs! These must be resolved first (testing) before your expexted error handling can work. Do not try to catch them.
  • A zero devision (one of your examples) might be an expected error or a bug. In the first case try to handle it as local as possible.
  • Keep exception handler as simple as possible. You can also remember thing_X has happend, and then proceed outside of the except block. This should allow you to handle an expected thing_Y afterwards probably and in combination with thing_X.
18

Here’s a better question: why are you calling a function that’s likely to raise an exception inside an except block? While all functions can raise (such as raising a KeyboardInterrupt if the user presses ctrl-C in the console to intentionally terminate), most never will. In particular, you basically shouldn’t ever do except BaseException - 99% of the time, you want those exceptions to go all the way up