PEP 765: Disallow return/break/continue that exit a finally block

[iritkatriel]

@steve.dower

So is it better if the error is still raised? Or better if everyone who uses your module gets a warning (that breaks their own tests/users) whether that error ever occurs or not?

The latter, because the exception will only be raised in some edge case in prod so the whole thing would slip through testing.

[steve.dower]

@iritkatriel

The latter, because the exception will only be raised in some edge case in prod so the whole thing would slip through testing.

But surely the fix to avoid the warning is to make sure that the error is actually raised? Both of the “Error Cases” shown in the research would be automatically fixed by reraising, and neither would get any worse in production. The reactions from most authors suggest that they hadn’t intended to suppress all exceptions, and so reraising would meet their expectations.

I’m missing some crucial piece here that I assume you found during the research. Can you point me to it? (Or better yet, call it out in the PEP in the new section where you’ll reject automatic reraising )

[zhangyx (Yuxuan Zhang)]

The intention of my “deleted” question is to demonstrate optional tasks - it’s good if they succeeded but it does not harm if they didn’t. Just carry on anyway (hence catch all and return).

I agree that the demo code I wrote will be much clearer if “except :” is used instead of finally:. (edit: and make the behavior correct)

folded

I vaguely recall there be a discussion on making bare except equivalent to except Exception: i.e. skipping other BaseException subclasses. If this change takes into effect, then finally will become the only truly catch-all clause unless user explicitly write except BaseException.

[steve.dower]

@zhangyx

I agree that the demo code I wrote will be much clearer if “except :” is used instead of finally:.

Not just clearer, it would be correct But if the return in the finally was conditional, then it may still have been correct. An unconditional return in a finally block is likely not ever going to be useful.

@zhangyx

I vaguely recall there be a discussion on making bare except equivalent to except Exception: i.e. skipping other BaseException subclasses. If this change takes into effect, then finally will become the only truly catch-all clause unless user explicitly write except BaseException.

Only if you also want to catch success cases. I’m not sure where this argument is leading, but I think you may be confusing things (either yourself, or at least introducing confusion to the discussion). In no way should finally be thought of as a “catch-all clause” - that’s not the purpose for it. It isn’t meant to “catch” anything at all, it’s meant to execute code regardless of whether the operation succeeded or failed.

[iritkatriel]

@steve.dower

The reactions from most authors suggest that they hadn’t intended to suppress all exceptions, and so reraising would meet their expectations.

It’s easier to respond to a SyntaxWarning pointing to the place in the code where the issue is, than to some random exception that started showing up when you upgraded to 3.14, and you don’t know where it’s coming from and why it didn’t happen in 3.13. How would you know that some finally used to suppress it?

Also, there were a handful of non-error cases where they did intend to suppress the error, and they assumed that the semantics are as stated in the documentation.

Indeed, I will add a section to the PEP about this.

[steve.dower]

@iritkatriel

It’s easier to respond to a SyntaxWarning pointing to the place in the code where the issue is, than to some random exception that started showing up when you upgraded to 3.14, and you don’t know where it’s coming from and why it didn’t happen in 3.13. How would you know that some finally used to suppress it?

But that forces everyone to respond and churn, while a linter rule can also point straight to the line of code.

I’d prefer to not turn the compiler into a linter. Unless we’re strictly defining the code as illegal, it’s legal. And I’m not convinced this use should be illegal.

[zhangyx (Yuxuan Zhang)]

@steve.dower

I’m not sure where this argument is leading, but I think you may be confusing things (either yourself, or at least introducing confusion to the discussion).

Agreed. My apologies. I’ve folded that paragraph. (I am new to this kind of discussion and am still learning how to properly communicate my thoughts).

[steve.dower]

@zhangyx

(I am new to this kind of discussion and still learning to properly communicate my thoughts).

Glad to have you here participating

[Nineteendo (Nice Zombies)]

@iritkatriel

It’s easier to respond to a SyntaxWarning pointing to the place in the code where the issue is, than to some random exception that started showing up when you upgraded to 3.14

We could add a note to the exception to explain this. (exceptions are now re-raised on exit of the finally block)

But even with that, this would only be feasible if you get notified of the behaviour change during testing (not specifically whether or not the exception is supressed). So, we might want a syntax warning either way, for at least 1 Python version. And of course, code wouldn’t be compatible with both old and new Python versions.

[carljm (Carl Meyer)]

@steve.dower

“Finally blocks executed due to an exception will always re-raise the exception after completion, even if exited early due to control flow (other than a new exception)” seems an easy enough proposal to explain.

To me it seems harder to explain than the current behavior, and no more useful.

The contract of the finally block is that the code within it executes, even if an exception is raised. I don’t see how one could claim that the statement return None has executed if the function does not then return the value None.

These control flow statements would then become unique in a finally block in that, when reading the code in the finally block, one would have to keep in mind that these particular statements are really provisional; they may or may not actually have their stated effect, depending how the finally block was entered.

IMO, “a statement that executes has the effect it says it does” is a much stronger mental-model invariant than “only except blocks can silence exceptions.” (The latter is already falsified by context managers.)

So I think that if these statements are to be legal at all in a finally block, the current semantics for them are preferable.

I do agree that this option should be discussed in the PEP as a rejected alternative.

[tim.one (Tim Peters)]

@Nineteendo

Don’t you mean always reached?

Yup! Good catch.

[steve.dower]

@carljm

The contract of the finally block is that the code within it executes, even if an exception is raised. I don’t see how one could claim that the statement return None has executed if the function does not then return the value None.

That’s a good framing, I can’t disagree. Though I’d still argue that making control flow illegal in a block that’s being used as a finally block isn’t worth the inconsistency.

@carljm

IMO, “a statement that executes has the effect it says it does” is a much stronger mental-model invariant than “only except blocks can silence exceptions.” (The latter is already falsified by context managers.)

Agreed on the first part; the second part makes perfect sense in the context of the try/except statement, but has no need to generalise further. The fact that it’s an except statement determines whether or not the exception will be automatically reraised independent of the content of the code block. Nothing in that disallows other syntax from also handling exceptions (it’s the with syntax that silences the exception; the context manager merely advises the syntax).

@carljm

So I think that if these statements are to be legal at all in a finally block, the current semantics for them are preferable.

This is also an option that minimises breakage, so I’m happy with it as a result. Though it does just mean rejecting this PEP, so it’s not as interesting to discuss

I’m still presuming that some reason was found why it’s urgent to cause some breakage in order to begin to fix other breakage. I’d love for that reason to be clearly explained, especially since I’m going to have to explain it to my (corporate) users who are already very nervous about updating to 3.12.^[1]

---

1. Unjustifiably nervous, I’ll add. But even if I tell them that they’re still going to be ultra careful, and every random little break is going to come back at me with “we rolled back a million-machine deployment because of this message, why is it there?” (not a hypothetical…) ↩︎

[carljm (Carl Meyer)]

@steve.dower

I’m still presuming that some reason was found why it’s urgent to cause some breakage in order to begin to fix other breakage.

I don’t know if there was a single precipitating reason, but the PEP’s argument (with evidence) is that in practice this will not break ~any code, it will just alert people to existing bugs in their code, because ~nobody is currently using this feature correctly and intentionally.

Personally I tend in principle towards “allow compositions of syntax that have a reasonable interpretation, even if it’s not obvious at first” and “the compiler is not a linter” — but I do find it hard to argue with the research evidence in the PEP, which leaves me somewhere around +/- 0 on the PEP.

(Of course, a lot of existing code is not visible to that research; I’m sure that somewhere someone is using this correctly and will be inconvenienced by making it illegal.)

[tim.one (Tim Peters)]

@Nodd

. I associate try: with exception handling more than except, as in “something will happen when an exception is raised”, and the fact that this “something” happens in finally seems perfectly logical to my. Maybe I haven’t used python for enough decades

You’ll get there after another decade . “Something” is nuanced: it may be suppressing the exception entirely, or it may be executing some code regardless of whether an exception occurs. except is the job of the former, and finally the job of the latter.

The only coherent contract finally can make is “the suite will be executed no matter what, but if an exception was pending when the suite was entered, that exception will be propagated when the suite exits”. That contract is currently violated if the finally suite exits via break, continue, or return, and that’s a design error.

In some cases it may not matter, but the research they put into writing this PEP showed that this (mis)feature does a wrong thing more often than not in real code.

In this case finally functions as a bare except:

Yes, but only because my example had a return in the finally suite. In the absence of break/.continue /return, finally acts in no way like any spelling of except. As at the start, except and finally have very different purposes. “Well, sometimes finally acts like a bare except too, but sometimes not, depending on whether the finally suite falls off the end” is a mental mess.

While I’m no longer Guido’s oracle, I can speak for what he would have thought 30 years ago: no frickin’ way .

[dimaqq (Dima Tisnek)]

Following same logic, raise A() within finally is also suspect, isn’t it?

[tim.one (Tim Peters)]

@dimaqq

Following same logic, raise A() within finally is also suspect, isn’t it?

Oddly enough, that one does not suppress the original exception. Instead it adds a new one to the chain:

>>> z = 0
>>> def f():
...     try:
...         x = 1/z
...     finally:
...         raise NameError()
...
>>> f()
Traceback (most recent call last):
  ...
ZeroDivisionError: division by zero

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
    ...
NameError

[dimaqq (Dima Tisnek)]

How about this then: within finally I’d raise a new exception and catch it and then fall through? With or without a helper function call?

[tim.one (Tim Peters)]

@dimaqq

How about this then: within finally I’d raise a new exception and catch it and then fall through?

Does what I expect: catches the new exception internally, and goes on to raise the original exception when the finally suite falls off the end.

With or without a helper function call?

I don’t see why that would matter, so didn’t try it. You’re free (and encouraged!) to run your own experiments, though.

[steve.dower]

@tim.one

Oddly enough, that one does not suppress the original exception. Instead it adds a new one to the chain

Since we added exception chaining, at least (around 3.3 or 3.4, IIRC?). Clearly someone decided then that preserving the original exception mattered, despite the previous behaviour being to suppress it.

[breaker]

As an outsider, I find the idea that the removal of “for…else” is even being considered at all to be shocking.

Python already has a reputation as having the worst long-term language-level stability and churn of any widely-used language. Backwards-incompatible changes should be extremely well-justified. In this case, “for…else” is the Python feature I find myself looking for the most when I’m using other languages. It’s incredibly useful and widely used. Maybe a better syntax could be chosen and the current one could get a warning, but the idea of making that an error at any point should be beyond consideration.

For the actual subject of this PEP, I’m not thrilled but if there’s data showing it’s genuinely rarely used, okay, that’s fair enough. But “for…else” is an awesome construct that sets Python apart in the best way and it’s widely used. Please don’t create more compatibility churn, and cause the community lose more unmaintained libraries yet again, without good reason.