PEP 765: Disallow return/break/continue that exit a finally block

Following the recent discussion, we would like to propose a PEP to specify that python emits a SyntaxWarning or SyntaxError for return, break or continue in a finally block.

Our suggestion is that CPython will emit a SyntaxWarning to avoid breaking working code, but other Python implementations can make it a SyntaxError if they want.

Irit and Alyssa

26 Likes

Overall, the pep seems well-reasoned in terms of motivation, but I think thereā€™s a part here thatā€™s problematic.

CPython will emit a SyntaxWarning in version 3.14, and we leave it open whether, and when, this will become a SyntaxError . However, we specify here that a SyntaxError is permitted by the language spec, so that other Python implementations can choose to implement that.

In a hypothetical case where another implementation takes this and makes it an error, syntax is not portable across implementations. This is not a place I see any benefit in being more permissive of what other implementations do.

4 Likes

break and continue not always worked in the finally block. See issues:

There were also contemporary discussions on Python-Dev or Python-ideas mailing lists about supporting such syntax.

5 Likes

This is an excellent piece of research and perhaps we should do more PEPs based on such investigations.

That said, I am personally sad to see this syntactic corner case disappear. To me, when I designed it, the semantics were always completely logical. (E.g., if you have two nested try/finally blocks, if the inner finally returns, the outer finally still gets to run, and can even override the control flow!)

The idea was that Python has many constructs, and they can be combined in ā€œorthogonalā€ ways. (Something I got from Algol-68ā€™s stated philosophy.) Other languages with finally seem to do it the same way (Iā€™ve only tried TypeScript).

But I understand that in this case users pay a price for that orthogonality, and I have given control of the language to the steering council, whom I totally trust.

Next you might want to have a look at for a[i] in ...: ... or perhaps even the mysterious else clause on loops, which is so often misunderstood.

23 Likes

The PEP is great, and I suppport it. I think the writeup you posted in finally/README.md at main Ā· iritkatriel/finally Ā· GitHub is extremely persuasive; my only comment is that I think youā€™re doing yourself a disservice by only linking to it inline in the PEP (not everybody will follow the link). To me, the analysis is a crucial part of the motivation of the PEP; I think it deserves to reproduced in the PEP in full as an appendix.

10 Likes

As I mentioned in the PEP, we already violated this orthogonality by disallowing return, break and continue in except* clauses. I think itā€™s not a coincidence that both cases are related to exceptions: whatā€™s happening here is that the two program control flows (normal and exceptional) are interacting, and this can become confusing.

AFAIK, at the time that ALGOL '68 was developed, finally wasnā€™t a thing, nor was bare raise. Once an exception handler was handling an exception, this exception was no longer in play and we were back in normal control flow. Now this is no longer the case - we move back and forth between the program and the exceptionā€™s unwinding process, and this complicates matters in a way that ALGOL '68 didnā€™t need to contend with.

An appendix is a good idea, Iā€™ll do that. Thank you.

7 Likes

Iā€™ve never used return / break / continue inside a finally block, and I do not tend to do that in future. So I do not have any opinion on this specific change.

But I do have a more general question: should the interpreter step into userā€™s logic and emit warnings or errors for a likely misused feature (even if the code is logically sound)? If the answer is ā€œyesā€, how much (or how deep) should the interpreter step in? Or is it better to leave these warnings for a static analyzer (linter)?

In my experience, things like that are far less likely to be unintentionally misused.

Iā€™m one that always has to do a double take on forā€¦else constructs to make sure I understand what it means. But for presumably the same reason, I rarely see people write them.

for a[i] in ā€“ I have never seen used, but it also feels completely obvious to me what it means as the ā€œloop variableā€ is always in my mind the same as a LHS of an assignment, so whether perceived as a ā€œgoodā€ idea or not, it is natural that the language support it. I suspect other implementations could omit that and I wonder how many years it would take anyone to notice. :stuck_out_tongue:

Permissive is beneficial. If I understood past discussions correctly, an example was that MicroPython had to jump through hoops in its implementation to even make breaking out of finally clauses work at all? Thatā€™s an example where just sticking with SyntaxError would be beneficial as it simplifies an implementation whoā€™s goal was to be simple.

Many other Python implementations do not have a goal of pedantic compatibility with CPythonā€™s implementation, nor can we expect them to. Really the only one Iā€™m aware of actually doing so is PyPy. This is fine. Other implementations are always free to do what they want. We canā€™t force them to do anything^.


^ The PSF owns the Python trademark so if something were calling itself Python but was not in keeping with the spirit of that, the PSF can legally intervene. Very rare.

12 Likes

I have never personally had a use for this bit of syntax, but I have had it explode on me. Iā€™m a huge +1 on this PEP. Thanks very much.

4 Likes

Maybe people have stopped trying to understand it. I recall vividly seeing it misunderstood ā€“ there seems to be some other language that has a special clause if the loop is empty (PHP? Django templates?) which seems useful for web pages that want to display something like ā€œAll done for the day. Enjoy your empty inboxā€ in such cases, and people were trying to use forā€¦else and pulling their hair out when it didnā€™t work. (The way to remember it is that whileā€¦else is similar to ifā€¦else.)

Agreed. Another thing I took from Algol-68 that definitely went too far.

4 Likes

Sorry, but nothing can ever be changed :wink:.

Iā€™ve used that a lot in some contexts. Typically in backtracking searches, along the lines of:

def searcher(n):
    result = [None] * n
    def inner(i):
        if i >= n:
            yield result
            return
        # compute candidates for current level
        # ...
        for result[i] in candidates:
            yield from inner(i + 1)
    return inner(0)

Very Pythonic in context. One thing Iā€™ve never used is this integer literal: 845653548654354. So make that a syntax error & Iā€™ll never notice :wink:.

14 Likes

Thatā€™s a nice, short and focussed PEP. Iā€™ll add that forbidding these cases may make the compiler or interpreter simpler. I realize things have probably changed a lot on that front, but the exception handling logic used to be a thorny area of the eval loop.

1 Like

Strangely I think that removing return/break/continue is wrong. Iā€™m almost sure I have used this in the past and have certainly used the loop else cases many times.

It seems obvious to me that return statements should always return from the enclosing method/function and break/continue should behave as expected for the applicable loop.

Clearly usage of these statements can be syntax errors if they donā€™t appear in the right scope, but that is true wherher or not they are used in a finally block.

For me Guidoā€™s othogonality is a ā€˜goodā€™ idea; try, except, else, and finally all end up as suites, why treat finally differently?

1 Like

To provide comfort to those with broken code during a transition period, how about a SyntaxWarning in Python 3.14, and consideration of elevating that to a SyntaxError for Python 3.15, to be discussed by the community while Python 3.14 is in place?

The following (meant in jest) has been hidden by request ā€¦

Summary

Incontrovertibly, this forms the basis of a grand idea! The only hitch is that, as a composite number, it may have become someoneā€™s favorite integer literal during observations of the results of some multiplications. Therefore, to gain universal acceptance, allā€™s we need to do is up that number to the next prime, namely 845653548654401. :-;--:

Python 3.14.0 ...

>>> ...
        return 845653548654401

    SyntaxWarning: that forbidden decimal literal

>>>

:-;--:
2 Likes

I assume this was a joke, but to make sure there is no confusion about the process here - weā€™re not proposing to disable features simply because they are not used in practice. We are proposing to disable them because they are typically used incorrectly in practice.

14 Likes

Which is why forā€¦else deserves similar research.

I find the argument of misuse to be a bit silly. I imagine there are many features of python that are misused is it proposed that all such features should be prohibited. Surely a better approach would be to find out why these usages happen and improve the documentation to address those.

3 Likes

While I donā€™t use for ... else every day the times it can be used itā€™s such a nice feature, even if misunderstood and sometimes forgotten. I remember seeing a talk by Raymond Hettinger I think where he argued it probably shouldā€™ve been called nobreak to make it easier to understand.

10 Likes

Of course.

Yes. That an exception can be suppressed in the absence of an explicit except clause is a bug magnet on the face of it. Iā€™m +1 on the PEP.

7 Likes

I expect thatā€™s much harder. Detemining whether for/else or while/else are ā€œcorrectā€ can require deep knowledge of intended semantics. For example, hereā€™s a primalty tester Iā€™ve been using for a very long time:

Code
import math
from random import randrange

def pp(n, k=20, *, gcd=math.gcd, pri=3*5*7*11*13*17*19):
    """
    Return True iff n is a probable prime, using k Miller-Rabin tests.

    When False, n is definitely composite.
    """
    if n < 4:
        return 2 <= n <= 3
    d = nm1 = n-1
    s = 0
    while d & 1 == 0:
        s += 1
        d >>= 1
    if s == 0:
        return False # n >= 4 and even
    if n >= pri and gcd(n, pri) > 1:
        return False
    ss = range(1, s)
    for _ in range(k):
        a = randrange(2, nm1)
        x = pow(a, d, n)
        if x == 1 or x == nm1:
            continue
        for _ in ss:
            x = x*x % n
            if x == nm1:
                break
            if x == 1:
                return False
        else:
            return False
    return True

Is the for/else there correct? Yes. But most readers will have to take my word for it :wink:.

One thing that could be done is to trigger a compile-time error if the loop body doesnā€™t contain a break at all (then the else: clause is unreachable). Iā€™ve seen newbies trip over that more than once on Stackoverflow.

More common, but very much harder to detect:

for ...: # or while
    if condition found:
        # do _everything_ needed if it's found ...
        break
else:
    # do everything needed if it's not found

Thr focus of the search loop is on searching. So parts of the ā€œ# do everything needed if itā€™s found ā€¦ā€ part tend to end up, by mistake, after the loop construct is over. So ā€œbetterā€ syntax may have been a ā€œloop/ifbreak/elseā€ structure:

for ...: # or while
    if condition found:
        break
if break:
    # do everything_needed if it's found ...
else:
    # do everything needed if it's not found

Then the search loop body is solely devoted to searching, and the ā€œfoundā€ and ā€œnot foundā€ suites would be at the same indentation level too.

In any case, while I donā€™t use for/else often, itā€™s a very good fit when it is used, I would sorely miss it, and Iā€™d be astonished if any of my uses didnā€™t work as intended. Itā€™s not hard, just a bit of a learning curve due to novelty.

12 Likes