It doesn’t really matter now, the PEP’s been submitted, but won’t be reviewed until the next steering council starts work.
I’m not going to participate in any more discussion. My -1 remains on the current PEP text, and my reasoning is fleshed out as well as ever in the above posts. I’ll only chime in again if people start claiming that consensus exists.
The point here is that we don’t think this should be valid python code, it’s not useful and it causes people to write bugs. We want to remove this from the language but without breaking stuff. Linters are not in a position to make such a declaration. Should there be no way for python to do that?
One question - do syntax warnings trigger when the code is compiled? Because if so, the biggest problem I have with warnings (that they appear at runtime, to an end user who is likely not the developer responsible for the offending code) may not apply.
But of they work like normal warnings (only triggered when the offending line of code is executed) then I agree with Steve, we’ve had too much experience of warnings causing more problems than they solve to think this one will be any different.
Yes, they would show up at compile time.
Yes. Stick this in, say, a.py:
assert (3, 5)
If you import a, you’ll get a SyntaxWarning. But the .pyc file goes on to be created, and on later runs import a won’t complain again (so long as the .pyc file still exists).
OK, thanks.
The more I think about this, the more confused I get, though. The first time code with this problem gets imported in a fresh environment, the user will see a warning, but never again after that. But if they use -Werror, they’ll see the error every time, unless they didn’t use -Werror the very first time, when they’ll never see the error. Or something like that… And all of this applies both to the library developer and to any users of the library?
And what’s worse, the only way of fixing it is for the library developer to change the code (and release a new version). Because as far as I can see, the runtime warning filter doesn’t work for syntax warnings:
❯ py -c "import warnings; warnings.simplefilter('ignore', SyntaxWarning); assert (1,2)"
<string>:1: SyntaxWarning: assertion is always true, perhaps remove parentheses?
While I appreciate that this problem may be very rare, a syntax warning seems like it will cause at least as many problems as we’ve had with other forms of deprecation warnings. IMO, if we think this needs to be prohibited, we should stand by our decision, and simply make it an error. If there’s too much risk of disruption from doing that, I think there’s also too much risk of disruption using a syntax warning.
This explanation was in the previous thread, but I need to double check it made it into the PEP text: the fact end users mostly won’t see the warning (even with -Werror) is a feature of the suggested solution rather than an oversight.
The warning will be emitted when translating the source code to an AST. This means static analysis will trigger it, compilation will trigger it, but mere execution of already compiled code will not trigger it.
With those triggers, maintainers of a project will almost inevitably see the warning (even if they don’t run static analysis, CI often doesn’t have files precompiled), but the only way for an end user of an affected project to see it is if they skip precompilation at installation time, check installation time warnings, or run static analysis over their dependencies.
Does it matter whether it’s done in src->AST or in AST->bytecode? (Do we need to by so specific in the PEP)?
If it’s in the second stage, static analysis won’t trigger it, rendering it less effective.
While that’s more a “quality of implementation” decision than part of the spec, it is also based on the general idea that if something is a syntax error, we shouldn’t produce an AST for it, and a syntax warning should be emitted at the point where the error would be detected and rejected (if it was classified as an outright error).
If I have a correct understanding of the current state, this would be the first time a SyntaxWarning is issued for code that could be correct for the real world. All other examples I can think of are for situations where python knows that the code will not/is no guaranteed to do what the user wants:
assert (a, b) → assert can never be tricked, not what the user wanted(5 (2 + 4)) → int is not callable, not what the user wanteda is 5 → while technically always doing what the user wanted within CPython, currently, the very similar code a is 500 does not do what the user wants, and different Python implementations might be even more different.C:\Users\... → during the time this was a SyntaxWarning it did technically do exactly what the user wanted, but it was strictly decided that this will change soon.So this PEP introduces a new kind of SyntaxWarning: The code probably doesn’t do what the user wants, but we are never going to make it a full error.
I don’t like this. Python shouldn’t guess what the user intended to do (see Zen of Python). So unless there are plans to make this a SyntaxError, this should not emit a SyntaxWarning.
I expressed similar concerns in this post above (but not as well-worded as yours, hence did not receive any reply yet).
Edit: this does not mean I don’t support this proposal - I do support it. I just think the approach to fix it needs to be discussed.
I don’t see this as being particularly different. In fact, the backslash escape example broke a lot of “perfectly working” code, where this is going to highlight a case where there’s a high likelihood that the code you wrote isn’t doing what you thought it was. (I didn’t have that issue with Windows paths, but I’d often stick “\e[…” into something, expecting an ANSI sequence, and it wouldn’t work.) Granted, the backslash escape issue fixes a wart in Python where nearly every other language already had that as an error; but that’s not a fundamental distinction.
Right, I didn’t fully say this in that first sentence: I am willing to accept the SyntaxWarning for invalid escape sequenes because there were clear plans to turn it into hard errors in the future. Originally it was a FutureWarning you would only get when you opted in, and was then turned into a SyntaxWarning to notify more people before actually going through with the change.
I haven’t seen a concrete time line when return in finally will turn into an error. And I don’t think we should add these kinds[1] SyntaxWarnings unless we actually plan to follow through and completely forbid it.
I.e. the code has perfectly sensible semantics ↩︎
Or do both. The PEP currently allows for that implementations other than CPytnon can call it an error right away, and that CPython may do so too, but at an unspecified time.
I too would like the clarity of a statement that this will become an error in a specific future CPython release, and that the warning release(s) is(are) just to ease the transition.
Although I have mixed feelings in some ways, overall I don’t think the PEP is a good idea. I agree with @steve.dower that the rarity of use of the construct is an indication that there is no major problem to be solved, rather than an indication that we can solve it painlessly.
The reason my feelings are mixed is that in general I’m not opposed to breaking things if it makes the language better, and I think that “Would we do things this way if we weren’t constrained by past decisions?” is a good way to think about things. The problem, though, is that breaking small things in a piecemeal fashion gets you most of the user irritation with little of the benefit. I have serious doubt that a bunch of people will be overjoyed to get a warning and find that this change caught some lurking bug in their code; instead I’d bet that the rare users of this functionality will be incensed that their code was broken to prevent such an obscure bug (and perhaps they will come out of the woodwork to complain about it).
When such small changes are made here and there over time, they create a frustrating slow drip of incompatibility errors over many Python versions. In my view, it’s actually better to adopt a “punctuated equilibrium” philosophy in which breaking changes are infrequent but may be large. This gives the opportunity to provide more substantial benefits (by baked-in discarding assumptions or constraints that have blocked evolution in certain directions, and thus opening up real new possibilities), and makes the choice clearer for those who are considering upgrading. Basically, having to do a big compatibility overhaul every once in a while is less of a pain than having to do small ones very often.[1]
So overall I think breaking compatibility is sometimes a good thing, but it shouldn’t be done by just finding individual warts and fixing them here and there, rather by taking deliberate, holistically considered steps that may include multiple changes. This means that, paradoxically, although I don’t support this PEP because of how it breaks backwards compatibility, I might support an even more radical PEP that does what this one does and also breaks backwards compatibility in additional ways. 
I know that many people believe that the Python 2/3 transition showed this was not the case, but I disagree. I think that transition could have been handled in a way that resulted in far less pain even with the same breaking changes. ↩︎
If the intent is to remove this feature at some point, then adding a deprecation warning sounds like the natural way to do this from a CPython POV.
However, as Steve noted, such warnings tend to hit the user more than the developer, often to the effect that people simply mute such warnings (the data science world is full of deprecation warnings and those don’t look nice in notebooks).
IMO, we should probably reconsider the approach to such deprecation notices. Linters are indeed being used a lot during software development and the outputs are indeed noticed early by developers (unlike deprecation warnings at runtime).
This is why I believe that we should start reaching out to popular linter projects to get the word out early, rather than pester users with deprecation warnings, as we’ve done in the past.
Now, whether or not to remove this particular feature is a good idea or not, I cannot really say. I’ve only ever used try...finally when I had to make sure that cleanup code is run no matter whether an exception occurred or not.
I can imagine programmers who want to run cleanup code and in certain cases override any pending exceptions as a result of successfully running the cleanup, e.g. lets say you tested some API feature and found that it doesn’t work, but you don’t really care why it didn’t work.
Those programmer would then exit the finally clause with e.g. a return. The only other way would be to wrap the try...finally in another catch all try...except, which looks rather ugly, or you could have some logic using a catch all try...except and a boolean to mark success of failure, which doesn’t look nice either.
We’re proposing a syntax warning at compile time. Nothing at runtime. The last two comments are arguing against ‘breaking changes’, which this is not.
(Before that there were comments arguing that this is not breaking enough.)
Irit, I think we’re not aligned on what we’re talking about Let me clarify things a bit…
Sorry, when I said “runtime” I was thinking of running the application with CPython, which usually involves both compiling the code and running it.
You’re right, of course, that the SyntaxWarning would be raised a compile time, since it would be the compiler finding the feature being used and warning about it.
What I wanted to highlight is the difference in showing such warnings to users vs. showing them to developers.
Especially for deprecation warnings, the developer will be aware of these after a few runs, but may decide that it’s not the right time yet to do anything about them, while the user will always see them and then get confused.
I understood your comments to mean that you intend to make this a breaking change eventually and are preparing for this by introducing a warning now.
My comments were meant to provide an example where the feature does make sense, even though I myself have never used it, just to add a data point.
We’ve added the following paragraph to the PEP regarding the difference between runtime and compile time warnings:
The CPython implementation will emit the
SyntaxWarningduring
ASTconstruction, to ensure that the warning will show up during
static anlaysis and compilation, but not during execution of
pre-compiled code. We expect that the warning will be seen by a
project maintainer (when they run static analysis, or CI which
does not have precompiled files). However, end users of a project
will only see a warning if they skip precompilation at installation
time, check installation time warnings, or run static analysis over
their dependencies.
I agree that it is possible to use this feature correctly. I found eight places in 120 million LOC where it is possibly used correctly, alongside 149 cases where it is obviously used incorrectly. All the possibly-correct cases were easy to rewrite without the problem feature.