This is a not-quite off the cuff 4 point plan, that’s more process related than I would have liked.
Is there anything in this that’s problematic to turn into a pep? Is there anything missing?
No new errors should be added to the language that are not properly filtered by python’s warning filtering facilities.
Existing warnings should be fixed to conform with python’s filtering facilities (There is already a PR open for this)
There should be a new warning base class for “lint-like” things that are not uniformly errors so people can opt out of the effects this has on the signal to noise ratio.
The current SyntaxWarning in 3.14 for return/break/continue in finally should be a new error type inheriting from both SyntaxWarning (compatability with those already relying on it being a SyntaxWarning) and this new lint-like error category, as it is impossible for the language to know if it is intentional, and is just an opinionated guess without the willingness to just change the language to remove that syntax.
This 4 point plan prevents a repeat case where a warning is seen as valuable by some, added, breaks other systems because it doesn’t work.
It won’t fix the issue for those running into it for the currently not possible to filter properly warning added in 3.14, but it will prevent new issues of this type from not having suitable well defined and stable interactions with existing workflows.
I don’t want to turn this into a rehash of fighting about pep 765, but for those unaware, the problems with it were recently summed up in the thread about it.
This summation is relevent to points 1 and 2, and I believe that if points one and two are adhered to as part of process, it would prevent a repeat.
While I somewhat agree with 3 and 4, I would really prefer if such new warnings were just left to linters in the future.
Something that was proposed in that thread that sort of slipped by in much of the frustration was the potential for a core developer endorsed lint configuration that was tailored for things that are known common footguns, and a way to easily ship this with the interpreter, but as a seperate tool the interpreter runs (Even if it runs it by default to help those who don’t know to know to turn it on), not as part of the interpreter core itself.
Would this be a blessed linter of sorts that is part of the stdlib, or an external tool? If the former, I’m not sure it would be worth the effort for us to write a stdlib module that would necessarily be less up to date than third party offerings. If the latter, which would we recommend? We’re not in the business of picking winners, and there are many and varied approaches to static analysis on the market (including some prominent ones not written in Python!)
I don’t know. The only reason I bring it up was it being another middle ground between two extremes that I didnt want lost for consideration.
I would imagine that it could be provided as a pure python module, but the rules themselves could be reimplemented at a faster pace, and backported for prior python versions by any linter that wanted to, and that this would be an infrequently used mechanism, but help create a clear divide on attempting to help those who don’t know to configure specific tools to help them, and being too noisy for those who want it to only catch things that we can be confident are the result of something going wrong and requiring warning of.
For whatever it’s worth, I noticed this and it reminded me of this thread.
My opinions haven’t changed much since last year, but I didn’t feel that there wasn’t any signal that CPython devs wanted to add something.
I’m not game to start a new project right now but if nobody else acts first, I don’t mind starting to prove it out on PyPI.
I support the ideas in the OP here. They don’t seem that controversial and they leave room for features to be flagged as warning-worthy without breaking anyone for using those behaviors.
Of course, some teams will treat all such warnings as errors and forbid such usages. That is their prerogative and it shouldn’t impact the overall community judgement that some things are dangerous if used improperly but useful when used correctly.
While I spend all my effort to fix the warning filtering issues, I do not think any of them is critical.
Issue with filtering by module name is old. It never worked as you could expect, and there was even regression in 3.13. But this was not critical for 5 other syntax warnings.
Duplicate warnings were mostly just annoying for the REPL users. This was fixed in 3.14.0 at the cost of introducing the following bug. There were also duplicated warnings before 3.14.
Swallowing some warnings happens only with the default filter, and only if the warning is emitted at the same line in different modules. This can show less warnings that there are, and this can annoy when hunt warnings, but we don’t really know how large impact on real projects.
I’ll add also that starting with 3.15, you can use regular expressions for module name and warning message work in -W and PYTHONWARNINGS. This will help to manage warnings.
I think the reaction to PEP 765 is overblown, especially comparing to other syntax warnings which affected more code.
It may make sense to stop using SyntaxWarnings altogether and instead point people to the linters we have available. They have become quite powerful tools and are much more useful at hinting at possible quirks in the code than anything we could ever do with SyntaxWarnings and the filter mechanisms for warnings.
For the PEP, I’d also focus less on the particular issue with PEP 765, but more on a new general approach, e.g.
Make SyntaxWarnings opt in via a command line / env var flag for those who want to continue relying on them.
Start deprecating SyntaxWarnings altogether in favor of linters.
Perhaps someday we can also do the same for deprecation warnings, but we’d need an extensible standard for this, so that extension modules can flag their own deprecations to linters as well. Different idea and off-topic for this thread, though.
Does anyone know whether SyntaxWarnings are in current use by extensions ?
First of all, I’d like to thank you for the work you’ve done in reviewing the PR which was linked in the other thread, as well as the interactions and discoveries which that split off into being multiple separate actions.
I don’t think it’s reasonable to compare the prior syntax warnings with this one, there’s a reason this includes proposing a new warning base, and adding that to this warning.
At the same time, part of the reason it’s gotten to this point of wanting to propose formal process is that people seem content to write off broken warning filtering as not a big deal, or overblown because this time the impact seemed small to them. This wasn’t a last minute concern in the case of 765, this has been ongoing discussion with actual impacted users for months.
I don’t want to come on to discourse and have to argue relative impact to have faith in something the language provides, and the process argument is the last one I’d make.
(Taken from a now locked thread, into the one where I intend to do as suggested and try and improve the case for use of warnings.)
Given the tone of the response was in reaffirming 765’s acceptance, and not at all addressing that the issues that came up did not require reversing 765 to have been addressed, I want to reiterate that while I think it isn’t the place of the interpreter to act as a linter, I was fine with it up until it reached users specifically who weren’t supposed to reach it by the pep’s design [1] and that it wasn’t filterable using existing filtering mechanisms.
As has been said already, other syntax warnings weren’t filterable before. However, all prior existing SyntaxWarnings are unambiguously erroneous. The most favorable example is the below:
>>> a = 1; print(1 is 1)
<stdin>:1: SyntaxWarning: "is" with 'int' literal. Did you mean "=="?
True
In other words, the filter may have already been broken for syntax warnings, but prior syntax warnings were so conservative in what they warned for, that this wasn’t observed or fixed, because it was pointing out broken code and filtering it made no sense to do.
Given this context, and the tone of the response, I have to ask if the steering council was only aware of the theoretical disruption that comes with all new warnings, or was it intentionally pushed forward with the steering council aware that it was also not working as the pep advertised (the “clever” mechanism of using the bytecode cache didn’t have the intended effect of only the library author seeing it) and that it was also in the context of the warning not being filterable?
These issues were brought up in the discourse thread relatively early on, and there was a PR available to fix the filtering issue that did not make it into 3.14.
I’d like to be able to trust warnings as errors going forward, so I’d like to understand what parts of the process led here that I no longer feel I can so that I can propose something that actually addresses what failed here.
The pep specifically attempted to only reach library authors by use of the bytecode cache. It was shown early on in the release cycle that users that were testing ran into this with library code in CI. ↩︎
This was linked in the 765 thread, along with showing it was impacting more than intended relatively early on.
Unclear to me. Many of the arguments made seem to suggest that people involved in decision making don’t see this as an issue, which is part of why I’m trying to clarify what exactly the intent was in decision making. If it’s truly seen as a non issue by decision makers, and it’s deemed that no amount of warning behavior including the capability of filters is subject to stability, that’s also fine with me provided that’s documented so that I and others can point employers to something going “look, right here. we need our own solution, this isn’t meant to be stable.”
If a fix is backported, it still on it’s own won’t prevent a repeat issue or help anyone whose test matrix needs to include 3.14.0, and this will still incorrectly (by 765’s definition of who is meant to see it) show to people beyond the author.
Warning messages are normally written to sys.stderr, but their disposition can be changed flexibly, from ignoring all warnings to turning them into exceptions. The disposition of warnings can vary based on the warning category, the text of the warning message, and the source location where it is issued. Repetitions of a particular warning for the same source location are typically suppressed.
There are two stages in warning control: first, each time a warning is issued, a determination is made whether a message should be issued or not; next, if a message is to be issued, it is formatted and printed using a user-settable hook.
The determination whether to issue a warning message is controlled by the warning filter, which is a sequence of matching rules and actions. Rules can be added to the filter by calling filterwarnings() and reset to its default state by calling resetwarnings().
The printing of warning messages is done by calling showwarning(), which may be overridden; the default implementation of this function formats the message by calling formatwarning(), which is also available for use by custom implementations.
but later there’s fine print that SyntaxWarning’s can’t be for reasons I don’t really understand. (It’s been over thirty since I did the compiler course).
I have to imagine the interpreter could be refactored to buffer SyntaxWarnings until after parsing the *py file, and only issue them if they are not specifically suppressed.
Right, I’m aware of that fine print, though it was also written when Syntax Warnings were only ever things that weren’t ever correct. We’re now in an era where correct code, written using the language as designed (and this has been reconfirmed multiple times that the use was as intended by the design of the syntax) emits it. That language was also written at a time when it was more common to tell people the interpreter wasn’t a linter.
This is part of why it’s so hard to tell if it’s even possible to call a change that fixes this a bugfix (and therefore possible to do in a patch release), and why I was arguing that it shouldn’t be a SyntaxWarning in the first place.
The part where I think it isn’t acceptable to say “this isn’t breaking” is that the ability to turn all warnings into errors exists, and that works on this, yet the filters do not. If the filters aren’t going to apply, why are they promoted to an error by the same facilities?
While not explicitly documented directly that way, I believe that’s effectively the case so long as warnings are going to be added that may not be filterable and it remains common that warnings as errors are used in CI/CD pipelines.
I also don’t like this, because it feels like a technical loophole to not care about the impact on people who want to use warnings as errors, and prior conservative use of warnings didn’t need to promise more, because there was trust that warnings were meaningful.
Other parts of the standard library take care not to break users with additions. Take a look at the collections abcs. It’s not unheard of to understand that additions to some APIs can be disruptive in the standard library. There’s a giant block comment at the top of the collections abcs explaining why things shouldn’t be added to existing collections there.
I’m not going to make an extreme argument that we can’t create a warning system where additions can be reasonably managed, but I don’t find the status quo acceptable. As-is, it’s just a loophole to get around actual breaking change policies for syntax. The warning did break libraries and their users, and effectively required people change that syntax to unblock CI pipelines. It wasn’t an option to just acknowledge the warning and move on.
So, even if you’d be fine with a more explicit “this can’t be relied upon”, I don’t think the existing behavior or documentation of warnings should remain as-is, and I’d rather see warnings removed from the language entirely than left in their current state.
The entire point of warnings is to be a diagnostic that can have false positives.
When you use warnings as errors, you have opted in to sometimes changing your code to avoid false positives. The only way to avoid that would have been to limit diagnostics to only those that cannot ever have false positives. But then, that’s what we call errors.
Here’s what leaves me stumped. Using warnings-as-errors is a signal that you think it’s extremely important that no warning is ever overlooked or unhandled. But then there’s this call for there to be fewer warnings so that warnings-as-errors users don’t have to change their code. I don’t see how to reconcile those two sentiments. Either warnings are important to you, and then you should want more of them, even if it causes you extra work. Or they’re not, and then you should not be using warnings-as-errors.
It’s increasingly clear to me that a lot of people, and possibly even the entire steering council don’t actually get the point of warnings as errors.
The point is advance notice of possible issues. The problem is that in all reasonable use of warnings as errors, you make a determination when a new warning arises on what to do. If it’s something that doesn’t require a code change because you know it is okay, you suppress the warning.
This works best when warnings are either conservative and not a substitute for a linter, or when categories of warnings can be opted into (see things like clang options like -Wpedantic), and still requires that specific warnings can be suppressed.
This has become an issue worth discussing because the language has stopped being conservative with warnings (765 made using a language feature as it was designed to be used, a warning), and those warnings can’t just be acknowledged and move on, so the use of warnings as errors isn’t working for people who use it anymore.
The other issue with it not being supressible is of course that this is for your whole dependency tree. You may not be the one who gets to make the code change even if you were open to the idea of churn for every little thing, even those you know are fine.
This typically (at least in my use) means opening an upstream issue and possibly a PR if it is a real issue, while opening an internal tracking ticket + suppressing the warning once there is an internal tracking ticket for it if it’s considered not a regression or noncritical.
It’s not to permanently block CI pipelines, it’s to triage and make sure at minimum triage actually happens to unblock CI.
Okay, so please read the part where any option that actually allows managing the warnings as errors without code changes when they are false positives, or plainly stating that the intent is to have no promises here are both fine.
This is a process and expectation issue. I’m fine with added work, I’m less fine with inconsistent and unpredictable extra work.
I also don’t think that getting rid of warnings altogether would be beneficial, but if nothing changes, the options I intend to present to my employer are:
Stop using python
Maintain a patchset that redirects all warnings to a structured file we can parse in CI rather than use warnings as errors.
Allocate a few hours per year on working around any new warnings beyond detecting actual issues.
None of the options I would present include disabling warnings as errors, because this has caught real issues with runtime warnings that static analysis could not have caught, as well as notified us of things like dependency structure changes in version updates in libraries that utilize warnings
As an example of the library warnings case, we got a warning issued by a library that we were using an old api without a now optional acceleration library, the fix was easy, add the optional dep (that was previously required), and the update didnt unexpectedly degrade on performance. If we had a different balance of priorities, the library use of warnings gave us the advance notice to at least make a decision, and we could have just filtered it to get the dependency updates done now, and then made sure we used the new non-accelerated API in the next version, minimizing code review needed in blocking our update process.
The thing that happened to bring this to light is that a new warning exists, it has a significantly worse signal to noise ratio (every instance of it I ran into, the use was correct, and as designed, every prior syntax warning I have ever had python give me has been a real issue that demanded a code change) and is elevated to an error by the warnings module by one setting, but is also not properly filterable by other settings.
As a matter of process, any of the following (and probably other things I haven’t considered) would prevent me needing to say python no longer fits the needs we have without patching:
Commit to all warnings being filterable.
Give an alternative to warnings as errors, such as redirecting all warnings to a structured file that can be parsed in CI/CD pipelines.
Commit to warnings that can’t be filtered by the warnings module also being exempt from conversion to errors by the warnings module.
The status quo is also fine if it’s a decision the steering council thinks this is an unreasonable use pattern, but I’d like that to be explicitly stated if the expectation is no further support of this use pattern while being willing to break users of it without warning. Documenting that at least makes it a consistent expectation that I can use to propose how to deal with this in projects currently impacted.