Following a Twitter discussion, I was interested to get some feedback around the idea of making a PEP to have assert statements always execute, independently of Python’s optimize mode.
I will be taking some time in the near future to lay out what I think can be the pros and cons, but if some folks already have a very strong opinion about that, it would either give me better ideas for a rationale or convince me that it’s not worth spending the time.
Thanks!
I’m in favour of this. I’ve never seen any legitimate reasons to drop assert statements, but I’ve seen toil imposed by the existence of the behaviour.
For example, Django recently changed some uses of assert to raise ..., purely due to the risk of users running with -O: #32508 (Raise proper exceptions instead of using "assert".) – Django .
Also coincidentally today someone asked on the Django forum if it’s okay to use pytest due to the risk of their asserts not running: pytest, assert keyword and optimized mode - Using Django - Django Forum .
I’d agree, except that suppressing asserts is the only thing that -O does. No other optimizations depend on this flag. (-OO also drops doc strings)
-O also sets __debug__ to False. if __debug__: blocks are then removed by the peephole optimizer.
I’ve brought this topic up before, but couldn’t gather support for taking action.
The general agreement was that changing the behavior of assert is looking for trouble, and that people can get invariant asserts with:
if not condition: raise SomeException()
I’m in favor of introducing a new keyword for assertions or invariants that never go away, like invariant, unless, or ifnot.
Invariants (preconditions, postconditions) are important in maintaining the health of complex code. Bertrand Meyer explained it all many years ago, and all we need is a pythonic way of asserting invariants.
…afterthought… @guido, We could get away for a while with an interpreter option different from -OO that introduces all optimizations available, but leaves assert alive?
I don’t have a preference for the naming or exact syntax, but I very much liked the semantics of our last discussion on this topic.
unless condition:
# a full block like in `with` statements
stats = stats_about_this_error(avalue, andanother)
raise AssertionError(stats)
So your proposal is for assertions to stop being assertions?
Having assertions disappear when running under -O is fundamental to
what makes them assertions.
Assertions are not just a lazy coder’s way to check a condition and
raise an exception in one line. They are statements about code
correctness.
I think can I write out your pros and cons in two sentences:
Pros: lazy coders can feel good about writing lazy, misleading code that
raises an inappropriate exception when it fails.
Cons: assertions stop being assertions.
I have to say, that Twitter thread is painful to read.
them. Grepping the stdlib would find 200+ examples, and a quick
spot-check suggests that they are being used correctly.
No, assert is not fragile if you use it for what it is intended for.
No, the caller of the code should (almost) never catch AssertionError.
(The rare exception is for code like error handlers that catches all
exceptions, logs them, and exits.)
suites are “plain wrong”.
assertions about your code’s correctness, e.g. for checking data
validity, you are doing it wrong. It is never appropriate to make
an assertion about external data, e.g. user-supplied data, arguments
passed to a public function by the caller, etc.
to the caller, then AssertionError is almost certainly the wrong
exception to raise.
You wouldn’t write these validation tests for a public function:
if not isinstance(arg, int):
raise AssertionError('arg must be an int')
if arg <= 0:
raise AssertionError('arg must be greater than zero')
because they are the wrong exception class. So why would you use assert?
And you wouldn’t expect the caller to write this:
try:
value = data_table[index]
except AssertionError:
value = default
again, because it is the wrong exception for the error.
If you do expect the user to catch AssertionError, you should have a
long, hard look at your API.
Django devs: if you “worry” that users will run your code under -O and
disable your error checking code, then your code is broken.
Do C programmers abuse the assert macro like Python programmers abuse
assert? Are there masses of C programmers complaining that people can
turn off assertions from their code, or using it to perform data
validation?
https://ptolemy.berkeley.edu/~johnr/tutorials/assertions.html
I’d rather deprecate -O and -OO than change the language.
Invariants are supposed to go away under optimization. If you are
testing an invariant, then it can never fail, and it is safe to remove
it. The reason we test invariants is that we are fallible and what we
think may be a program invariant may not actually be invariant, and so
testing it can reveal bugs. Hence we can choose to remove the
assertions, or choose to keep them, depending on how confident we are
about the code correctness.
"Bertrand Meyer explained it all many years ago, and all we need is a
pythonic way of asserting invariants."
That would be assert.
https://www.eiffel.org/doc/eiffel/ET-Design_by_Contract%28tm%29%2C_Assertions_and_Exceptions
Preconditions, postconditions and invariants are all specialised forms
of assertions, depending on where and when they are tested. They are
designed to be safe to remove in production.
I am not interested in further debate here. Let me know when the SC has approved the PEP.
I don’t know, @steven.daprano!
What I’m thinking of (and I think @guido too) is the work done by Bertrand Meyer and others on preconditions, invariants, and postconditions as important part of software correctness (of course, they are all assertions about the current state, even if with other names).
Please excuse a digression…
Once, very young, I was at a conference, and the speaker was talking about a specification language that somehow attached to actual programming languages would make programs correct. I asked, “Why not make the specification language executable?”, and thus got shut out of the presentation.
So, onto basics, I don’t think we want to have an argument about the ways to so software correctness (and I have my objections over unit-test coverage, for example).
Some of us, Python programmers (perhaps including @guido), would like to have preconditions, postconditions, invariants, assertions that that are always checked.
@steven.daprano Your replies by email are littered with extra lines and line breaks. That makes them very hard to read. I stopped reading after the first paragraph. Could you please fix your posts?
PS: I just noticed that you are replying by email. Could you look into the configuration of your mail client?
A new keyword is very hard to add. There is a reason why we build new constructs using existing keywords or soft keywords. It takes at least two Python releases (>2 years) to introduce a new keyword. New keywords break existing software, too.
How often do you think __debug__ blocks are used?
Why does Django care whether users run with -O or not? Is there some fundamental functionality or assumption that breaks when users do that? It sounds like if that’s the case, then changing them to raise is the right thing to do regardless of whether assert gets no-op’d or not.
I never write asserts expecting them to ever be triggered. Well, except for pytests that is.
(Aside: this is the one aspect of pytest that bugged me for like forever, and stopped me from adopting it. But then I stopped trying to paddle upstream and now I’m fine with the pytest use of assert and certainly agree it makes writing tests a lot more readable than self.assertBlah() calls.)
I’m in the camp of people that think asserts don’t make much sense in the current form. If they were changed to always run, they’d make more sense to me.
As a person in charge of operating a large-scale Python deployment, am I expected to run with -O in production or not? If the answer is ‘yes’, I’ve been doing it wrong for a large number of years and we absolutely need to communicate this better, and it complicates the deployment story further. If the answer is ‘no’, asserts should always work.
And I think the simpler case here would be ‘no’.
Click, a popular CLI library I maintain, uses the __debug__ variable and assert. There are a lot of complex interactions when configuring the cli objects, so we want to check that the developer didn’t make certain mistakes. However, startup time for a cli is very important, so those checks should not run in production. assert accomplished this without requiring any Click-specific runtime configuration. Whether anyone actually distributes thier CLI tool so that it runs with -O, I have no idea, but it did seem like a useful feature that shouldn’t be changed.
Basically, for anyone using assert for its current behavior, removing that behavior seems undesirable. What would the alternative be if this is changed?
I’m… less so.
I have a few functions whose state is complex or fragile (prone to
breakage from small changes). Regrettably, one in particular is also
performance critical, at least in that it’s the busy part of a common
flow. As such, it is littered with asserts. I would like to run it with
the tests disabled at times. (I also use the icontract decorators, which
also disable themselves with -O.)
The other aspect of asserts I like is that they are a very concise and
only evaluate the message side if the assertion fails. That feature
means I can put quite expensive (but informative) messages when
appropriate.
The:
if __debug__ and assertion_test: raise exception
feels quite cumbersome by comparison, and visually noisy particularly if
multiline. Sanity checks should be encouraged, and therefore should be
easy to write. And they should also be considered " almost free", in
that a mode exists to turn them off; this also removes mental obstacles
to writing assertions.
Since -O only disables asserts (and fiddles __debug__) surely people
in the “no” camp can simply never use -O to acheive their preferred
mode?
Cheers,
Cameron Simpson cs@cskk.id.au
Right, I fully understand asserts are convenient to write and useful while testing.
Let me put it this way: should your code (with a lot of asserts) raise AssertionErrors in my production environment?
Right now that question is up to me, but I’m not really qualified to answer it since you wrote the code and we as the Python community haven’t really settled on a shared understanding of the answer.
So for me there’s a mental mismatch here that I don’t know how to reconcile.
(Note I use ‘you’ and ‘me’ very loosely here to illustrate a point, as an open source author myself I’m on both sides of the equation anyway.)
Right, I fully understand asserts are convenient to write and useful
while testing.Let me put it this way: should your code (with a lot of asserts) raise AssertionErrors in my production environment?
My present stance is that AssertionErrors should not happen in
production. Certainly I’m against anything which catches them as
contingency handling 
Right now that question is up to me, but I’m not really qualified to answer it since you wrote the code and we as the Python community haven’t really settled on a shared understanding of the answer.
So for me there’s a mental mismatch here that I don’t know how to reconcile.
For me as well. I struggle frequently with the tension between a concise
assert and a sanity check which raises, say, ValueError. Often the
situation is clear - an assert indicates something the algorithm should
never do, and a ValueError (etc) indicates an environmental circumstance
which should not have occurred. But not always.
(Note I use ‘you’ and ‘me’ very loosely here to illustrate a point, as
an open source author myself I’m on both sides of the equation anyway.)
Aye, me too.
Cheers,
Cameron Simpson cs@cskk.id.au
The errors are errors regardless of environment. Presumably you’ve addressed them before going to production, in which case you can optimize them out.
if __debug__ blocks are rarely used. I have seen it in very few libraries, mostly older code. Over time developers realised that libraries were abusing assert for control flows, which made python -O unreliable.
Part of the problem with assert is the fact that a failed assert does not abort the process. It’s too easy and too common to catch and ignore the result of a failed assert. Python’s assert should really work more like C assert. In C a failed assert(3) results in an abnormal termination of the program. In Python assert False raises an AssertionError, which is a subclass of Exception. We should have changed the base class to BaseException or treat a failed assert as unrecoverable error (e.g. abort with Py_FatalError).
Nathaniel explained to me that pytest’s use of assert is not affected by the optimized flag. He wrote:
pytest transforms the source AST for test files before compilation, to replace
asserts with annotated always-present assertions. that’s also how if you doassert a == bit can tell you whataandbwere if it fails