It’s sensible to catch an AssertionError to, for example, bring down the system orderly, leaving it in a consistent state. If the program is properly modularized, then the error could shut down only the affected module and alert operators.
At least since the yearly release cycle, code that uses the new features in a release will not work on interpreters for previous releases.
Having a way to tell Python that code is being run in production
mode rather than development is a useful feature in Python.
This includes being able to add debugging code which is skipped
in production and doesn’t incur any runtime overhead, so the
-O option serves that nicely.
So -1 on dropping -O and -OO.
It would be good to better educate users on the effects of
using the assert statement, though, since I see a lot of code
written even by senior developers, which obviously assumes that
assert cond, errormessage works as short for (approximately):
if cond:
raise AssertionError(errormessage)
when, in fact, it maps to:
if __debug__ and cond:
raise AssertionError(errormessage)
pytest has certainly added to this by using assert as main
vehicle to write tests. The fact that pytest actually does clever
AST manipulation to extract information and change the meaning
of assert seems little known.
The tendency to shorten code has (for whatever reason) started
creeping into Python coding and it’s showing similar effects
to what you often find that other scripting language 
You can see this in code abbreviating package names, complex list
comprehensions, .this().then().that() style chains, etc. etc.
A little more typing can often go a long way in producing more
reliable and readable code.
2 Likes
This is something that’s been on my mind for a long while, but I’ve never gotten around to measuring click/decorator overhead on start up times. At my $work, we’re heavy users of click (wrapped in our own CLI convenience library) and start up times is one of the biggest complaints from users of our Python CLIs. Now, some of that is our fault (e.g. dependencies doing way to much work, or network access on import, etc.) and our Python stack “suffers” from a ton of extra functionality that other language stacks don’t have, so it’s not an entirely fair comparison. But for example, the most commonly used CLI (literally, every dev will use it multiple times a day), is written in Python and they’ve done a lot of custom hacks to improve the fast path start up time. I’ve suspected that heavy use of @decorator syntax, which all get executed at import time, is a contributor to the start up problems, but I don’t have the data to back that up.
Oh and I should mention, we don’t run our production CLIs with -O, although we do generally run them in isolation mode.
What I was trying to say was, I never expect asserts to ever trigger. An assert (to me) is an indication in the code of “here’s a situation that should never happen unless I, the developer, am wrong about some assumption”. Of course, I’m never wrong <wink> so therefore an assert should never trigger.
3 Likes
So, let’s try a first pass a summarizing ideas here:
Some people insiste that asserts should not be used for control flow (except AssertionError), but as a last resort action to spot that something went wrong in the code (and not in the user-controlled data) and they put an assert to have the code stop as soon as possible, to avoid the code breaking later and having to reverse engineer the exact spot where something bad happened. Though people do not agree whether we want that to happen in production too.
Other people occasionally use it as a control flow feature, e.g. in a try/except where we want errors or checks evaluating to False being treated in the same except block.
I think we can all say that in theory, the code we put in production is always flawless and on practice, our Sentry instances tell a different story. Which leads me to believe that mistakes WILL happen in production and I always want the mistakes to guide me to the bug as clearly as possible, independently of whether I want, say, the docstrings to be ignored.
The fact that it’s often not the same people who write the code and who run it (especially in the case of dependencies) makes it hard to push the idea that "it’s alright because you can always choose not to use -O", and this alone steers people to use neither -O nor assert.
It’s mentioned that in some cases, we really want the ability to have asserts not run.
Also, on the things that were not mentioned, but that I wanted to add:
- I don’t know about you, but I’m not a fan of code behaving differently in production and in test, especially not obviously. That’s a good way to create hard-to-reproduce issues. If I’m reading a Sentry report, and following the code on my repo, and I see an assert, I will assume that the assertion is run. Before I realize that I might be following along the code wrong because the asserts are ignored, I might have lost quite some time.
I have a few suggestions, it doesn’t have to be a package, and we can cherry-pick, but I think it could be interesting and I’d like your opinion.
-
-Ostops ignoring asserts, though it continues to define__debug__toFalse. - People who want to write assert that don’t get executed (which I believe is not the majority ? but that’s probably my confirmation bias speaking) can put them behind
if __debug__, which makes it more explicit. - not parsing/storing docstrings is exposed through a dedicated flag, independant of optimization
- (trying to think if exposing the “ignored asserts” through a dedicated flag would make sense or not. The idea would be that it comes with a warning in the doc that not all code has been written with the idea of the asserts being optional, so it’s 100% on the person who runs it to decide if they want this or not, not on the person who code, who can then use assert however they want.)
1 Like
Assertions that cannot be disabled are not assertions. Assertions should always pass. If you expect them to fail (apart from a general pessimism that any code can be buggy) then they are not assertions.
The hint is in the name:
assertion (noun) a confident and forceful statement of fact or belief.
When we read an assertion, we should be reading it as a checked comment. It is a comment about the code (namely that the condition holds) which the interpreter checks for us.
If you have a test that you expect to sometimes fail, and so you don’t want it optimized out, the right way to do it is with a test and a raise. You can even write it on a single line:
if condition: raise WhateverException('message')
As for your question about whether you are expected to run Python with the -O flag, that is 100% a business decision for your management to answer.
That answer will probably depend on how you ask the question:
“We can run under -O mode that will reveal lots of bugs in our error handling and give an insignificant speed-up. Should we do it?”
“We can run under -O mode that will disable extra program validation checks which always pass, giving a small but meaningful speed-up. Should we do it?”
7 Likes
I am sympathetic to your concern about code behaving differently in production and in development, but really, that’s not your decision to make.
Unless, as the vendor, you take that decision out of the hands of the customer. E.g. you just tell them “don’t use -O” and then enforce it with a simple check at application startup:
if not __debug__:
raise SystemExit('running under -O mode not supported')
But that’s an admission of failure in my eyes.
By the way, does Sentry report whether Python was running under -O mode? If not, that seems like that would be a good enhancement.
Christian suggested:
“Part of the problem with assert is the fact that a failed assert does not abort the process.”
How would you feel about a Perlish construct
assert or die condition
which differs from assert in that it aborts with Py_FatalError? “die” could be a soft keyword, so it is completely backwards compatible.
… deprecate
-O
I’ll take that!
In fact in every company I worked at, I fought hard (and usually won) to ensure that -O is never used, not in production or otherwise. Because it always ends up surprising developers!
assert statement is a great feature, akin to first class def syntax. I use it and intend to continue using it. When I assert, I expect the assertion to be evaluated.
P.S.
Although the below may raise some hairs, I accept (and sometimes write) code like this:
def load_jwt(token) -> dict:
try:
header, body = token.split(“.”)
assert json.loads(…)[“alg”] == “es256”
…
except (AssertionError, ValueError, TypeError, …):
raise CustomException(…)
1 Like
I’d like to add some per-file option to prevent optimization.
The -OO option has same problem.
It can strip significant RAM usage when using heavily documented library (e.g. SQLAlchemy).
But if you use at least one tool using docstring (e.g. docopt), you can not use -OO option.
4 Likes
@steven.daprano, that at least some of us think differently is proof enough that you’re taking a fundamentalist view on the role of assertions.
The epistemological problem we’re trying to solve is that:
every statement and expression in a program is a "a confident and forceful statement of fact or belief"
We don’t “try” to make programs correct. We are certain (enough) that programs are correct when we submit them to production. Yet, our bug registries prove that we’re often wrong! Asserting the expected state catches so many oversights that it makes it worthwhile (Why the oversights? Because our brains don’t work as computers).
There’s also that we often call code we did not write, and that code may fail in subtle ways.
There’s also that the computers in which we run our programs may fail one bit at a time. Using assertions help us detect such failures and enter recovery mode.
Why use assert instead of if not condition: ...? Because the syntax is convenient, and and we want to assert the expected state, which makes using if not awkward.
After reading all the comments, my opinion is that we will keep discussing these themes for a long while before we reach some consensus about what’s good to do in general.
For now, I propose a minimal compromise:
- add a
-Aoption to thepythoninterpreter that makes assertions be always be on - add a
PYTHONASSERTIONSenvironment variable for the same purpose
It’s not too much work, and it’s a way to let everyone have it the way they think is best for them.
My own preference would be for assert to allow introducing a block that raises AssertionError if no other assertion was raised from the block, but that’s a bigger change probably worth more discussion.
1 Like
To not go into fundamentalisms myself, I just thought that those of us keen to use assertions that are always enabled could get away with something like:
with unless(condition) as u:
# prepare
u(f'message {arg}')
# if no `raise`, `AssertionError` will be raised
No changes required to the language or the interpreter.
I may publish the above to PyPi this weekend “unless” someone else beats me to it 
EDIT: This doesn’t work because
withis committed to always executing the block, while the requestedunlesswould execute the block only ifnot condition.
There’s no consensus, but that doesn’t mean the status quo needs to be the best option.
That being said, I’m not sure it’s worth writing the PEP before we have a core dev sponsor, so if any core dev wants to volunteer, let me know.
(Otherwise, well thanks everyone for your diverse opinions, and I’ve learned a lot in this thread !)
One option I have not seen that would be better than the status quo:
- turn
assertoff by default - make
-Obe a noop - add a command line switch, say
--debugor--assertthat enablesassert
Presto chango, problem solved! People stop using assert inappropriately.
10 Likes
I too will collaborate with a PEP if there’s sponsorship.
Or, better, just add the behavior to -X dev (at least in addition to its own flag or -X option, if not instead of it), so there’s not one more flag that users need to know and remember to pass to get the strict, informative behavior they expect when running Python for development purposes (locally, running tests, in CIs, etc). This is very much in line with what the current -X dev changes are intended to do:
The Python Development Mode introduces additional runtime checks that are too expensive to be enabled by default
The problem is if people don’t use it and don’t realize the change has been made, their asserts silently stop getting checked in development, CIs, etc., which is much more serious than them not running in production/when deliberately passing the -O flag.
2 Likes
Oh, I like that!
2 Likes
It sounds like what some people want is a handy way to check arbitrary conditions in a single line, without the specific semantics of the assert statement.
This reminds me of Scala’s require() built-in function, which would look something like this in Python:
def require(condition: bool, message: str = None):
if not condition:
# If not ValueError, then perhaps a new built-in RequirementError.
raise ValueError(message)
Some example invocations:
require(x is not None)
require(x % 2 == 0, "x must be even")
require(balances[user] > 0, f"{user} has no money.")
A new built-in function would avoid the difficulty of changing existing behavior, or adding new keywords.
I don’t feel strongly about this issue, but I mention this because it seems like the Scala folks have made their own distinction between assert and require, which may apply here as well. To summarize the points made in that thread, in Scala:
-
assertis for program invariants. It is elidable with a compile-time parameter. -
requireis for constraining parameters. It is not elidable.
1 Like
Aside from the shortness, for which any of us can roll a utility
function, the other aspect of assert (which is possible probably only
because it is a statement) is that the message side is only evaluated if
the assertion fails.
This makes me feel free to put computationally burdensome things there
to get verbose messages (or alternatively, concise summary info of a
complex object), or at the least not to care what I put there in terms
of runtime cost.
A function call evaluates its arguments before the call, which would
necessititate evaluating the message part.
Cheers,
Cameron Simpson cs@cskk.id.au
1 Like