Mismatch between assert's semantics and how it's used (-O, -OO, disable)

Chris, I appreciated several of your contributions in this thread.

I’m glad that you’re a strong voice for backward compatibility (this is something we need to take very seriously) and your passion for today’s Python shines through (I love this language too). Some of your messages are really useful, e.g. I thought your audit of assert statements in Lib/threading.py was a fantastic contribution to the discussion!

With all that said, I found your last couple messages quite hostile and your rhetoric needlessly inflammatory. Consider a little less flame?

James, consider sleeping a little longer between posts as way to avoid escalation.

13 Likes

Fair. I’m definitely stepping away from this thread. If it has any chance of being productive, it’s without me.

So, with how strongly people feel against this change, I can’t help but think I’m simply not understanding something. And that doesn’t sit right with me, because I’d hate to be misinformed or have incorrect assumptions (the irony of which in this thread is not lost on me).

So, can someone help me see what I’m missing?

Here’s what I have for the technical:

  • asserts most certainly are performing exactly as documented and expected from a technical standpoint. They aren’t broken or incorrect
  • asserts may be stripped given certain incantations of executing Python
  • the usefulness of stripping out asserts is a bit of a performance bump, as the statement is not executed
  • this is perceived as large enough a problem to some to recommend not using asserts as a blanket security measure. Some

And then the more subjective/cultural:

  • Although it is documented that asserts may be stripped, not everyone who writes Python knows this. And given the popularity of Python (especially among new programmers) and the size of the community, this percentage is likely significant (maybe not large, but significant)
  • Largely, Python isn’t run under these incantations that strip asserts. That unfortunately strengthens the incorrect assumptions that some make.
  • because of this some security-minded people/tools recommend not using asserts as a way of guarding against this as a potential security concern

And then to the proposal to deprecate the current flags and introduce a new one:

  • Most asserts won’t change. Since most executions of Python dont strip them, it’s reasonable to assume most assert authors are OK with them being executed (even in prod situations)
  • the asserts that do need to change will ideally be obvious. These are perfomance sensitive code, and once the flag is deprecated, the tester/runner will know something has changed. They hopefully are also performing performance testing.
  • Python, a language usually touted for being beginner friendly, is now MORE beginner friendly
  • additionally, some places that incorrectly used asserts for verification are no longer a security concern. Python got a little bit more secure

Lastly, somewhat editorial observation:

  • People who cared enough about performance to strip asserts only get bitten by the deprecation when they upgrade. BUT thanks to faster CPython your code got faster just by upgrading :stuck_out_tongue:

That’s my observations. And as they stand, from my viewpoint the pros outweigh the cons which is why I’m in favor. But, again I get the sense either my perception is wildly off of the pai upn of the cons, or in just missing additional cons.

(And please, I’m being genuine in wanting to understand how other people perceive this. I’d love if others also treated this as an educational/collaborative discussion)
.

4 Likes

I’ll try. But honestly, I don’t have much more to say that hasn’t already been said, so I have little interest in further extending this discussion.

There’s a whole bunch of questions I have about your claims and assertions (even some of the “factual” points you make). But I’m going to skip over those in order to be very clear on the major, glaring problem with your proposal.

This is where your proposal is completely missing the point. You are not proposing to change asserts, you’re proposing to change -O. And in a way that will affect every current user of -O. You can’t assume that every user of -O is OK with it being deprecated. Nor can you assume that there are “too few users of -O to matter”.

Everything you say after this point (and most of what you said before) is irrelevant, because it’s not addressing the real concern with this proposal, which is that it deprecates a current behaviour of Python, without looking at the impact on users of that behaviour. Instead, you look at users of assert, who by your own claim are mostly not users of -O.

The only reason I’m still responding is because people make this mistake all the time, of making a proposal but not actually looking at who it will affect and whether the impact on them is justified or acceptable. So I appreciate that you’re trying to understand the problem here. Hopefully this helps.

PS The other proposals in this thread have different problems, and that has muddied the waters a lot. But the problem with your proposal is relatively simple. I think you’re mostly just confusing objections to other proposals with objections to yours - which is easy to do given the mess this thread has become :slightly_frowning_face:

7 Likes

Oh that helps immensely. I appreciate you taking the time and effort to chime in again with very useful perspective. I think that’s great feedback for myself, but also for the OP. Truly, thanks.

@SonOfLilit I think honestly, you have your next steps before this can reasonably continue as a PEP or as a general discussion. You need to reasonably show the impact of the change. What percent of people use -O? What’s their pain? What’s their migration like? What percent of people benefit from the proposal? Why isn’t education enough?

And most importantly, the answers to these questions should clearly show that the change is truly a net win for Python (otherwise it wouldn’t be a proposal of an enhancement :wink:)

2 Likes

… Hold on, I’m confused now. Did @thejcannon make a concrete proposal that’s different from the one in the OP?

Not all, just trying to help OP in channeling some of the energy/brains here. (As well as learn myself)

FYI the admins have gotten a lot of flagged posts on this topic. I have turned on slow mode in hopes it cleans up, else I will lock it.

And FYI, you need to directly mention the admins or flag posts for us to see things as we do not read every single post.

5 Likes

I agree about evidence on the effect of deprecating -O and am trying to think of how to gather it.

If anyone here knows anyone who uses -O or an industry where it is common, I would love to hear from you please.

I use it.

It means I am much quicker to write asserts: I don’t hesitate to consider if it might negatively affect performance, because I know it won’t. Without -O, many of the asserts that I write would not be written. They would not be replaced with other checks, because they’re for something that I “know” won’t happen.

Removing -O would compel me to check all the asserts that I’ve written for the last 20 years, almost 2000 of them, to see if any needs to go. (The easy fix is to unconditionally remove every single assert, but I probably wouldn’t do that.) I’m not complaining about the work, most asserts are very simple and I could probably do that pretty quickly, but it would feel strange investing time in an effort that makes the code worse.

2 Likes

I think in the OP, your remedy if you wanted to keep your asserts would be to unconditionally replace your assertion statement to use __debug__ or ...

So assert thing becomes assert __debug__ or thing. They would get executed, but the penalty would be a globals lookup and a conditional.


Have you measured the timing difference in your programs with and without -O?
Any numbers you can share?

We use -O2 in production at Instagram. The primary motivation is memory savings from stripping docstrings (this we absolutely must have), but it’s currently impossible to get that without also stripping asserts.

Personally I think it would be nice if one could opt in to docstring stripping and/or assert stripping orthogonally from each other, rather than in an arbitrary layering. (In what way is stripping docstrings “more optimized” than stripping asserts?) This wouldn’t require deprecating -O or -O2, they could continue to have their current meaning, it would just mean adding new options. It would require changing the pyc file tagging strategy a bit, but that’s not a big deal.

I don’t actually know how much efficiency gain we get from assert stripping; I suspect not much. I know at some point in the distant past there was an outage because an engineer used an assert wrongly, and assert stripping in production made a problem worse than it would have otherwise been. Because of that, there’s been some cultural holdover ever since (even reified in the form of lint rules) to avoid asserts entirely :confused: So that’s a point of evidence that developers not understanding assert stripping can cause problems.

I still don’t think assert stripping as an option should ever go away, though. It is a useful tool to be able to write (possibly costly) invariant checks for development and testing, and skip checking them in production where performance matters.

9 Likes

There is no globals lookup – __debug__ is replaced by True or False as appropriate at compile-time:

$ python -m dis <<<"x = __debug__"
  0           0 RESUME                   0

  1           2 LOAD_CONST               0 (True)
              4 STORE_NAME               0 (x)
              6 LOAD_CONST               1 (None)
              8 RETURN_VALUE
$ python -O -m dis <<<"x = __debug__"
  0           0 RESUME                   0

  1           2 LOAD_CONST               0 (False)
              4 STORE_NAME               0 (x)
              6 LOAD_CONST               1 (None)
              8 RETURN_VALUE
1 Like

That’s even better. Hell, some smarty pants could come along and omit the bytecode for the assert if we undeniably know it’ll evaluate to true (__debug__ or ...).

(That probably explains why assigning to __debug__ is illegal)

That’s what happens! If __debug__ is false, the bytecode for the assertion gets omitted! It’s a brilliant optimization!

Did you just reinvent the original semantics?

I was kinda expecting this response. :slightly_smiling_face:

Somewhat, but the key differentiator is that the assertion is optimized out locally based on the assert, and not globally for all my code and all code that my code depends on, and so on…

So having a way to write some assertions so that they don’t follow the global default behaviour would be sufficient? How about using

if not CONDITION: raise AssertionError(MSG)

for the ones you want to prevent being optimized out?

:slightly_smiling_face:

8 Likes

(I get the feeling I’m being made the butt of a joke here, but I am truly, earnestly just trying to suggest that we can discuss whether a change could be made to help with beginners to the language)

I think you may have forgotten the context here, which is that people (mostly beginners) dont always think about the fact that the global default behavior is toggleable. So, again, the suggestion is making the default beginner-friendly and allowing those who have gone beyond “beginner” to choose how they want to proceed.

So acknowledge the retort, but I don’t think it acknowledges the truth behind the proposal. If Instagram avoids asserts because a developer fell into “the trap”, and they aren’t alone, I think this is worth discussing.

I understand you disagree, but hopefully we can keep the discussion going weighing the proposal, and background, seriously and with info shared in-context

3 Likes

I can’t tell what’s going on or what sides there even are in this topic anymore. Many people are pretty clearly against it, or for it, or swapping between the two, which suggests there’s no consensus to change the current behavior. People keep proposing other ideas and hypotheticals, making it hard to follow what the current state of the idea is. I think the course of action at this point is to create a PEP so that a specific specification and reference implementation can be discussed and have a decision made about, rather than going all over the place.

2 Likes

Not the butt of a joke, precisely, but the point is that you’re arguing about asserts (again) when what you are actually proposing is to deprecate -O. It’s very hard to understand your point when you keep changing the focus, so we’re reduced to slightly confused rebuttals of the immediate point.

If you want to deprecate -O in favour of having explicit optimisation options, then:

  1. Say so, explicitly, and stick to that proposal.
  2. Address the question of how this impacts people currently using -O, and in particular, using it correctly. Don’t dismiss their use cases as not relevant, or assume they are happy to change their (currently working!) code. Acknowledge the costs. Quantify the benefits, and compare the two.
  3. Don’t keep explaining how people can “fix” their asserts. We know, it’s just that having a way to do so doesn’t mean the cost vanishes. Quite the opposite - it means you’re saying that code does need to change, but you’re dismissing the cost.

If you want to pursue this, as @davidism has said, you’re probably at the point where you need to write a PEP, expressing your arguments in a standalone document, taking note of everything that’s been discussed here so far and addressing it. But for a PEP, you need a core developer to sponsor the PEP, so you should start by trying to find a core dev who supports the idea (or at least, supports taking it through the PEP process). But while I haven’t checked every post, I don’t recall seeing any core dev speak in support of this proposal, so you may have a problem there.

Otherwise, drop the matter. Accept that you’ve made your points, they were listened to, but you didn’t convince the right people. Move onto a different discussion that interests you. If it still matters to you, come back to this idea later, when there’s new information to offer, or circumstances have changed. But accept that for now, the debate has run its course.

4 Likes