It’s the same reason as with numpy.ndarray: not and and are looking at the truthiness of the object, not broadcasting over the array. In this situation it might just be a type-checking error?
2 Likes
Oh, my mistake. Didn’t see the pandas call. I’m not sure if this is related to this issue then since the objects don’t have type bool?
I think it’s basically splash damage related to this issue. Peter was able to get away with letting the type checker infer the result of x == y to have type bool because they didn’t do anything that couldn’t be done with integers. Now that there’s a distinction between what’s permitted on bool and NDArray[np.bool_], specifying the type is required.
2 Likes
Exactly, but then the problem is the result type of == on arrays.
And that would have been a problem anyway since you may do something like
some_condition = array1 == array2
some_condition.ndim
some_condition.shape
# etc.
It’s actually because df.file_name registers as Any, and and equality with Any registers as bool. I can see why they made that choice, but it’s not correct here.
bool.ndim is marked as an ‘unknown attribute’ which doesn’t cause any trouble so long as as it’s contained within a reasonably short function. One has to label function return types explicitly anyways, so it doesn’t cause trouble if those return types can’t be deduced by pylance.
Turns out that switching in df['file_name'] allows the type checker to work properly.
And yes, this is splash damage from deprecating ~bool.
I don’t know whether this is within the purview of Python, but it would have been great if there was a noqa to tell my linter “Yes I really want to ~bool here.”
1 Like
Ah, makes sense. Sounds like some choices made because they’re practical.
Oh, I think you should probably do that anyway. Overloading __getattr__ as an alternative to __getitem__ is a (somewhat common) code smell in my personal opinion. No matter how convenient it seems, I think it’s bad design.
1 Like
That’s oversimplification. Before C99, there was no _Bool type and <stdbool.h> didn’t exist. In other words there was no 1 value of true and (before C99) (1 == 1) & (1 < 2) could (in theory) evaluate to 0. C89/C90 solved it by introducing logical operators, and C99 solved this by enforcing what most (or even all) compilers did. So C made arithmetic on bool-s a defined behavior after Python, not before Python.
That being said, conversion from bool to int is an “integer promotion”, while int to bool is not. So to for *modern C programmers implicit conversion to integer type might make sense.
I also want to add for possible PEP, that allowing overloading not operator for iterables (returning iterable) is also a possible solution.
1 Like
I would add that this operator is also useful and commonly used in Pandas indexing. E.g.
nan_values = df.isna()
print(df.loc[~nan_values])
1 Like
The deprecation only applies to bool values (~x, where x is a bool). df.isna() returns a DataFrame. Thus your example is not affected by the deprecation.
6 Likes
Thanks, sorry my linter got confused and thought df was a bool for some reason. Hence the warning put me in an unnecessary panic…
1 Like
Was a PEP ever written for the ~bool deprecation?
1 Like
Not sure this answers your question but…
I need functions that I can call by inputting whether python values whether arrays, for vectorization purposes. If I require an “and”, “or” or “not” operator I cannot use neither and, or, not because they fail on arrays, neither &, |, ~ because they are wizardy, I have to use np.logical_and, np.logical_or, np.logical_not, which are clumsy and make me feel I am the wizard, and if I switch from numpy to another computation framework, I need another function.
Not to my knowledge.
1 Like
There hasn’t been any PEP. Discussion was in bool(~True) == True · Issue #82012 · python/cpython · GitHub and the concrete solution was discussed in bool(~True) == True · Issue #82012 · python/cpython · GitHub and following comments.
Edit: A comprehensive summary argument can be found in ~bool deprecation - #17 by timhoffm
1 Like
Please don’t error on ~ for any int.
This would make a special case that people have to worry about whenever they use ~ on an int.
Yes, it will surprise some people as it is, but you can’t get rid of the surprise with this change. People who understand that bool is a subclass of int will be surprised by the error.
So you’re not getting rid of the surprise, you’re just moving the surprise to be less consistent.
6 Likes
I agree that neither solution is ideal, but claim that the deprecation is the lesser evil. There’s a different quality in surprise: The new behavior warns/errors with an explicit message. The previous behavior often silently does something unexpected, e.g. in if ~some_bool:.
That said, do you have real world examples, where you want the existing behavior?
1 Like
How did a behavior change like this make it to a PR without a PEP?
3 Likes
Making a PR was proposed by @guido in bool(~True) == True · Issue #82012 · python/cpython · GitHub.
1 Like
Not every change requires a PEP.
I actually had some code affected by this.
Ints aren’t only “integers” in Python, they can also be viewed as bitsets representing a subset of U = {0, 1, 2, …}. In theory, any finite or cofinite subset can be so represented. A set S is “cofinite” iff its complement, U-S, is finite. And, in Python, the complement is conveniently spelled as ~S (while U is spelled as ~0 (my preference), or as -1).
Anyway, I had some code with very delicate end conditions. Sometimes the caller wanted to deal with a 0 in the set, but other times it wanted to ignore a 0 (if present). “The obvious” way to write the code was
S &= ~ exclude0
where exclude0 was a bool argument. If False, the RHS evaluates to U, and the intersection is a no-op. If True, the RHS evaluates to the complement of the set containing only 0 (only bit 2**0 == 1), so the intersection clears out the 0 bit regardless of whether it was set.
Obvious to you? Don’t care 
. Common? Heck no. Will I live? Heck yes. Was I delighted to change the code? Heck no. Just another low-level irritation to “deal with”.
All computer languages have quirks. Python is, IMO, too mature and widely used now to risk changing much of any visible behaviors, short of screaming bugs, or (but less compellingly so) accidents of implementation that were never documented as “advertised” behavior.
There’s nothing surprising about ~bool to people who learn the language. bool is a subclass of int in Python, period. I don’t give a hoot how it works in other languages. The time for that kind of argument was when Python’s semantics were first crafted. It’s too late for that now.
If some change isn’t worth forking the language for, leave it alone .
22 Likes