Which has been fixed in py3.12 by adding is_integer to int. I would much prefer to have int gain all at least somewhat sensible methods that float has to make it a proper subtype instead of having to remember to write 123.0 instead of 123 everywhere. This is IMO one of the worst “features” in quite a few other languages.
But aren’t hex() and fromhex() still missing? It seems like an awful lot of trouble for very little benefit, i.e., not having to write float | int.
5 Likes
Yes, those two are missing. They don’t make sense to add to int() because they output a format that is tied to the internal representation of floats and does not make sense for ints.
Perhaps this is the wrong place to ask and there is discussion elsewhere that I am not aware of, but I had been hoping that this would work.
import numbers
import math
def log3(x: numbers.Real) -> numbers.Real:
result = math.log(x, 3)
return result
But both mypy and pylance tell me that float is incompatible with numbers.Real.
number_types.py:6: error: Incompatible return value type (got "float", expected "Real") [return-value]
I assume that there must be a good reason for that not working, but I was very disappointed to discover that it doesn’t. Still, it feels to me that the abstract Number types really should address the problem.
Numeric Generics - Where do we go from PEP 3141 and present day Mypy? this thread discusses numbers related classes. There have been couple other lengthy discussions elsewhere too.
At it’s core type system relies on two mechanisms,
- Nominal types: This is normal subclass relation.
class Bar(Foo), Bar is a subclass of Foo. - Structural types: This is protocols/duck like typing. Here we define interface.
The numeric types mostly do not offer a useful nominal relationship. Basic subclass relationships don’t exist across most of them and are missing a few methods. For structural side there’s little that can be usefully said with existing abcs and adding something meaningful there breaks backward compatibility. So they’re mostly unusable classes today and fit in very badly with two type system approaches. You could invent approach 3 only for sake of numbers system, but given it’s status today would still be major lift to define reasonable rules/add stubs/hints everywhere across ecosystem.
I think main way to get something like them is to use protocols with specific methods you want (multiply/add/etc), but main friction there is most typing ecosystem is not written with that in mind. That is not a change for type checkers, but change for stubs/libraries throughout to agree on protocol(s) and use it consistently.
1 Like
I figured there must be a reason for the behavior. And it is a good reason. But still disappointing.
I haven’t yet made use of protocols, but that may well be the way to go in a duck-typing world.
Today, playing around with pyright playground I noticed that pyright has more complex special casing when it comes to float vs. float | int than is proposed here i.e. the two annotations are not equivalent, see the following example:
Code sample in pyright playground
class Foo:
x: float | int = 2
y: float = 2
Foo.x.hex() # errors
Foo.y.hex() # doesn't error despite being just as unsafe
So it looks like pyright only expands the type to the union when checking for assignability, but not for anything else. In procedural code that uses literals this is less of an issue, since the type will be correctly narrowed:
Code sample in pyright playground
x: float | int = 2
y: float = 2
x.hex() # errors
y.hex() # also errors
x = 5.0
x.hex() # this is fine
So it seems like a good idea to test the proposed rule against mypy_primer output, since I fear that the more simple rule of always expanding the type may lead to new errors, that are currently hidden by pyright’s approach.
2 Likes
.hex() on int could first convert the integer to float and then call .hex() on it. That’s probably what I would expect to happen when I’m accidentally calling .hex() on an int.
The missing class method seems less like a problem (because it’s usually called with float.fromhex()), though you could do a similar thing there.
Absolutely right.
But in an ideal world, we would get a third mechanism, which supports ABC registration. Then, the numeric tower would work, and so would a variety of other ABCs. The difficulties are discussed in the linked thread for anyone interested.
Overall, I’m in favor of Jelle’s proposed change. The wording is clearer and better matches how type checkers behave today.
I do want to point out that this is going to be a little confusing/surprising for anyone who isn’t deeply familiar with the type system:
def func1(f: float):
f.hex() # attribute error on `int`? where did `int` come from?
The current wording already implies that it would be correct to report an attribute error here, but none of the type checkers I tested actually do so. So adding this case is to the conformance test suite is effectively requiring a change in behavior for type checkers that wish to stay conformant.
I think this is acceptable, given that hex and fromhex aren’t particularly widely used, and int and float are pretty compatible otherwise (and intentionally so, from what I understand, so the situation shouldn’t worsen over time). But I think it’s worth explicitly calling out that we’d be codifying this unfortunate (IMO) consequence of the int/float/complex special case.
6 Likes
I finally got back to this and submitted the proposed spec change to the Typing Council: Spec change: float/int promotion is implemented through implicit unions · Issue #46 · python/typing-council · GitHub
After reading back through the relevant threads, I remain convinced that this is the best way to make this special case a little less special and a little more in line with the rest of the type system.
2 Likes
It is a bit unclear from the text of the spec change and discussion in the PR but is the suggestion that the following would be flagged as an error by type checkers?
def f(x: float) -> str:
return x.hex()
f(1.0)
The error would be that int does not have a .hex method. Is that the expected behaviour?
I see that as a move in the wrong direction because it means that not only is it impossible to write proper annotations for float and complex but type checkers even reject correct code. I’ll admit though that this is a principled disagreement rather than a practical one given that e.g. float.hex is very rarely used.
Really I am just disappointed because I had hoped that the typing people of Python would one day come together and agree to remove this wart. In the discussion here it seems that the opposite is happening and the type checker makers are agreeing how to live with it permanently.
One of the things that I like about static type checking is using it to avoid runtime checks and redundant conversions so this float/int mixup has always seemed like a terrible idea. The fact that both the runtime and also many Python programmers allow floats and ints to be mixed up in so many places is precisely why it would be useful to be able to separate them in type annotations. Talking about methods like .is_integer and .hex misses the real issue that there are huge differences between int and float that are not expressible in the type system e.g.:
>>> ten_f = 10.0
>>> ten_i = 10
>>> ten_f ** ten_f ** ten_f
...
OverflowError: (34, 'Numerical result out of range')
>>> ten_i ** ten_i ** ten_i
^C
KeyboardInterrupt
I used Ctrl-C there because I didn’t want to wait but the output would otherwise be a 4 gigabyte sized integer and would take some time to compute. If you wait then printing the number will actually fail with:
ValueError: Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit
The difference between str(x) for x an int rather than a float was considered a serious enough security bug that it warranted breaking basic language features in patch releases. Conversely now you can’t even trust that str(x) will succeed if x is possibly an int.
There are other very basic differences like whether or not x + (y + z) and (x + y) + z compute the same value or have equivalent computational cost. Nontrivial numerical code can never really mix int and float in an uncontrolled way regardless of whether they mostly have similar methods with compatible type signatures.
The type system cannot express the fact that floats are approximate and fail with overflow error while ints are exact and have no respect for the memory limits of your system. What the type system should be able to do though is distinguish between these two basic types so that you know which of the two very different behaviours you have when seeing x ** y in your code.
8 Likes
Yes, it would mean that the .hex() call there would yield an error. You can currently get around that by first writing assert isinstance(x, float). In the future perhaps we’ll have negation types and you can write float & ~int to mean “exactly float”.
There’s been a lot of discussion about potentially removing the special case; my takeaway is that the special case helps many use cases and removing it would be too disruptive. However, the current wording in the spec is very vague, which is why I am proposing a change that makes its meaning very explicit. If you’d like to remove the special case entirely instead, you should make your own proposal.
1 Like
As has been pointed out before, negation types won’t work for this, because every single use has to prove the negation at runtime still for typecheckers to accept it.
Removing the special case is the only option here that removes redundant runtime checks.
I saw that claim in the thread you started but I don’t follow. If a library annotates a function as taking float & ~int, then float literals are assignable to that type (since type checkers should infer they are of type float, not float | int). A variable that is of type float is not assignable, which means that users calling the library with nonliterals need to also use the float & ~int type or add explicit isinstance() calls. Inconvenient to be sure, but doable.
Even if typecheckers properly handle literal float construction, Existing return types (that’s an annotation, so by this proposal reintroduces int) of float erase knowledge of it being “just a float”, meaning difference types would be just as disruptive, while being less obvious.
Here’s an example that falls apart without more special casing than what this proposal offers:
1.0 # float
1.0 * 2.5 # float | int, return annotation in float stubs
There’s also no way to correct this without internal special casing since we don’t have negation types expressible by users.
If we assume negation will happen “Eventually” and we’re intentionally kicking it down the road to then, this is just as disruptive as requiring writing int | float is today:
When we get to existing user defined functions, one of these two has to change their annoations to be accurate and usable with the other’s intent:
def x(f: float) -> str:
return x.hex()
def y(a: float, b: float) -> float:
return a + b
if x rewrites as:
def x(f: float & ~int) -> str:
return x.hex()
then anything that got their float from y doesn’t work until y rewrites as:
def y(a: float, b: float) -> float & ~int:
return a + b
But this isn’t right, and actually requires overloads
3 Likes
I’ll just cosign everything @oscarbenjamin and @mikeshardmind have said, but there are more issues with this proposal. Making float mean something different as a value from as an annotation creates problems.
What should a type checker infer here? How will this be consistent in library code across typecheckers?
def convert_as[T](value: Any, type: type[T]) -> T:
# raises custom exceptions and does parsing safe for untrusted web input
...
convert_as(1, float).hex() # error?
here you have a case where this causes conflict between a runtime type as a value and a type expression.
For fun, I’ve also expanded the required overloads in Michael’s prior post.
@overload
def y(a: float & ~int, b: float & ~int) -> float & ~ int: ...
@overload
def y(a: float & ~int, b: float) -> float & ~ int: ...
@overload
def y(a: float, b: float & ~int) -> float & ~ int: ...
@overload
def y(a: int, b: int) -> int: ...
I think it’s clearly better and more friendly to work toward seperating these rather than entangling more complexly than they already are.
Here’s what it would look like with these meaning what they actually are at runtime:
@overload
def y(a: float , b: float) -> float: ...
@overload
def y(a: int, b: float) -> float: ...
@overload
def y(a: float, b: int) -> float: ...
@overload
def y(a: int, b: int) -> int: ...
In both cases, there’s implicit extra cases when either a or b might be “either a float or int” that the overload rules capture.
1 Like
I think Jelle’s point here is important. This particular spec change proposal is not choosing between:
floatas an annotation actually meansfloat | int- Remove the special case that
intis a subtype offloat
If you want the special case removed, that has to be its own proposal.
Anything that needs specifically a float (and not an int) already can’t be annotated and that doesn’t change regardless of the outcome of this proposal. If anything this proposal makes those cases slightly easier because now the spec explicitly states what the isinstance(x, float) narrowing behavior should be
The special case is already in the spec and accepting or rejecting this proposed spec change does not change that, nor does it have any effect on the ability to remove the special case in the future
9 Likes
I’m not sure that it does make things easier. Currently mypy accepts this:
from typing import overload
class Int: ...
class Float: ...
@overload
def convert(x: int) -> Int: ...
@overload
def convert(x: float) -> Float: ...
# def convert(x: float | int) -> Float: ...
def convert(x: int | float) -> Int | Float:
if isinstance(x, int):
return Int()
elif isinstance(x, float):
return Float()
else:
raise TypeError
If you switch the commented overload line so that float becomes float | int then mypy will reject this:
$ mypy t.py
t.py:7: error: Overloaded function signatures 1 and 2 overlap with incompatible return types [overload-overlap]
Found 1 error in 1 file (checked 1 source file)
It is complaining because Float and Int are not related classes but float and int are also unrelated classes.