Options for a long term fix of the special case for float/int/complex

Having just read this thread I see that there are a few mentions about the idea that mixing int and float is intentionally part of Python the language so it is good that float is interpreted as float | int. I also see a few mentions of the idea that this only creates problems because of methods like .hex or .fromhex. I just want to clarify that this significantly underestimates the issues here.

The relationship between int and float that they are often allowed to be mixed is not unique. Many libraries have analogous types that relate in a similar way for example NumPy has np.int64, np.float64 and many more. There are functions that convert int and float into these:

>>> np.array([1])[0]
np.int64(1)
>>> np.array([1.0])[0]
np.float64(1.0)

This is absolutely not something that is unique to numpy e.g. there are many array libraries and even the array API standard specifies this behaviour. There are also many other mathematical libraries like sympy, gmpy2, mpmath, and many many more. These libraries all have their own types that are analogous to int, float and complex and their own functions for converting different possible inputs into their own types.

It is currently impossible to express using type annotations the fact that a function turns an int into one type and a float into another:

from typing import overload

class Int: ...
class Float: ...

@overload
def convert(x: int) -> Int: ...
@overload
def convert(x: float) -> Float: ...

def convert(x: int | float) -> Int | Float:
    if isinstance(x, int):
        return Int()
    elif isinstance(x, float):
        return Float()
    else:
        raise TypeError

The error from pyright is:

$ pyright c.py 
c.py
  c.py:7:5 - error: Overload 1 for "convert" overlaps overload 2 and returns an incompatible type (reportOverlappingOverload)
1 error, 0 warnings, 0 informations 

There is no “overlap” in the runtime types here but pyright interprets float as meaning float | int in the overloads. If Int was a subclass of Float then it would be accepted. Having Int be a subclass of Float makes just as much sense as having int be an actual subclass of float though: it makes no sense at all because the internal representation of the data is completely different.

The inability to properly type the convert function here makes it impossible for type checkers to infer types correctly for users of mathematical libraries that allow passing int or float in place of their own native types. For precisely the same reasons that the stdlib allows math.cos(1) a library such as NumPy will allow users to use int or float in place of its own native types:

>>> import numpy as np
>>> np.square(2)
np.int64(4)
>>> np.square(2.0)
np.float64(4.0)
>>> np.square(np.int64(2))
np.int64(4)
>>> np.square(np.float64(2))
np.float64(4.0)
>>> np.square(1j)
np.complex128(-1+0j)

It is impossible though for a type checker to infer the types of any of these things if it cannot distinguish int and float properly even in overloads. These different types can be mixed just as freely as int and float can. However it is not possible for a library like NumPy to replicate the weird special casing of int, float and complex in the type system.

If there were a StrictFloat type then that could be used in the @overload annotations to avoid the overlap but then the overloads would still be incompatible with anything annotated as float:

def f(x: float):
    y = np.square(x)
    reveal_type(y) # np.int64 | np.float64

If it seems nice that you don’t have to write int | float consider that in exchange you now have confused types like np.int64 | np.float64 everywhere else even though your actual code is not confused about the types at all. NumPy cannot reach into all the type checkers and hard-code a special rule that says np.int64 | np.float64 should be treated as being equivalent to np.float64 in annotations/inference just because users like to mix these types. Allowing float to mean “maybe float” is an inconsistency in the base of the type system that breaks everything built on top.

The proper type to use most of the time for a public function parameter that can accept int or float should really be SupportsFloat. This is better than float | int because it also accepts Fraction, Decimal, np.float64 etc. The good way of allowing the different types is to do the conversion as early as possible at the public API boundary:

def cos(x: SupportsFloat) -> float:
    return _cos(float(x))

def _cos(x: float) -> float:
    ...

Internal numeric code like _cos() should be much stricter about the difference between int and float regardless of whether the public API nicely allows int. Having an int in place of a float (or a 32-bit float in place of a 64-bit float etc) is a serious bug in proper numeric code. If you do want to aloow float | int somewhere then you can write that but I would not use that for any of the parameter/return type annotations for either the public cos function or any internal function like _cos.

The fact that both the Python runtime and also many Python programmers allow ints and floats to be mixed freely is precisely why it would be useful to have static type checkers enforce the distinction strictly. A single int in a large list[float] is unacceptable and static analysis is the way to enforce this properly. Accidentally writing return 0 in place of e.g. return 0.0 (or some other kind of zero) is a common bug and is precisely the kind of thing that a type checker should be able to help with.

Currently type checkers cannot even check SupportsFloat properly because of this special case. All type checkers accept this:

import math

class Number:
    def __float__(self):
        return 1

math.cos(Number())

At runtime though:

$ python c.py 
Traceback (most recent call last):
  File "~/c.py", line 7, in <module>
    math.cos(Number())
    ~~~~~~~~^^^^^^^^^^
TypeError: Number.__float__ returned non-float (type int)

The math.cos function is implemented in C. It can allow non-floats by calling __float__ but it needs that to return the real nominal float type whose C-level struct has a real 64-bit float as payload in actual bytes in memory. There is no duck-typing or subtyping or whatever here: __float__ must return an actual float.

The SupportsFloat type should be able to capture this requirement perfectly to support ducktyping with e.g. math.cos in a well typed way. Type checkers wilfully misunderstand its return type though:

class SupportsFloat(Protocol):
    def __float__(self) -> float:
         ...

There is quite simply for float (just like every other type) a need to be able to refer to the actual nominal type.

5 Likes