I would also appreciate it if you’d unpack this because it doesn’t make any sense to me. If we had a type Floating defined as int | float would that violate all concepts of consistent subtyping? It seems like the only rule that is violated is that “float in type annotations means the same thing as float in an isinstance check” which, sure, isn’t great, but there’s whole types for which you can’t even pass them into isinstance and they don’t “violate all concepts of consistent subtyping”.
2 Likes
I can imagine a few ways here, but none of them are going to be easy or happen immediately.
I’m quite averse to breaking users without warning or reason. I believe we have a reason to fix this (but that it would break users). If option 1 is viable, it’s then how do we ensure this is surfaced to users.
for what it’s worth, this should not break any runtime users:
>>> isinstance(1, float)
False
As for communicating this to users with enough time, I’m not sure what’s considered enough time for typing changes like this, I don’t think we’ve had anything with this large a potential impact since typing became as widely adopted as it is.
I can come up with a few ways this could be done, but I was more focused on convincing that it’s a problem effecting people, and that a further bandaid (like options 2-4) are likely just as disruptive as fixing it properly. I’ll admit that this approach did not strongly address all of the concerns vs the status quo as a first priority.
Some quick ideas, but I would want to think through this more:
- type checker per project setting “I am ready for numeric tower changes” that if set, type checkers should not warn for bare use of
floatorcomplex - Announcement as part of a python release
- Informational pep
- Run it in the typeshed, mypy-primer, pyright-primer and reach out to libraries.
This one’s fine, absent the numeric tower special cases.
The problem of consistent subtyping is that all interactable properties (not python @property, but tangible ways one could interact with it) of the supertype must exist compatibly in the subtype. There’s really no way to reasonably add all the methods of float to int and complex to float and int in a way that is sensible. If we could, this would be a viable approach, however, there are a lot of methods that exist on these types that just don’t apply to all of them. If this worked, waiting for difference types would be a disappointing, but at least reasonable option in that case.
if float allows int because int is a subtype of float, there’s a problem if something works on all floats but works on no ints.
This is before getting into issues of nominal subtyping, which I do think if we could reasonably have entirety of method overlap is possible to have a special case that doesn’t break total consistency.
I have a few thoughts.
The proposed spec change in Clarifying the float/int/complex special case - #19 should in fact make it easier to spell the “float but not int” type, because that spec change makes it clear that the float/int special case applies only at the syntactic level; it is not a special case in the subtyping rules. If this proposal is accepted and if difference types are added to the type system, then float - int (or float & ~int) would in fact be a viable way to spell “float but not int”.
I was curious about the impact of this idea, so the other day I made a draft change to mypy to remove the special case: Experiment: Remove int/float promotion by JelleZijlstra · Pull Request #17279 · python/mypy · GitHub. The mypy-primer tool, which runs on mypy PRs, tells us about any changes in type checker output caused by this change in various open source projects. Unsurprisingly, the impact is huge: over a thousand lines of new errors across the mypy-primer corpus. I was able to reduce those by about half by codemodding : float to : float | int across the standard library in typeshed. (That’s a less sophisticated version of the migration tools that some previous posts suggested; my codemod wouldn’t have changed a parameter annotation like x: str | float.)
Part of why we have this special case is that the Python standard library takes pains to always supports ints where floats are supported. The argument parsing APIs for accepting floats in C code all also accept ints. Therefore, it is really very rare (though not impossible) for a function to accept only float but not int. The proposed option 1 would imply that virtually every parameter annotation float in Python type would have to be changed to float | int.
I don’t have the stomach for such a disruptive change. I would prefer to change the spec now to make it clear that this special case applies only at the syntactic level, and then wait for difference types to be added so that we have a way to spell “float but not int”.
9 Likes
Could we special-case define float - int and complex - float now? Whatever “difference” end up being defined as, it would surely have to have the obvious definition for these two.
(And possibly include any other “obvious” cases while we’re at it…)
That’s option 4 in the first post, and points out that float would be incompatible with float & ~int, meaning either this is similarly disruptive or requires code flow analysis for type checkers to not turn float syntactically to float | int when it interacts with float & ~int
It may be equivalent to option 4, but I don’t read option 4 as written as implying this solution at all, so worth having it explicit I think. Let’s call it 4b.
(splitting this for the benefit of mailing list users because an edit was not quick enough after an erroneous enter)
The above also means that
requires float & ~int everywhere that currently accepts a float and intends only a float, and as you pointed out:
If even float() itself returns a float (this makes sense) then this doesn’t work.
What if this was only syntactically replaced in input parameters, and in output paramters it wasn’t?
that doesn’t work either
def div(q: float, d: float) -> float:
return q / d
would then be an error
div(4, 2)
edit: replace div with add and / with +, the example there wasn’t a good choice, but the problem persists that there are functions that might sometimes return an int if float means float | int in input parameters, and float (Exactly) in output ones, and that this makes no intuitive sense to end users.
Okay, yeah. Difference types shouldn’t be used for this. float & ~int needing to be everywhere doesn’t work if the standard library’s own functions try to allow both.
I assume type-checkers would internally have a type representing “the type returned by float()”, let’s say RuntimeFloat, which has the full set of methods of the runtime float type. float as an annotation would actually mean int | RuntimeFloat to them, while float - int[1] would mean RuntimeFloat. No code flow analysis required, everything works exactly as if you’d done a huge community pivot to redefine float.
which I’d much rather see than
float | ~intby the way, as the latter to me clearly means “anything which is either a float or not an int”, i.e. it meansobject↩︎
Ignore my footnote, I see where I was going wrong 
(kept seeing & but for some reason reading it as |)
I should’ve been more specific and phrased that: “Wait and see what the work on ADTs comes up with”.
Otherwise, fair point.
I don’t think the difference types work for this. For any API to be usable as “just a float” we’d need all of the things that only ever return a float to be modified to this difference type too. Any change which actually solves the missing use case here is going to be disruptive.
This is now two special cases, and just as much change. this one would need to exist in all return values where only ever a float is returned to be compatible with people wanting just a float. Admittedly, this could be done as a more incremental change in the ecosystem, but two special cases, and the only time you need a difference type in the mix to spell a builtin type, other than the similarly maligned Sequence[str] & ~str (for Sequence[str]), which is at least a combination of types
This is a major benefit, though.
It also keeps the value in the current system where most code annotated as float actually does allow both, and will be correctly annotated as it is the easiest thing to write. If you need the other methods, the type checker can easily tell you how to spell it when you try to access them (“you tried to do X, did you mean to use float - int?”)
At the cost of obviousness, multiple special cases, and multiple features the type system doesn’t even have yet.
The type checker could just as easily inform people to use float | int “did you mean to annotate this with float | int” when it sees you attempt to pass an int to a function in your code or “This function only accepts floats” when in code that isn’t in the code base being type checked.
1 Like
It can’t rewrite a decade of code and retroactively change old pypi libraries though
I really don’t like the idea of a type not meaning what it says it is here.
If it’s not possible to “break” types to fix an issue, then nothing should ever be accepted without a formal proof that it is a consistent feature, otherwise we’re stuck with it forever
1 Like
I count one special case (float the annotation doesn’t mean literally float the runtime type), which already exists, and one type system feature, -, which I suggested would only work for this case, so isn’t really a feature yet. What did I miss?