without special cased support beyond just the annotation, there’s no way to ever have a valid instance of a float - int to assign in the first place, everything is returning float right now.
See this.
This is the status quo. Obviously you can’t add information to old libraries that wasn’t there. But I don’t see how you can argue it’s better for them to be completely unsafe to use, with no way to even identify which libraries might be in this state. Like you said, you cannot assume something that says it returns a float doesn’t actually return an int. You would need to V2 the wheel/sdist or something and point blank refuse after some cutoff date to even install old libraries.
The status quo is that these might be unsafe to use. if people write float and think nothing more of it because they haven’t personally been bitten by it, that doesn’t mean the function behaves properly at runtime with an int
Removing the special case would actually allow type checkers to catch that. The status quo, including saying “just use difference types”, does not
I feel like I’m in a bizarre reality where up is down here trying to actually point this out. complex and float at runtime is less inclusive of potential assignments than it is at type time currently, and that’s the problem.
I feel like a lot of this has focused on float, but this applies to complex as well, and with complex, no such effort exists to just have complex “just work” in places in the standard library. cmath is a whole separate library from math for a reason.
Literals would all be the new type, automatically, since as I said, the type-checker knows “this is what float() returns”. I guess that’s a special case but it’s the same special case I already said.
Any code path that did if not isinstance(int) would also have the right type.
I concede you’d need to reannotate the whole of cpython. That’s true for both plans. Indeed, for any conceivable plan.
So would adding in float - int, because now you can enable a “strict mode” where you have to use the correct annotation to access things that only work on RuntimeFloat. Over time, as things get annotated correctly, that mode can be made the default.
Python the language has been designed carefully to allow mixing floats and ints at runtime (some corner cases notwithstanding).
The issue appears to be about interfacing to other languages, where Python’s type system is too abstract. Maybe we should define new types that fit those languages better, e.g. int32 and int64, and float32/float64. IIUC numpy does something like this, and we could probably just use its names and conversion/coercion rules.
8 Likes
Can I suggest we table complex and focus on float for now? The solution we pick doesn’t actually need to be the same for both, after all, and they have different API surfaces and usage to think about.
I thought about this, but I think it has the same problems as option 4 in the original post. Without the type system keeping track of float vs int and flow analysis, this still means everywhere that currently returns a float needs changing to then return the specific ffi type, otherwise places that return float can’t be used with ffi intending only a float. This just permeates everywhere.
I don’t think there’s anything wrong with just spelling it how it is:
The standard library accepts floats and ints? float | int
There’s also still just standard library differences at play here, while most of these are pretty uncommon for users to be calling, these really aren’t compatible types even within python:
def boom(f: float):
return f.hex()
boom(1)
and as was pointed out in the related thread this spawned out of:
(with those two referring to .hex and .fromhex)
You mean, apart from the massive problem of backwards incompatibility already discussed?
Yes, I compete agree. Anything annotated incorrectly (i.e. too widely) would need to be updated to be annotated correctly (i.e. narrowly). I still see this as VASTLY better than silently updating them all without anyone checking if the sudden new meaning is correct.
No, I mean including taking that into account, but with the acknowledgment that this isn’t a trivial change that can be made overnight, already addressed and would need further discussion on a transition plan.
already addressed:
float is too wide at type time right now (it allows things silently that can error), narrowing it to match reality can only catch bugs at runtime or introduce places where people should write float | int as the intent. This will have no impact on runtime.
You mean fixing all the code that behaves in a way that only makes sense to people who are so deep in python typing they are involved in discussions on python.org for it? doesn’t sound like a downside to me.
1 Like
Already addressed here:
May I suggest the problem is you have already decided on a solution? You seem to be missing the coherent picture of the alternative solution because you keep thinking of it in terms of how individual pieces don’t work in isolation in the context of your ideal world. Missing the woods for the trees, so to speak. (Not to mention this being quite a rude thing to say, then repeat again by quoting. I get it, you think I’m being stupid.)
1 Like
I don’t think layers of special casing that negate each other and a need to run a special strict mode to get even that to catch issues is a good direction.
The less the type system just follows the rules and composes, the harder it is for people to reason about.
The special case here is causing real problems, and the solutions people have offered sound like mental gymnastics to avoid acknowledging a deep-seated problem is going to suck to solve no matter what.
No, I don’t. You’ve made your points well enough and reasoned through them.
My point with that was not to rudely prod at people, but to point at what seems to be clinging to “we must not break people” when people are already broken by the status quo in what feels like absurdity to me.
We’re wrapped around the axle on this (and have been for over 7 years, given all the times this has been brought up) to keep a special case that is incorrect, is currently preventing a simple and accurate expression of intent from being possible, and the reason why is because it might break people.
The special case is itself incorrect to rely upon. It’s not safe to do so. Any change is going to be disruptive, any lack of change will continue to have holes in it, both for what users would want to express, and what is currently checkable. The approaches which would be “less disruptive” still require rewriting most current uses of float to float - int (or something else that would behave like it)
So why layer on more special cases that don’t fully solve it instead of work on a way we could communicate this change to users with appropriate forewarning? Why would we cling to something which has broken behavior instead of fix it?
3 Likes
On further thought, it doesn’t even need to be all that. A type typing.Float, meaning float only, would have served me well enough.
4 Likes
You don’t provide any evidence for “most” here.
That’s the easy-to-explain-and-use migration route that your proposal is missing, the thing you keep avoiding: how to actually do the massive breaking change to get the type annotations correct.
Provide an option! Right now you’ve only offered a complete break of the Python ecosystem. Suggest something practical!
Both options end up with annotations correct, unambiguous, and well-defined, and the same set of errors statically catchable. If nobody can suggest a practical option for implementing one of those two approaches, I don’t think “I don’t like it” will win the day.
1 Like
I’d go for it. Users of static type checkers are very used to sometimes having to change a lot of things when changing the version of the tool.
It would certainly be better to be explicit rather than to rely on a thing that is, i daresay, completely unknown to most users.
8 Likes
complex has no methods/attributes that float/int don’t have (well, except for an implementation of __complex__), so from a runtime perspective, passing float/int where complex is annotated is generally save.
float on the newest python only has the methods hex/fromhex (the latter of which is a class method, so IMO a bit of a different story) extra compared to int. And those two are very rarely used functions specific to the internal representation of float. So saying that from a runtime perspective there is “no way to reasonably add all the methods” is just wrong.
In fact, you could add methods hex and fromhex to int, and they could have sensible semantics (although wouldn’t be compatible with float.hex/float.fromhex, which is an understandable deal breaker)
1 Like
As a newcomer to Python I once got confused by float behavior when reading type-hinted code, and I was wondering if int had been really a subtype of float (no).
I’m using pyright and it helpfully treats float as float | int in annotations only and not elsewhere. The learning curve around the numeric tower is quite steep for me though.
float as float | int feels like a hack: it’s wrong, but generally useful, and usually does not result in trouble for me once I have got used to it. Perhaps the feature really should have been implemented as typing.flint or something similar in the first place back then, but (1) use cases that accept both float and int are probably more common than those that do not, and this is likely by design, and (2) the ship has probably sailed and it would be a breaking change to make float only accept float now.
A special-case typing.Float (or FloatOnly or similar) will be a more practical solution for the use cases (mainly when interfacing with external data) where only floats and not ints are accepted, and this special case can itself be removed/reimplemented in ADT when that arrives.
6 Likes
I don’t think accusing people of not arguing in good faith is productive, and it probably shouldn’t come from someone that said breakage was ignored, had to delete their own post, followup correcting it, and then later still said that breakage wasn’t addressed.
Maybe calling it a pedantic argument went too far, but I’m having a hard time reading some of this and not seeing how people would get frustrated having to re-link the same things already said, only for a distinction between “a lot” and “most” to be what someone is nitpicking.
Can we refocus this on what options might actually be viable and what those options would require to actually happen?
Option 1, change it to be correct. This may or may not require notifying users about types breaking (no runtime breakage of working code). Strategies for doing so in a cordinated manner with prior announcement mentioned above, including settings, notification in release notes, and setting a cordinated date for the earliest for switching the behavior.
Options 2, 3, and 4 share a flaw, but something like option 4 received the most support from users who have tossed their hat in.
A new type for this would still require a lot of old code to be updated. This would only be surfaced in cases people manually caught, and would likely require most places in the standard library that return a float at least, to type this as the new type.
There’s also “do nothing till difference types”. I don’t like the idea of hedging this on something that might never come to fruition, but it also would require a lot of old code be updated to return a difference at minimum.
Finally, there’s “do nothing at all”. This leaves the problem unsolved and require no further action, but leaves certain things untypable.