Numeric Generics - Where do we go from PEP 3141 and present day Mypy?

Responding in part to @mdickinson via python/mypy#3186:

I’d love to see proposals for a reworked numeric tower based on typing use-cases; I don’t have a huge amount in the way of time and energy to offer, but would be willing to sponsor a PEP, or at the very least to review a PEP authored / sponsored by others.

I would love to get to this point, too. If I can carve out time, and if others are willing to provide patience and guidance (with tolerance of my being slow, but I’ll definitely try to get there eventually), I am willing try my hand at this. (Others should not be dissuaded from doing the same. I understand that pesky day jobs have a tendency to get in the way.)

Trying to mutate or adapt the existing numbers module to fit the needs of the typing community seems like a much harder proposition, not least because published interfaces are horribly rigid (add anything new and you break implementers of that interface; remove anything and you break the users). I don’t see a strong need to deprecate PEP 3141; I’d just accept that it’s not all that useful for typing purposes, ignore it and move on.

I think it’s at least important to signal to newcomers via the most popular surface (i.e., the standard library documentation) that PEP 3141 is not a typing system. That’s currently unclear, and those who don’t monitor mailing lists, GH issues, etc. can easily mistake it for one and then spin their wheels for a long time trying to get it to work as one. But, as you point out, we don’t have to reach that decision until we’re confident in a viable alternative, so first things first, which I think is just a less clear way of restating your proposed next steps. :ok_hand::grin:

I think there’s a set of really hard problems to do with mixed-type arithmetic that it may not be reasonable to expect any kind of typing framework to solve. Suppose I have two different Real-like classes A and B; there are all sorts of ways that mixed-mode arithmetic between A and B, A and float, B and int, etc. might work or not work; it may simply not be reasonable to try to capture all of the possible permutations in type declarations. (For an example of how not to do it, take a look at the GAP framework. But this at least demonstrates that there’s a hard problem to solve.)

I don’t think ubiquitous inter-implementation operator interoperability should be a requirement. For example, I don’t think the standard library should impose support for all math operations between numpy.int64 and sage.rings.real_mpfr.RealNumber as a condition of participation of either. One shouldn’t prevent authors of one or both of those from explicitly supporting each other, but requiring it of them seems unreasonable.

I do think it’s worthwhile to target use cases where one could say, “Give me your interoperable implementations of Float, Rational, and Integer, and I’ll perform this algorithm using those primitives.” That might converge to the same problem, but at this point, I’m hopeful it’s simpler. Maybe requiring that implementations are interoperable with standard library primitives is good enough? Dunno.

FWIW, I’ve done a fair amount of numeric coding both at home (mostly number-theoretic or concerned with esoteric floating-point stuff) and at work (mostly data-science’y and based on the standard NumPy / SciPy / tensorflow / … scientific stack), and in the time since the numbers module was brought into existence I have yet to find it useful. What has been useful are typing protocols like SupportsIndex and SupportsFloat. (SupportsInt, not so much.) The kind of use-cases I personally have, and would love to see addressed in any new proposal, are things like “usable as a float”, “usable as an int”, “can be converted to a Fraction”.

I probably don’t have nearly the exposure you do, but my experience echoes yours. Maybe Supports… is the way one gets to standard library interoperability?

2 Likes