PEP 3141: __ratio__ instead of numerator/denominator

Ideas
1

SageMath (an open source Python-based mathematics program) can’t fully implement PEP 3141 (numbers ABC) because the numbers.Rational ABC requires a denominator property while SageMath uses a denominator() method. Note that you can’t blame SageMath for this, since it predates PEP 3141.

One way out would be to change PEP 3141 to use dunder names like __denominator__ instead (this could be a property or method, it doesn’t really matter). This would be more Pythonic since Python typically uses dunder names for special attributes. For example, next was replaced by __next__.

Of course, the change that I proposed requires some backwards compatibility measures, but that’s a solvable problem.

1 Like
2

SageMath (an open source Python-based mathematics program) can’t fully
implement PEP 3141 (numbers ABC) because the numbers.Rational ABC
requires a denominator property while SageMath uses a
denominator() method. Note that you can’t blame SageMath for this,
since it predates PEP 3141.

Does denominator need to be a method in SageMath? Why couldn’t they have
used a property?

(Python has had properties since 2.2, which came out in 2001. SageMath’s
initial release was 2005.)

Does the same problem apply to numerator?

Unless there is a solid reason why SageMath has to make the denominator
API a method call instead of a property, they should just deprecate the
current behaviour and use a property.

One way out would be to change PEP 3141 to use dunder names like
__denominator__ instead (this could be a property or method, it
doesn’t really matter).

Of course it matters. In one case the API becomes:

number.__denominator__

and in the other it becomes:

number.__denominator__()

neither of which is the slightest bit Pythonic.

This would be more Pythonic since Python
typically uses dunder names for special attributes. For example,
next was replaced by __next__.

Not quite. We replaced calling iterator.next() method with calling a
builtin function next(iterator). That follows the standard pattern in
Python where dunders are typically used for the implementation of
special features, not the public API of those features:

  • we say len(x), not x.len

  • we say x + y, not x.add(y) or y.radd(x)

  • we say next(it), not it.next

Unless you are planning to add a pair of builtins denominator() and
numerator(), I don’t think we should change the API to use dunders
instead of regular attributes.

Of course, the change that I proposed requires some backwards
compatibility measures, but that’s a solvable problem.

To me, it seems that it would seriously break backwards compatibility
for everyone who has implemented the Rational API. Why break everyone
else’s Rationals to save SageMath from having to deprecate their code?

How would you solve changing the Rational API without breaking people’s
code?

1 Like
3

We shouldn’t be looking at this as a “blame” issue at all. The PEP process is intended to collect feedback on a proposal, and I guess SageMath’s behaviour was somehow missed (either because no-one knew to ask them, or because they didn’t know to speak up). But that’s history now.

But I agree with @steven.daprano, this is now a public API in Python that’s existed for years, and we are not going to be able to easily change it. And making dunder methods part of a public API (as opposed to being the way of implementing a protocol) is not normal practice, so even if we were to change it, __numerator__ isn’t a particularly attractive choice.

What’s not obvious from this proposal is why this has suddenly become an issue for SageMath now. If they’ve been using this API for years and no-one has been affected by the discrepancy, what’s changed? Knowing that could open up alternative possibilities for a resolution (some of which might well involve changes to SageMath, of course).

4

That doesn’t really matter. Maybe SageMath could have used a property, but they didn’t. It’s just a choice, there is nothing wrong with a method.

Yes.

I don’t see how to do that technically. What kind of object would q.denominator then be?

OK, so let’s add such functions to the numbers module.

The function numbers.denominator() could try __denominator__ and denominator.

No particular reason: somebody posted it as a bug report, that’s all. Personally, I don’t think that PEP 3141 matters that much. But given that SageMath has a rational numbers class, it seems like it should support PEP 3141 but it cannot.

Sure, as long as it won’t involve massively breaking backwards compatibility.

5

There was quite a lot of discussion about this during Sage Days 100, where I originally raised this bug. Several approaches that could be used to tackle this were proposed and I have implemented one of these in branch public/28234. This involved changing ~200 lines of the Sage kernel but virtually all of this was done by a single short (Python) script. As it now passes all of SageMaths doctests, it can be put up for discussion for merging into the master branch.

This approach:

  1. Decorates the numerator() and denominator() methods of sage.Integer and sage.Rational with the @property decorator
  2. Adds a __call__() method to sage.Integer which returns themselves
  3. Replaces all uses of callable(x) with callable(x) and not isinstance(x, numbers.Integral)

This means if, for example, x = 5 then both q.numerator and q.numerator() return 5. And so it is at least technically possible to make numerator and denominator properties within Sage whilst maintaining backwards compatibility.

1 Like
6

Let me mention that SageMath is also concerned by real and imag that numbers ABC defines as attributes and that SageMath defines as methods. The four numerator/denominator/real/imag have to be thought together.

As Mark Bell mentionned, it seems technically possible to make a transition in SageMath from methods to properties for Integer.numerator and Integer.denominator. But there are much more objects than rational numbers that have a numerator and denominator methods (e.g. rational fractions that are callable, fractional ideals in a Dirichlet domain). This does make a lot of backward incompatible changes. For example the numerator of a rational function is a polynomial which is callable so that my_fraction.numerator()(x=4) will have to be changed for my_fraction.numerator(x=4).

More importantly, the rationale in SageMath is that there is no attribute in the public API. It simplifies by far the usage: you don’t have to guess whether x.foo is an attribute or a method, it is a method. Furthermore, SageMath basic number types do not have Python attributes. They are extension types built from C or C++ types. I dislike the fact that executing x.numerator (that looks like an attribute lookup from a user perspective) would actually trigger computations.

From a SageMath perspective, chaging the Python numbers ABC from “implement a numerator attribute” to “implement a __numerator__ method or attribute” would be a trivial transition. And as the OP mentioned, special attributes/methods in Python are mostly implemented as dunders.

1 Like
7

Mostly for fun (but you never know if somebody wants to take this seriously), let me mention that Python 3.8 allows abusing the CALL_METHOD optimization to make a “method” call like obj.meth() behave differently from x = obj.meth; x(). So it is technically possible to define a special descriptor denominator such that q.denominator and q.denominator() both work. But that’s very much relying on internals of the CPython interpreter.

8

Shouldn’t it be quite easy to have the property return a private method? I’m not sure if this is considered particularly elegant, but at least one would not have to change anything, because one can keep calling numerator/denominator as before.

class Integer:
    def __init__(self, value):
        self.value = value
    
    def _numerator(self):
        """Method for numerator"""
        return self.value
    
    @property
    def numerator(self):
        # property wrapper around _numerator for PEP3141
        return self._numerator  # returns callable function-object

    def _denominator(self):
        """Method for denominator"""
        return 1
    
    @property
    def denominator(self):
        # property wrapper around _denominator for PEP3141
        return self._denominator  # returns callable function-object

i = Integer(42)
print(f'Numerator: {i.numerator()}, denominator: {i.denominator()}')
9

That doesn’t solve the problem. PEP 3141 says that q.denominator should be the denominator of q.

10

But it would solve it from an ABC-interface perspective, no? If the sage devs want Integer to follow not just the letter but also the spirit of the PEP, they would have to change the API.

Or do you envision that both i.denominator() and i.denominator return 1? Re:

11

And yet you seem to be happy with suggesting that Python break backwards compatibility? It sounds like there’s no way of unifying things without one or the other project changing.

(I’m speaking here as someone who doesn’t use SageMath, and has code that uses Rational.numerator, so I clearly have a bias ;-))

12

My proposal can be implemented in an entirely backwards compatible way (I know that it would be immediately rejected otherwise). Supporting __denominator__ does not require removing support for denominator.

13

OK. I don’t understand how, from what you’ve written, but I;ll take your word for it. As long as I can continue using the property Rational.numerator to get the numerator of a fraction, that’s fine for my code.

14

[quote=“steven.daprano, post:2, topic:2037”]

Does denominator need to be a method in SageMath? Why couldn’t they have

used a property?

[/quote]

That doesn’t really matter. Maybe SageMath could have used a property,

but they didn’t. It’s just a choice, there is nothing wrong with a

method.

It matters in case SageMath need to pass arguments to the denominator()

method. That would seriously limit their ability to change to a

property.

It matters because SageMath, or rather you on behalf of SageMath, are

asking us to change an API for the entire Python ecosystem, likely

breaking masses of code, so that they can meet the Rational ABC without

changing their APIs. Why should we ask everyone else to change their

code to save SageMath a bit of effort?

[quote=“steven.daprano, post:2, topic:2037”]

they should just deprecate the current behaviour and use a property.

[/quote]

I don’t see how to do that technically. What kind of object would q.denominator then be?

I don’t understand your question. If I take it literally, it implies

that you don’t know what properties are, and surely that’s not the case.

q.denominator would be a property object, which is a kind of

descriptor, that returns whatever kind of object SageMath wants to

return. Probably an int for numbers, possibly a polynomial object if

they support polynomial division, possibly other things.

Assuming that SageMath wants to support the Rational ABC, that means

they would go through a deprecation period where q.denominator() raises

a warning. At the end of the deprecation period, change the method to a

property.

Surely I don’t need to explain to you how to change a method into a

property? I can’t help but feel we must be talking past each other.

def denominator(self):

    warnings.warn("denominator will stop being a method in version X")

    # implementation goes here

becomes

@property

def denominator(self):

    # implementation goes here

If they want to avoid a hard cut-off from one behaviour to the other,

they could return a value with a call method that returns self. That

way, q.denominator() and q.denominator will both return the same object.

15

My proposal is meant to be backwards compatible. It’s perfectly possible to support both denominator and __denominator__ at the same time.

For me, a deprecation means a period where the new way already works, while the old way still works with a warning. Since denominator can’t be a method and a property at the same time, this doesn’t work here. That’s really the essence of the problem why SageMath cannot “just” support PEP 3141.

Taking a step back, the reason why Python uses dunder names for special methods is to avoid name conflicts like this. For example, __next__ has a very specific meaning while next could mean anything (not being a reserved name). That’s the deeper reason for this proposal.

That’s probably what we’ll end up doing if this proposal is rejected.

16

I’d like to modify my proposal slightly: have just one special method __ratio__ (name subject to bikeshedding) returning a pair (numerator, denominator). This would generalize float.as_integer_ratio(), so it would be the special method for converting a number to a fraction.

17

I’m going to go back on my statement that I’d take your word that this can be done in a backward compatible way. Mainly because I don’t see how you expect __ratio__ to work any better. (Honestly, I think you should flesh out your proposal to be a lot more precise, to avoid this confusion).

If I currently have code:

def my_function(frac: Rational) -> Integral:
    return frac.numerator + frac.denominator

This works today, and can be used for any input that has a type compatible with numbers.Rational.

I don’t see how you expect things to be in the future so that SageMath fractions can be numbers.Rational and yet my code can remain unchanged. If you pass a SageMath rational to my code, it’ll break (because numerator is a method, not a property). And if you can’t pass a SageMath rational to my function, then either you’ve changed the meaning of my type annotation, or SageMath rationals aren’t Rationals

So either we have different ideas of what “backwards compatible” means, or I’ve badly misunderstood your proposal.

2 Likes
18

Yes, it will break. But that’s not a backwards compatibility issue. That’s code which doesn’t work today and which still won’t work.

Indeed my proposal is changing the meaning of numbers.Rational, consider it an amendment to PEP 3141. So your code will need to be changed to support SageMath rationals. But the point of backwards compatibility is that nothing will break if you don’t change your code.

19

The code works today. Not with SageMath rationals, certainly, but the type annotation makes that fact clear.

OK. So we’re now clear, this is a change to the currently documented definition of numbers.Rational. Fine - but I wish you’d simply stated this clearly from the start.

Huh? My code will be required to change if its annotation is to remain valid. In what possible way is that “not breaking if I don’t change my code”? Or are you suggesting that breaking annotations is not subject to backward compatibility rules?

And how do I need to change my code so it’ll still work? What do I replace frac.numerator with? Your proposal doesn’t say, it just leaves us to assume that you’re asking me to use __ratio__ (or another dunder method) in user code, which is not normal practice :frowning:

I guess I just disagree with what you’re proposing in the absence of a sufficiently precise proposal. Whether my view will change when you produce a full proposal, I doubt, but we can leave that question until the full proposal is posted.

2 Likes
20

My first post in this thread talked about changing PEP 3141, so I assumed that this was clear. But it’s good that this misunderstanding is over now.