Rounding to significant figures - feature request for math library

Rosuav · January 22, 2023, 11:46am

If it isn’t, then how can we do any form of sane arithmetic on it? What is it if not some number?

(I’m not counting infinities and nans here. Assume we’re talking about a finite value here.)

NeilGirdhar · January 22, 2023, 12:31pm

I agree with you that arithmetic requires such a mapping h. So addition of elements a, b would be something like f(h(a) + h(b)).

So you’re right that h is special. I just sympathize with the asker of the question that other representatives are reasonable for display purposes.

Python4Ever · January 22, 2023, 9:43pm

This particular Python user is happy to see discussion continuing about rounding to significant figures. Other computer languages are grappling with the same issue. ref. Javascript .toPrecision toFixed, etc. with many solutions resulting with a number wrapper around some string result ref., Significant Figures in JavaScript - Stack Overflow

Python does have a complex datatype and part of the computer storage/representation solution might theoretically involve using floating point numbers automatically converted into imaginary = complex numbers, where the floating point number would be identical in format as it is now, but an additional complex part added which when applied with the floating point number sub-part would then correctly calculate to the desired significantly precise math or algebraic desired result. Again, this is attempting to overcome a theoretical construct deficit of floating point number representations, while also giving some opaqueness to additional storage and or computational requirements - needed to more closely and opaquely allow complex number and floating point arithmetic.

This is just one “out-of-the-standard-toolbox” proposed solution toward implementing rounding to n significant figures, especially in conjunction with widely multivariate exponents / powers of 10 - physicists and astronomers often encounter widely varying powers of 10 numbers. There may be other solutions, such as easier, more opaque (to Python users/programmers) methods for algebraically manipulating floating point numbers and strings. Python Math has pi, sqrt, log and exponential functions, surely a significant digit concept is also theoretically and realistically possible.

MatLab rounding to significant digits:

steven.daprano · January 23, 2023, 2:55am

Before we continue this discussion, we really should clarify what you want, because this thread has been ambiguous.

If all you want is a function analogous to round, except it works with sig figures instead of digits before/after the decimal place, that’s pretty trivial to implement since round does the heavy lifting for us:

def round_sig(x, n):
    """Round x to n significant figures."""
    return round(x, n - len(str(int(x))))

I make no claim that this is the best or fastest implementation, or that it is bug free and catches all the odd corner cases. It certainly doesn’t work correctly for NANs and INFs.

But it does demonstrate that rounding to N significant figures is not “entirely different” than the exiting round function.

If that’s all you want, then just add this to your toolbox, and we’re done

But note that, like round, this does not track significant figures in a calculation. You cannot configure how many sig figures a calculation should use, or how many sig figures a float has. All normal floats have 53 significant bits or approximately 16 significant decimal digits, and rounding doesn’t change that.

There is no way to get floats to know that they should have 8 sig figures so that they display as [3.1800000e15, 7.4700000e-12] by default (without additional processing using string formatting).

And it doesn’t resolve the perennial “binary versus decimal” issues.

If you want individual floats to know how many sig figures they should display, and to automatically keep that precision through calculations, I stand by my earlier position that it is not practical to add this to floats.

It would require completely re-engineering the float data type from using the machine 64-bit Binary64 data type and the platform math routines, to an arbitrary-precision data type similar to Decimal. In which case, why not use Decimal?

Of course it could be done, if we were motivated enough, but in practical terms I’m pretty sure the core developers will not be even a bit interested in creating and maintaining a second arbitrary precision floating point number type.

And changing the builtin float type to use this would have performance and backwards-compatibility consequences.

steven.daprano · January 23, 2023, 3:49am

I’m sure that has just clarified the entire matter for the average Python programmer. Floats are a finite ring, gotcha.

Actually, they’re not. Aside from the problems caused by NANs, two infinities and two zeroes, floats are not associative under addition or multiplication:

a + (b + c) != (a + b) + c
a * (b * c) != (a * b) * c

for all a, b, c. Nor do they obey the distributive law.

I’m not sure what sort of algebraic structure floats would be called, but lacking both associativity and distributivity, it certainly isn’t a field and therefore not a ring.

At this point I think our audience has fallen asleep, so let’s bring it back to practical matters.

I think that we agree that:

floats are not real numbers, and thinking of them as if they were reals can only take us so far before hitting trouble;
the float that is represented by the literal 32.7 does not exactly equal the decimal number 32.7;
that there are many real numbers which are mapped to, or represented by, a single float like 32.7 (to be precise, an uncountable infinity of them);
and even if we limit ourselves to rationals rather than reals, there are still an infinite number of numbers that are mapped to or represented by that single float.

Do we agree so far?

Let’s move back to the feature request:

It is easy enough to round a float to a certain number of significant figures. That’s just a minor adjustment to the round() function.
But it would be a big job to re-engineer the float type to carry information about significant figures, at least as much as the decimal module, and the core developers are unlike to want to do this.

notatallshaw · January 23, 2023, 4:32am

Steven D'Aprano:

If all you want is a function analogous to round, except it works with sig figures instead of digits before/after the decimal place, that’s pretty trivial to implement since round does the heavy lifting for us:
def round_sig(x, n):
    """Round x to n significant figures."""
    return round(x, n - len(str(int(x))))

Am I right in assuming this only works with numbers >= 1? I immediately got some very confusing results:

> round_sig(0.555, 2)
0.6
> round_sig(-1.35, 2)
-1.0

Rosuav · January 23, 2023, 4:36am

Toss an abs() in there and it should be fine. Though the number of odd edge cases here MIGHT be an argument in favour of sticking this into the stdlib somewhere, since it’s easy to get it slightly wrong.

notatallshaw · January 23, 2023, 5:03am

Indeed, e.g. abs fixed cases for numbers <=-1, but still doesn’t work for numbers x where abs(x)<=1.

def round_sig(x, n):
    """Round x to n significant figures."""
    return round(x, n - len(str(int(abs(x)))))

> round_sig(0.0555, 2)
0.1

The best I could come up with in a few mins is:

def round_sig(x, n):
    """Round x to n significant figures."""
    if abs(x) > 1:
        return round(x, n - len(str(int(abs(x)))))
    t = str(abs(x)).replace('.', '')
    return round(x, n + len(t) - len(t.lstrip('0')) - 1)

But I’m not sure I’d trust committing that to any actual code base.

steven.daprano · January 23, 2023, 6:30am

Remember the bit when I wrote this?

“I make no claim that this is the best or fastest implementation, or that it is bug free and catches all the odd corner cases. It certainly doesn’t work correctly for NANs and INFs.”

You can add numbers less than 1 to the list

Dealing with these corner cases is not hard, you just have to account for them carefully. How about this?


import math



def round_sig(x, n):

    """Round x to n significant figures."""

    if x < 0:

        return -round_sig(-x, n)

    if not math.isfinite(x):

        return x

    assert x >= 0

    offset = len(str(int(x))) - (x < 1)

    return round(x, n - offset)

Any other bugs or performance fixes?

pf_moore · January 23, 2023, 10:00am

Or you could leverage the decimal module. The following is only lightly tested, and I’m not an expert in the decimal module, so could probably be improved:

from decimal import Context

def round_sig(x, n):
    return float(Context(prec=n).create_decimal_from_float(x))

Caveat: I’m not 100% sure that the “precision” used in the decimal module is exactly the same as what the OP means by “significant figures”. It seems like it’s the same concept to me, but so much of this discussion has hinged on fine distinctions that I feel it’s necessary to mention this point…

storchaka · January 23, 2023, 11:23am

It does not work for 0.000123.

Code that works for any float is:

float(f'{x:.{n-1}e}')

“Significant figures” are mostly meaningful in the context of string representation of a number. Therefore it is related to formatting. And perhaps you do not even need to convert the result of formatting to float.

NeilGirdhar · January 24, 2023, 11:34am

My mistake, you’re right.

(I was just trying to get at the closure under various operations.)

Do we agree so far?

Yes.

PythonCHB · January 25, 2023, 6:03am

Code that works for any float is:
float(f'{x:.{n-1}e}')
“Significant figures” are mostly meaningful in the context of string representation of a number. Therefore it is related to formatting.

This is the argument that’s been made every time this comes up:

“significant figures are about formatting, so the formatting tools are all you need”

And that is strictly true, I suppose. But I know I’ve had use for for “rounding to significant figures” in values before formatting (for passing to the json module, for instance)

And I don’t see it as fundamentally different than the built in round() either:

In [20]: round(1.145678, 1)
Out[20]: 1.1

In [21]: f"{round(1.145678, 1):.20f}"
Out[21]: '1.10000000000000008882'

you still aren’t actually getting what you ask for.

And it’s also clear from this discussion that it’s not obvious how to write that function, and easy to get slightly wrong – so a really good candidate for a builtin (or library, but really, do we need a one line function on PyPi?)

As to the implimentation, most folks seem to use log10 to do the job, or the built in formatter, as you showed (though I always used the g formatter:

In [29]: def sigfigs(x, n):
    ...:     return float(f'{x:.{n}g}')
    ...: 
    ...: 

In [30]: sigfigs(1.2345e5, 3)
Out[30]: 123000.0

In [31]: sigfigs(1.2345e-5, 3)
Out[31]: 1.23e-05

But it does seem inefficient (maybe in a way that doesn’t matter) to convert to/from a string. While the log10 method (I think) has issues with edge cases at the precision limits, the string formatting is implemented somewhere, so could presumably be copied to the math module directly in C.

But the first step is to decide to put it into the stdlib – and then I’m sure someone smart will figure out how to implement it.

And I’d love to see this.

Rosuav · January 25, 2023, 8:13am

“Significant figures” is either about display, or about actually keeping track of them all through a calculation. Which we don’t, and won’t, have facilities for doing.

You’re right. It isn’t different from round(). For what it’s worth, I practically never use round(), only string formatting.

ttamg · January 25, 2023, 8:50am

I’ve been watching this thread. I’m the original requestor for considering adding this to the math standard library.

I concluded some time ago that there is not appetite or clarity to add this into the core math library and hence bowed out. I think that’s fine to not include significant figure rounding in the core library on this basis. It can still be done as a separate function and there are some pypi libraries to do it.

This is not just a formatting topic as per my original use case. In real world business use cases where some inputs and calcs have a lot of uncertainty there are many cases where I might want to round to (usually) 1, 2 or 3 significant figures and then carry that forward in other analysis. It’s a business short-hand way of recognising that there is uncertainty in the estimates without doing what would be theoretically correct (and frankly beyond the stats of most business analysts) which is to estimate the uncertainty and carry that forward too. So my real world work as an actuary, typically people do the significant figure rounding and then use that number going forward. If you carry the full precision number forward then people can get confused as the numbers don’t add up, or sometimes a junior analyst takes that precision to mean it is a real accurate number which in business can be an issue. These are just practical things that happen rather than them being theoretically the right thing to do.

Saying all of that I think this can already be served through pypi packages and the short sig figs rounding scripts.

So unless anyone disagrees I suggest this thread and request is closed.

Thanks,
Matt

ttamg · January 25, 2023, 9:23am

I hope you mean that in jest Chris. It’s not entirely clear. And even if you do, I can’t see how a toxic comment dissing any broad-brush group should have any place in a diverse open source community, like the python community.

Rosuav · January 25, 2023, 9:59am

Mostly jest. I mean, they must do at least SOME things right, or all that money would have been stolen already. But when you look at the rules that banks invent for our passwords, the way that all their policies protect the bank first and the customer a distant second, and so on, I think you’ll agree that we can’t warp our policies around what banks consider best practice.

So I’m going to stick to the notion that “round to significant figures” isn’t a float-to-float conversion, it’s a stringification.

John_Carter · January 25, 2023, 11:51am

import math

def sigfigs(x, n):
    return float(f'{x:.{n}g}') # f and g formsating give different results

def sig(x, p):
    if not x:
        return x
    n = math.floor(math.log10(math.fabs(x))) +1.0 - p
    n10 = math.pow(10.0, n)
    sx = n10 * round(x / n10)
    #~ print(x, p, n10, n, sx)
    return sx

s = 2
p = 10
for i in range(0, 200, 5):
    x = i/111.
    y = sigfigs(x, s)
    z = sig(x, s)
    print(F'{x:.{p}f} {y:.{p}f} {z:.{p}f}')

There is a longish discussion about this on Wikipedia, mostly about uncertainty but there is an equation for this, see code above.

One definition seems to be:

ignore leading and trailing zeros and the decimal point
then the n significant figures are the first n digits, padded out by trailing zeros
and a rounding rule for the n + 1 digit.

This works well however sometimes floating pont precision gets in the way and sometimes the number required has no floating point representation. Run the code with p = 20 to see this.

eg.x = 0.09009009009009008584 z = 0.08999999999999999667

If I cared about the accuracy of the calculation I would either use a fixed precision (integer *) or analyse the error on the result from uncertainty on the input.

PL/1 and COBOL have these builtin

steven.daprano · January 25, 2023, 12:04pm

Thanks! Rounding is harder than it looks, and rounding to significant figures is even trickier.

“Mostly”, but not entirely.

The rules I learned in physics class were:

convert your input numbers to the same number of significant figures;
use as many significant figures as you can for physical constants;
carry out your calculations to the full precision available;
and round to the lowest number of significant figures in your input data.

The first step is definitely not just about string representation; the last step may or may not be. (It depends on whether you go on to use that number in further calculations.)

steven.daprano · January 25, 2023, 12:14pm

Funny, the longer this discussion goes on, the more I am coming around to the position that perhaps it should be in the std lib.

There are many good reasons for rounding numbers before doing arithmetic on them (so not just for display), in physics, chemistry and economics. Ultimately though, any rounding to binary floats is going to surprise people who think that they represent exact decimal numbers.

There is no need to close this thread (if it is even possible). So long as people still have something to say, let them say it.

Topic		Replies	Views
Support rounding of datetime / a nicer way of rounding Ideas	16	7451	June 19, 2023
Please help me find this: Volume of Sphere in Python Python Help	9	4490	August 12, 2023
Include `math.sign` Ideas	80	3141	February 29, 2024
Mean method in math module Python Help	4	583	September 7, 2023
Datatypes: get back 80-bit long double, 80-bit float support in general, Python Help	14	1427	September 7, 2023

Rounding to significant figures - feature request for math library

Related Topics