Contexts for floating-point math?

Ideas
1

The floating-point arithmetic in CPython and a decent part of the math module — just a thin wrapper to capabilities, presented in the C language standard. The cmath module was also designed to be compatible with the Annex G of the C standard.

But not to all. For example, C standard provide functions (fegetround, fesetround) to control rounding modes. It also specify returned values for mathematical functions when floating-point exceptions are set.

Consider how now the error handling happens for the cmath module (taken from module comments):

Each of the c_* functions computes and returns the C99 Annex G recommended result and also sets errno as follows: errno = 0 if no floating-point exception is associated with the result; errno = EDOM if C99 Annex G recommends raising divide-by-zero or invalid for this result; and errno = ERANGE where the overflow floating-point signal should be raised.

Then the ValueError raised for EDOM and the OverflowError - for ERANGE, but the Annex G result is hidden from the pure-Python world. Though, it might be helpful for applications. E.g. clog(-0+0i) returns -∞+πi and clog(+0+0i) returns -∞+0i - correct one-sided limits in the pole of log().

The mpmath and the gmpy2 (per default, if trap_divzero and/or trap_invalid context options aren’t enabled) rather return special values per the C standard, not raise exceptions. And the mpmath also uses builtin float’s and math/cmath functions for the fp context. Thus, to override current behavior of the stdlib - we need to catch ValueError from the called function and then process function arguments to return special values, i.e. essentially re-implement handling of special values.

So, why not support custom contexts for floating-point math, like we do in the decimal module? This will offer less controls c.f. the gmpy2 contexts (e.g. we have fixed precision setting), but allow us to customize at least 1) whether functions/operations will raise an exception or return special values, 2) rounding modes.

All this is backed now by the C standard and require just an interface to access such capabilities. E.g. issue Pass return value on ValueError exceptions in the cmath/math modules · Issue #133895 · python/cpython · GitHub shows how easy to attach special values, available at C level to exceptions. All this is for free!

Here few examples of how it’s working from the gmpy2, in part that probably does make sense for the stdlib:

>>> from gmpy2 import *
>>> ctx = get_context()
>>> ctx.trap_divzero = True  # current defaults: raise an exception
>>> log(0)  # ValueError in CPython
Traceback (most recent call last):
  File "<python-input-13>", line 1, in <module>
    log(0)
    ~~~^^^
gmpy2.DivisionByZeroError: division by zero
>>> ctx.trap_divzero = False
>>> log(0)  # now get a special value instead
mpfr('-inf')
>>> ctx.round == RoundToNearest  # default rounding mode
True
>>> sin(1)
mpfr('0.8414709848078965')
>>> ctx.round = RoundUp
>>> sin(1)
mpfr('0.84147098480789662')
4 Likes
2

Thank you for opening a discussion.

You suggest to attach a special value returned in C to the exception if it is raised, instead of dropping it. In principle, I have no objections. The code is pretty small and clear, and I believe the cost is small. I just feel like the scope of this feature could go beyond modules math and cmath.

If attach a special value to ValueError and OverflowError raised in math.pow() and math.pow(), then should not it also be attached to ZeroDivisionError and OverflowError raised in the builtin pow() and the power operator? And not only for floats and complex numbers, but also for integers. And should not it be attached to ZeroDivisionError raised in 1 / 0 and 0 / 0? And to only for mathematic calculations. It may be useful to be able to differenciate overflow towards infinity from overflow towards negative infinity, division of non-zero by zero from division of zero by zero.

We need to develop a general principle for what information is attached to arithmetic exceptions.

ValueError has a very broad scope, so could we introduce an additional exceptional class specifically for mathematical errors? OverflowError and ZeroDivisionError are more specialized, so they all can get an attached special result value.

1 Like
3

No. That was suggested in the linked issue.

Yes, we might do, and — probably — should, if we adopt that solution.

But here I started with a more generic and, I think, cleanest approach.

Instead, I present the mechanism to disable exceptions and return some special values (specified by the C standard) instead. This is notion of the floating-point arithmetic context, just like in the decimal module.

In particular, this allows us to control some math-related Python exceptions. But we can do more, i.e. expose other floating-point exceptions, provided by the floating-point environment, e.g. FE_UNDERFLOW. Just like the decimal module did:

>>> import decimal
>>> ctx = decimal.getcontext()
>>> ctx.flags[decimal.Underflow]
False
>>> d = decimal.Decimal('-10000000').exp()
>>> ctx.flags[decimal.Underflow]
True

Currently CPython can’t provide information (see example in C) about underflow at all:

>>> import math
>>> math.exp(-1000.0)
0.0

The context notion also allows us to expose rounding modes, supported by the floating-point environment.

#include <math.h>
#include <stdio.h>
#include <fenv.h>
#include <float.h>

#pragma STDC FENV_ACCESS ON

void show_fe_exceptions(void)
{
    printf("current exceptions raised: ");
    if(fetestexcept(FE_DIVBYZERO))     printf(" FE_DIVBYZERO");
    if(fetestexcept(FE_INEXACT))       printf(" FE_INEXACT");
    if(fetestexcept(FE_INVALID))       printf(" FE_INVALID");
    if(fetestexcept(FE_OVERFLOW))      printf(" FE_OVERFLOW");
    if(fetestexcept(FE_UNDERFLOW))     printf(" FE_UNDERFLOW");
    if(fetestexcept(FE_ALL_EXCEPT)==0) printf(" none");
    printf("\n");
}

int main()
{
    double x = -1000.0;

    show_fe_exceptions();
    x = exp(x);
    printf("---\n");
    show_fe_exceptions();
    printf("%lf\n", x);
    return 0;
}
4

Worth looking at how numpy handles this:

Missing appears to be any support for “inexact”, or for querying/clearing which sticky flags (inexact, oflow, uflow, invalid, divby0) have been set. decimal has very complete support for all that stuff.

1 Like
5

One thing that numpy does not handle well in comparison to decimal/gmpy2/mpath is that it does not provide a way to use contexts without manipulating global variables. Specifically I mean this sort of thing:

In [10]: from decimal import Context, Decimal, getcontext

In [11]: ctx = Context()

In [12]: ctx.prec = 5

In [13]: ctx.divide(1, 3) # no global variables
Out[13]: Decimal('0.33333')

In [14]: Decimal(1) / 3
Out[14]: Decimal('0.3333333333333333333333333333')

In [15]: getcontext().prec = 10

In [16]: Decimal(1) / 3 # uses global variable
Out[16]: Decimal('0.3333333333')

Making the global context thread-aware with thread local storage or async-aware with contextvar is not enough. There should be a way to use different context parameters while still using pure functions and not mutating global state. This means that contexts should provide functions/methods like add, divide, etc rather than assuming that someone would use a / b.

If the math module gains contexts then they should provide all functions without requiring manipulation of global variables.

4 Likes
6

The decimal module is a model for a usable way to offer every bell & whistle in the relevant standards. And going beyond them, like offering context objects that can be set to custom settings, and reused as-is when and where needed. And used with Python’s with context manager blocks.

Regardless, whatever comes of this, I expect it’s Big Enough that it will require a PEP.

1 Like
7

For what it’s worth, the floating point error state is stored in context variables these days: ENH,API: Make the errstate/extobj a contextvar by seberg · Pull Request #23936 · numpy/numpy · GitHub

1 Like
8

I know and I referred to this explicitly:

I understand that for existing libraries and interfaces changing these things is difficult but if we are designing new interfaces then I want it to be clear to everyone that this is important:

At one level contextvars are better than full-blown global variables but at another level they are still just global variables and the problems with those still apply.

9

Is this currently possible with the C standard math module? I had in mind that only a per-thread (?), but otherwise global, floating point context is provided for e.g. setting rounding modes.

Or do you mean applying the context parameter locally within each function and then resetting it? What’s the performance penalty of this?

IMO it would be completely unreasonable to not use C’s math.h (and fenv.h). If something like you want requires an implementation of floating point numbers (which decimal, mpmath, gmpy2 all are) then it’s IMO a no-go.

This is an important difference between the ones you used as examples and numpy & math.

10

I was tripped up by your referring to the errstate as a global variable and I thought you were referring to the pre NumPy 2.0 behavior where it truly was a global variable, with no way to activate a local context. Sorry for replying hastily. I wouldn’t normally refer to a context variable or thread-local variable as a global variable.

11

There is seterrcall() helper to specify callback function, though it seems useful mostly for logging, IMO.

I don’t think we should be too constrained by numpy design. Being close to C offers opportunity to prototype low-level code, for example.

I’m not sure that this is required. Why context manager is not enough?

>>> from decimal import Decimal, localcontext
>>> with localcontext() as ctx:
...     ctx.prec = 5
...     print(Decimal(1) / Decimal(3))
...     
0.33333

I think we should provide just a Context type (context manager) and a function to query the current context (i.e. getcontext()), stored in contextvar.

I’m afraid, it looks so.

Let me know if some from core devs are willing to sponsor such proposal. I can formalize this in a more complete spec, by feel that this idea probably already clear enough to be discussed.

I think not directly, but with a some backtracking from our side — it’s possible. fenv.h has all API to get/set different exceptions or status flags.

Certainly not. Yet I still think that Oscar’s proposal is more generic than enough (see above).

12

Just for a background, context notion could be also used to control exceptions, coming from “invalid” FPE in a different way, producing a complex results:

>>> import gmpy2
>>> gmpy2.sqrt(-1)  # gmpy2 has different defaults: to show Annex F special values
mpfr('nan')
>>> gmpy2.get_context().trap_invalid=True
>>> gmpy2.sqrt(-1)  # here is how math.sqrt(-1) behave (exception is a subclass of ValueError)
Traceback (most recent call last):
  File "<python-input-4>", line 1, in <module>
    gmpy2.sqrt(-1)
    ~~~~~~~~~~^^^^
gmpy2.InvalidOperationError: invalid operation
>>> gmpy2.get_context().allow_complex = True
>>> gmpy2.sqrt(-1)  # but we also could show a complex result!
mpc('0.0+1.0j')

See recent discussion on PEP 791, starting from this.

Similar context notion could be also useful for integer arithmetic. See this and this.

13

Absolutely fine to discuss first. There are a lot of design possibilities to consider.

As to a sponsor, that may be a problem. Core devs are short on “math people”. I fear that attaching my name to a thing would work against it.. Maybe @rhettinger would step up?

14

I am not very convinced by these functions since they use global state. Even if python makes them available using a context manager, like in

from decimal import Decimal, localcontext
with localcontext() as ctx:
     ctx.prec = 5
     print(Decimal(1) / myfunc(Decimal(3)))

functions called in this context, like myfunc in my example, get screwed up (=modified) by the global context.

For that reason, the c++ comitee tries to get rid of fesetround:

According to that paper, also the c comitee has “no real attachment to fesetround”.

15

I see them as a way of making it so that single-threaded code that uses global variables still works with multithreading or async. If you replace a global variable with a thread-local-but-otherwise-global variable then you just have a global variable in each thread. There are reasons for wanting to avoid global variables in single-threaded code that apply equally to thread-local and context variables.

Currently the functions in the math module are pure functions. Modules like decimal/gmpy2/mpmath provide context objects so that you can use e.g. ctx.add(a, b) to avoid depending on the state of the global contexts. Above Tim said that

I agree but I also think that the decimal module (and gmpy2/mpmath/everything else) are missing something which is that they only provide the option to avoid depending on global variables through context objects but those context objects are mutable. This means that the methods of the context object are not pure functions and that it is not safe to share the context objects themselves across threads if any code might mutate them.

Currently the functions in the math module are pure functions and the proposal here is to make them impure. I agree with the idea of having contexts but if the global context is mutable then immutable contexts whose methods are pure functions would be needed (and also I think that decimal etc should provide these). Such contexts need to supply all functions like ctx.add(a, b), ctx.multiply(a, b) etc because context control for a + b can only work with global variables.

Using context managers and other ways of manipulating global state is not a substitute for having pure functions.

3 Likes
16

decimal’s context objects also support their own .add(x, y), .subtract(x, y), .multiply(x, y) etc functions. While it’s clumsy to write code this way, I have indeed done things like:

_myctx = copy a decimal context
fiddle _myctx's settings to taste

def f(x, y, *, plus=_myctx.add, times=_myctx.multiply, ...):
    return times(plus(x, y), y)

This neither affects nor is affected by what any other code may do with decimal.

The biggest insecurity is that copying “a decimal context” to begin with copies whatever changes other code may have already made. So spelling out absolutely every setting you rely on is best practice. And tedious.

1 Like
17

We’d better be sure that _myctx isn’t getting shared around though. It can happen indirectly:

class Thing:
    def __init__(self, prec):
        self.ctx = Context(precision=prec)

    def method(self):
        ctx = self.prec
        oldprec = ctx.prec
        try:
            ctx.prec = oldprec + 4
            self.other_method()
        finally:
            ctx.prec = oldprec

Now Thing(10) cannot be shared across threads because calling method in different threads clobbers the state of the context.

There are ways around this by not mutating the contexts but then it would be better if the contexts were immutable in the first place.

1 Like
18

Sure. But that’s so of any code that mutates any shared state in a thread-unsafe way.

The idiomatic way of writing that kind of code is already safe:

    def method(self):
        ctx = self.ctx
        with decimal.localcontext(ctx, prec=ctx.prec + 4):
            self.other_method()

localcontext() makes a copy of the context passed to it, optionally fiddled as specified by keyword arguments, and temporarily sets that as decimal’s ruling context. self.ctx is never mutated then.

I don’t know that making contexts immutable would be a good thing overall. They spare some possible bugs, but also make some things more tedious. So many APIs start with global state because it’s obvious and easy.

19

This is why localcontext cannot be used in library code:

Imagine libraries A and B want to use the decimal module and need to control context parameters. Then maybe also the end user is fiddling around with the global context for their own purposes while using A and B. Think about it a bit and you realise that neither of A and B can use localcontext.

1 Like
20

Of course I addressed the specific example you gave. Other examples may require different approaches.

There’s nothing to stop A, B, and the end user from forcing whatever context they like whenever they like it. At worst, as already shown, they can create their own custom contexts and use the functional API each context object supplies (ctx.multiply(x, y), etc).

But that’s vast overkill for typical uses. Mutating context is frequent in real life, and that’s an issue of “backward compatibility” now too. Is it “safe” in all conceivable cases? Of course not. It’s conceptual simplicity and convenience that drives it, and perhaps overconcern about micro-optimization (copying a context object is a lot more expensive than, e.g., copying a small int - but it’s still “cheap” by absolute measures).

1 Like