Runtime warnings in Python

Python Help
1

Hi, I’m new to Python and I have some questions about how to handle runtime warnings/errors.

I have two examples originating in the same function.

RuntimeWarning: overflow encountered in power
  return d+(a-d)/(1+(x/c)**b)**g

RuntimeWarning: invalid value encountered in power
  return d+(a-d)/(1+(x/c)**b)**g

Its not really a problem as the program executes as intended. However, I would like to dump some variables if these warnings are thrown just to make sure nothing is messed up.

  1. If warning is thrown
  2. print(a, b, c, d, g)
  3. Keep executing
3

Update

I rearranged the expression like this: (a - d) * ((c/x)**(-b) + 1)**(-g) + d and managed to avoid the overflow warning.

4

The Python construct for exactly what you were looking for is try...except.

The set of exceptions built-in is here: Built-in Exceptions — Python 3.12.3 documentation . For this case, you probably want to catch ArithmeticError.

Rearranging the expression to avoid the problem is better, if you can figure it out.

Edit: Now I look at it, I’m puzzled by RuntimeWarning. This rather suggests something is already catching and replacing the original overflow arithmetic exception or value error. However, with your own try...except you can get there first.

5

You can use the warnings module to turn the warnings into exceptions that can be caught or to collect the warnings in a list.

For example:

from warnings import catch_warnings, warn

def power():
  warn("Overflow!", RuntimeWarning)

with catch_warnings(record=True) as w:
  power()
  if w:
      print("Uh oh, we got some warnings.")
      print(w)

Note that, as the documentation points out, catch_warnings modifies global state, so it’s not safe to use if your program runs multiple threads.

1 Like
6

I tried the try except thing you suggested but it seem I can’t manipulate the message, nor print variables if there’s a warning thrown. I tried:

    try:
        res = (a - d) * ((c/x)**(-b) + 1)**(-g) + d
    except:
        print(a,b,c,d,g)
    return res

Is the except statement never reached if there’s a warning thrown, i.e. not an error?

Edit, I missed your post Henrik Tunedal, will try that!

7

To be clear, these are the original warnings. Unlike Python’s buitlins, numpy only emits warnings (not excpetions) for dubious operations:

>>> import numpy as np
>>> np.array([-1]) ** 0.1
<stdin>:1: RuntimeWarning: invalid value encountered in power
array([nan])
>>> np.array([2]) ** 1e300
<stdin>:1: RuntimeWarning: overflow encountered in power
array([inf])
2 Likes
8

I don’t understand why you get a RuntimeWarning at all. When I abuse ** I get an overflow, and a proper stack dump:

>>> 20.0 ** 1000
Traceback (most recent call last):
  File "<pyshell#96>", line 1, in <module>
    20.0 ** 1000
OverflowError: (34, 'Result too large')

Are the variables all floats and integers or something exotic (like in SymPy)?

Are you running this code directly in Python or inside some kind of framework that is protecting you from the details?

9

Regarding the RuntimeWarning. Can it be that I call the function (asymmetricLogistic) in curve_fit operation?

return = curve_fit(asymmetricLogistic, x, y, predictions, full_output = False)

10

I missed @bschubert’s reply this while I was expressing my own puzzlement.

So they’re (probably) ndarrays. Right. I’m a relative newcomer to NumPy. Once more I find myself thinking “I can see why you may have thought doing that was a good idea, but I wish you hadn’t.”

All the binary operators then are on the elements of two arrays taken pair-wise, and what @DevastatingCharisma needs is a way of discovering at what index i in the arrays the dubious operation occurs, then to see the elements a[i], b[i], ... at that index.

And NumPy isn’t letting on.

Also, warnings are a “tell me once” kind of thing aren’t they, so there could be lots of indices.

11

Yeah kind of…

I run several, thousands, data sets with Cartesian coordinates to which I fit curves using the scipy.optimize library. I get a handful warnings thrown in my face each time I run the code.

The result of okay, but then again, the devil is in the details right? I would just like to print the variables causing the warning to occur…

12

You can make NumPy raise a proper error instead of a warning. I am fairly sure this will abandon the calculation (data set) in which it occurs.

But it would give you something to catch.

Then you could do the calculation again, in a for loop this time, and detect the rows on which that specific calculation fails. This will likely be slower than using NumPy. As I understand things, only a few of the data sets give the error, so maybe not a big slow-down in total.

You might do that just to debug the process, or you might do it so you can process the data set with “cleaning”.

If only the error contained an index. I’m sure NumPy knows :frowning:

13

You can use np.isinf on your data to tell you which values have overflowed (i.e. been set to inf)

>>> np.isinf(np.array([10., 1.])**309
array([ True, False])

If you’d like an index, use nonzero on the result.

If it makes sense to recover from the overflow, you could use np.nan_to_num(posinf=something) to replace positive infinity with a value that you can use.

1 Like
14

Hi,

when you get an error, you can copy that error and create your own exception as shown here to catch that specific error if you’d like (borrowed error result from @jeff5 ):

image

In the above example, I used user interaction. When testing, I noticed that if you use a float input, you will more than likely encounter an error. For both numbers, I converted the inputs to integers to avoid this possibility.

Maybe someone more experienced can chime in as to why a float will raise an exception vs integers.

15

I had an idea.

I use the non linear regression to calculate a value for each data set. When the estimation fails i dump an integer where the calculated value would go if the regression is succesful. If the warnings thrown are some how linked to the regression failing i should be able find the regression coefficients by backtracking data set generating this integer.