Getting some errors reported by numba

Python Help
1

Hi forum,

How can I fix these errors reported by numba. Can numba be used on complex project?

Is True a default argument value and are these two equal:
@jit(nopython=True)
@jit

I do not like the names: nopython, @njit. The latter one looks like nojit.

Thanks

Case 1:

import random
import math
import string
import uuid
from numba import jit


class K:
    def __init__(self, a='', b=0.0):
        self.a = a
        self.b = b

    def __hash__(self):
        return hash((self.a, self.b))

    def __eq__(self, other):
        return (self.a, self.b) == (other.a, other.b)

    def __lt__(self, other):
        return (self.a, self.b) < (other.a, other.b)


@jit(nopython=True)
def f():
    d = {}
    for i in range(1_00):  # 100_000_000
        d[K(uuid.uuid4(), random.random())] = random.random()  # line 27

    d = dict(sorted(d.items(), key=lambda item: item[0]))  # out of loop


f()

$ date && python3 main.py && date
Wed Mar 23 02:23:07 CST 2022
Traceback (most recent call last):
File “/Users/ljh/Documents/helloPy/appdir/main.py”, line 33, in
f()
File “/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/numba/core/dispatcher.py”, line 468, in _compile_for_args
error_rewrite(e, ‘typing’)
File “/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/numba/core/dispatcher.py”, line 409, in error_rewrite
raise e.with_traceback(None)
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Untyped global name ‘K’: Cannot determine Numba type of <class ‘type’>

File “main.py”, line 27:
def f():

for i in range(1_000): # 100_000_000
d[K(uuid.uuid4(), random.random())] = random.random()
^
$

Case 2:

a = ''.join(random.choice(string.ascii_letters) for i in range(100))

numba.core.errors.UnsupportedError: Failed in nopython mode pipeline (step: inline calls to locally defined closures)
The use of yield in a closure is unsupported.

File “main.py”, line 41:
def f():
a = ‘’.join(random.choice(string.ascii_letters) for i in range(100))
^

2

By ljh via Discussions on Python.org at 22Mar2022 18:55:

How can I fix these errors reported by numba. Can numba be used on
complex project?

I suppose. I’ve never used it at all though.

Is True a default argument value and are these two equal:
@jit(nopython=True)
@jit

The documentation says that the default value for nopython is False:
https://numba.readthedocs.io/en/stable/reference/jit-compilation.html#jit-functions
This took less than a minute to find.

As a general design principle I like my own default values in general be
0 or False or similar “empty/off/false/null” value.

I do not like the names: nopython,

That probably stems from the design principle above. It can be hard to
find a nice synonym of the right flavour, and the direct fallback is
usually a no or no_ prefix to the mode variable.

@njit. The latter one looks like nojit.

Unfortunate. But it is unlikely to mean “no jit”, because that seems to
be the state of any undecorated function.

[…]

class K:
[…]
@jit(nopython=True)
def f():
d = {}
for i in range(1_00): # 100_000_000
d[K(uuid.uuid4(), random.random())] = random.random() # line 27
d = dict(sorted(d.items(), key=lambda item: item[0]))
[…]
numba.core.errors.TypingError: Failed in nopython mode pipeline (step:
nopython frontend)
Untyped global name ‘K’: Cannot determine Numba type of <class ‘type’>

It may be that something like this (your K class) cannot be compiled
for use by a coprocessor.

Case 2:

a = ''.join(random.choice(string.ascii_letters) for i in range(100))

numba.core.errors.UnsupportedError: Failed in nopython mode pipeline (step: inline calls to locally defined closures)
The use of yield in a closure is unsupported.

File “main.py”, line 41:
def f():
a = ‘’.join(random.choice(string.ascii_letters) for i in range(100))
^

I think we’re missing some context here. Numba’s clearly saying that it
can’t compile a particular style of code, but I can’t see the closure to
which it refers. Have you got something more complete?

Cheers,
Cameron Simpson cs@cskk.id.au

3

Thanks Cameron,

PyPy can run this code without reporting errors. If I use range(10_000), both CPython and PyPy take about 2 minutes. There is no performance improvement using PyPy on this code.

$ date && ~/Downloads/pypy3.9-v7.3.8-osx64/bin/pypy main.py && date
Wed Mar 23 15:35:55 CST 2022
Wed Mar 23 15:37:57 CST 2022
$
$ date && python3 main.py && date
Wed Mar 23 15:39:22 CST 2022
Wed Mar 23 15:41:16 CST 2022

4

By ljh via Discussions on Python.org at 23Mar2022 08:00:

PyPy can run this code without reporting errors.

PyPy’s a general purpose Python compiler. I’d expect it to accept
anything and run it.

Numba is specicifcly for graph analytics using a GPU for accelleration.
It will only be able to compile a certin subset of Python code which can
be translated for such a target. The idea is that you annotate special
purpose suitable functions with @jit and let the normal Python
interpreter to run the rest of the code.

One of the things with JITs is their just-in-time nature - sometimes
that merely means deferring the compilation until the code is actually
encountered, and sometimes they benefit from have code run several times
to see what should be compiled and/or how. So a small single pass
programme may not show improvement.

range(10_000), both CPython and PyPy take about 2 minutes. There is no
performance improvement using PyPy on this code.

Shrug. It isn’t magic. I’d expect the result to be quite dependent on
the actual code. And, again remembering the JIT remarks above, this
remark in their features page: PyPy - Features | PyPy

There are two cases that you should be aware where PyPy will not be
able to speed up your code:

  • Short-running processes: if it doesn’t run for at least a few
    seconds, then the JIT compiler won’t have enough time to warm up.
  • If all the time is spent in run-time libraries (i.e. in C
    functions), and not actually running Python code, the JIT compiler
    will not help.

Cheers,
Cameron Simpson cs@cskk.id.au

1 Like
5

Thanks Cameron!