I gotta say, numba seems to be usable only in extremely simplistic use cases carefully designed to be presented in talks
I can run the following code just fine:
def rt(hi):
for i in hi:
hi_ = i == hi
t = hi[hi_]
return None
rt(np.array(['a','b','c','d'],dtype='U'))
But, when i decorate the above code with njit:
#njit
def rt(hi):
for i in hi:
hi_ = i == hi
t = hi[hi_]
return None
rt(np.array(['a','b','c','d'],dtype='U'))
I get the following error:
---------------------------------------------------------------------------
TypingError Traceback (most recent call last)
<ipython-input-34-eadef1d0ecee> in <module>
5 t = hi[hi_]
6 return None
----> 7 rt(np.array(['a','b','c','d'],dtype='U'))
~/miniconda/envs/IndusInd_credit_cards_collections_scorecard_/lib/python3.8/site-packages/numba/core/dispatcher.py in _compile_for_args(self, *args, **kws)
418 e.patch_message(msg)
419
--> 420 error_rewrite(e, 'typing')
421 except errors.UnsupportedError as e:
422 # Something unsupported is present in the user code, add help info
~/miniconda/envs/IndusInd_credit_cards_collections_scorecard_/lib/python3.8/site-packages/numba/core/dispatcher.py in error_rewrite(e, issue_type)
359 raise e
360 else:
--> 361 raise e.with_traceback(None)
362
363 argtypes = []
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<built-in function getitem>) found for signature:
>>> getitem(array([unichr x 1], 1d, C), Literal[bool](False))
There are 22 candidate implementations:
- Of which 20 did not match due to:
Overload of function 'getitem': File: <numerous>: Line N/A.
With argument(s): '(array([unichr x 1], 1d, C), bool)':
No match.
- Of which 1 did not match due to:
Overload in function 'GetItemBuffer.generic': File: numba/core/typing/arraydecl.py: Line 162.
With argument(s): '(array([unichr x 1], 1d, C), bool)':
Rejected as the implementation raised a specific error:
TypeError: unsupported array index type bool in [bool]
raised from /home/sarthak/miniconda/envs/IndusInd_credit_cards_collections_scorecard_/lib/python3.8/site-packages/numba/core/typing/arraydecl.py:68
- Of which 1 did not match due to:
Overload in function 'GetItemBuffer.generic': File: numba/core/typing/arraydecl.py: Line 162.
With argument(s): '(array([unichr x 1], 1d, C), Literal[bool](False))':
Rejected as the implementation raised a specific error:
TypeError: unsupported array index type Literal[bool](False) in [Literal[bool](False)]
raised from /home/sarthak/miniconda/envs/IndusInd_credit_cards_collections_scorecard_/lib/python3.8/site-packages/numba/core/typing/arraydecl.py:68
During: typing of intrinsic-call at <ipython-input-34-eadef1d0ecee> (5)
File "<ipython-input-34-eadef1d0ecee>", line 5:
def rt(hi):
<source elided>
hi_ = i == hi
t = hi[hi_]
^
How to subset a 1-d array using a boolean 1-d array in numba decorated function?
Related
I am having trouble compiling a simple function in no-Python mode with Numba:
#njit
def fun(x,y):
points = np.hstack((x,y))
return points
a = 5
b = 2
res = fun(a,b)
While this very simple script works without the #njit decorator, if run with it throws the error:
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<function hstack at 0x7f491475fe50>) found for signature:
>>> hstack(UniTuple(int64 x 2))
There are 4 candidate implementations:
- Of which 4 did not match due to:
Overload in function '_OverloadWrapper._build.<locals>.ol_generated': File: numba/core/overload_glue.py: Line 129.
With argument(s): '(UniTuple(int64 x 2))':
Rejected as the implementation raised a specific error:
TypeError: np.hstack(): expecting a non-empty tuple of arrays, got UniTuple(int64 x 2)
raised from /usr/local/lib/python3.8/dist-packages/numba/core/typing/npydecl.py:748
During: resolving callee type: Function(<function hstack at 0x7f491475fe50>)
During: typing of call at <ipython-input-41-7a0a3bcd4b1a> (28)
File "<ipython-input-41-7a0a3bcd4b1a>", line 28:
def fun(x, y):
points = np.hstack((x, y))
^
If I try to stack one scalar and one array (which it might be the case in the original function from which this problem arised), the behavior doesn't change:
b = np.ones(3)
res = fun(a,b)
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<function hstack at 0x7f491475fe50>) found for signature:
>>> hstack(Tuple(int64, array(float64, 1d, C)))
There are 4 candidate implementations:
- Of which 4 did not match due to:
Overload in function '_OverloadWrapper._build.<locals>.ol_generated': File: numba/core/overload_glue.py: Line 129.
With argument(s): '(Tuple(int64, array(float64, 1d, C)))':
Rejected as the implementation raised a specific error:
TypeError: np.hstack(): expecting a non-empty tuple of arrays, got Tuple(int64, array(float64, 1d, C))
raised from /usr/local/lib/python3.8/dist-packages/numba/core/typing/npydecl.py:748
During: resolving callee type: Function(<function hstack at 0x7f491475fe50>)
During: typing of call at <ipython-input-42-39bffd13df71> (28)
File "<ipython-input-42-39bffd13df71>", line 28:
def fun(x, y):
points = np.hstack((x, y))
This is very puzzling to me. I am using Numba 0.56.4—which should be the last stable release.
A very similar behavior happens with np.concatenate, too.
Any suggestion would much appreciated.
Thank you!
I want to use the numpy.round_ ina method of a class.
Any calculation done by methods of this class I want to accelerate by using numba.
In general, it works fine. But I somehow do not get numpy.round_ running.
When using numpy.round_ numba throws an error.
Here is my code of a reduced example:
from numba import types
from numba.experimental import jitclass
import numpy as np
spec = [
('arr', types.Array(types.uint8, 1, 'C')),
('quot', types.Array(types.float64, 1, 'C')),
]
#jitclass(spec)
class test:
def __init__(self):
self.arr = np.array((130,190,130),dtype=np.uint8)
def rnd_(self):
quot = np.zeros(3, dtype=np.float64)
val = self.arr
quot = np.round(val/3.0)
return quot
t = test()
a = t.rnd_()
It throws the following error:
TypingError: - Resolution failure for literal arguments:
Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<function round_ at 0x0000021B3500D870>) found for signature:
round_(array(float64, 1d, C))
There are 4 candidate implementations:
- Of which 4 did not match due to:
Overload in function '_OverloadWrapper._build.<locals>.ol_generated': File: numba\core\overload_glue.py: Line 131.
With argument(s): '(array(float64, 1d, C))':
Rejected as the implementation raised a specific error:
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<intrinsic stub>) found for signature:
stub(array(float64, 1d, C))
There are 2 candidate implementations:
- Of which 2 did not match due to:
Intrinsic of function 'stub': File: numba\core\overload_glue.py: Line 35.
With argument(s): '(array(float64, 1d, C))':
No match.
During: resolving callee type: Function(<intrinsic stub>)
During: typing of call at <string> (3)
File "<string>", line 3:
<source missing, REPL/exec in use?>
raised from C:\ProgramData\Anaconda3\envs\mybase_conda\lib\site-packages\numba\core\typeinfer.py:1086
During: resolving callee type: Function(<function round_ at 0x0000021B3500D870>)
During: typing of call at .......\python\playground\tmp.py (27)
File "tmp.py", line 27:
def rnd_(self):
<source elided>
val = self.arr
quot = np.round(val/3.0)
^
- Resolution failure for non-literal arguments:
None
During: resolving callee type: BoundFunction((<class 'numba.core.types.misc.ClassInstanceType'>, 'rnd_') for instance.jitclass.test#21b3c031930<arr:array(uint8, 1d, C),quot:array(float64, 1d, C)>)
During: typing of call at <string> (3)
What am I doing wrong?
Seems like you need to pass round's optional arguments as well. I could reproduce the error with an even smaller example:
#nb.jit(nopython=True)
def foo(x):
return np.round(x)
The fix to this is something like:
#nb.jit(nopython=True)
def foo(x):
out = np.empty_like(x)
np.round(x, 0, out)
return out
So for your case, it should be:
def rnd_(self):
quot = np.zeros(3, dtype=np.float64)
np.round(self.arr / 3.0, 0, quot)
return quot
Following example 12.4 from the following website https://python-programming.quantecon.org/numba.html#id4 i constructed a simple class to model an AR(1) process.
Although the code works fine without the use of #jitclass, the code stops working after I remove ("#").
import numpy as np
import numba
import matplotlib.pyplot as plt
from numba import float64
from numba import int32
from numba.experimental import jitclass
#ar_1_data = [('ρ', float64), ('z_0', float64), ('μ', float64), ('σ', float64)]
##jitclass(ar_1_data)
class ar_1:
def __init__(self, ρ = 0.5, z_0 = 1, μ = 0, σ = 1):
self.ρ = ρ
self.z = z_0
self.lnz = np.log(z_0)
self.μ = μ
self.σ = σ
def update(self):
self.z = self.z**(self.ρ) * np.e**(np.random.normal(self.μ,self.σ))
def sequence(self, n):
path = []
path_log = []
for i in range(n):
path.append(self.z)
path_log.append(np.log(self.z))
self.update()
self.sequence = path
self.sequence_log = path_log
a = ar_1()
a.sequence(100)
Here is the Error im getting after removing the "#":
---------------------------------------------------------------------------
TypingError Traceback (most recent call last)
Input In [83], in <cell line: 1>()
----> 1 a = ar_1()
2 a.sequence(100)
File ~\anaconda3\lib\site-packages\numba\experimental\jitclass\base.py:124, in JitClassType.__call__(cls, *args, **kwargs)
122 bind = cls._ctor_sig.bind(None, *args, **kwargs)
123 bind.apply_defaults()
--> 124 return cls._ctor(*bind.args[1:], **bind.kwargs)
File ~\anaconda3\lib\site-packages\numba\core\dispatcher.py:468, in _DispatcherBase._compile_for_args(self, *args, **kws)
464 msg = (f"{str(e).rstrip()} \n\nThis error may have been caused "
465 f"by the following argument(s):\n{args_str}\n")
466 e.patch_message(msg)
--> 468 error_rewrite(e, 'typing')
469 except errors.UnsupportedError as e:
470 # Something unsupported is present in the user code, add help info
471 error_rewrite(e, 'unsupported_error')
File ~\anaconda3\lib\site-packages\numba\core\dispatcher.py:409, in _DispatcherBase._compile_for_args.<locals>.error_rewrite(e, issue_type)
407 raise e
408 else:
--> 409 raise e.with_traceback(None)
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Cannot resolve setattr: (instance.jitclass.ar_1#2658a3e4d30<ρ:float64,z_0:float64,μ:float64,σ:float64>).z = int64
File "..\..\..\..\..\AppData\Local\Temp\ipykernel_17336\4275445632.py", line 9:
<source missing, REPL/exec in use?>
During: typing of set attribute 'z' at C:\Users\Hogar\AppData\Local\Temp\ipykernel_17336\4275445632.py (9)
File "..\..\..\..\..\AppData\Local\Temp\ipykernel_17336\4275445632.py", line 9:
<source missing, REPL/exec in use?>
During: resolving callee type: jitclass.ar_1#2658a3e4d30<ρ:float64,z_0:float64,μ:float64,σ:float64>
During: typing of call at <string> (3)
During: resolving callee type: jitclass.ar_1#2658a3e4d30<ρ:float64,z_0:float64,μ:float64,σ:float64>
During: typing of call at <string> (3)
File "<string>", line 3:
<source missing, REPL/exec in use?>
I made this function, but numba always give me error. Both chr_pos and pos are 1D arrays. What can be the problem?
#nb.njit
def create_needed_pos(chr_pos, pos):
needed_pos=[]
needed_pos=np.array(needed_pos,dtype=np.float64)
for i in range(len(chr_pos)):
for k in range(len(pos)):
if chr_pos[i] == pos[k]:
if i==1 and k==1:
needed_pos=pos[k]
else:
a=pos[k]
needed_pos=np.append(needed_pos,[a])
return needed_pos
needed_pos=create_needed_pos(chr_pos, pos)
The errors:
warnings.warn(errors.NumbaDeprecationWarning(msg,
<input>:1: NumbaWarning:
Compilation is falling back to object mode WITHOUT looplifting enabled because Function "create_needed_pos" failed type inference due to: Cannot unify array(float64, 1d, C) and int32 for 'needed_pos.1', defined at <input> (5)
File "<input>", line 5:
<source missing, REPL/exec in use?>
During: typing of intrinsic-call at <input> (9)
File "<input>", line 9:
<source missing, REPL/exec in use?>
The message
Cannot unify array(float64, 1d, C) and int32 for 'needed_pos.1'
is telling you that you are trying to assign an integer variable to an array. That happens in this line:
needed_pos=pos[k]
You can do that in normal Python, but Numba requires static types. You must assign an array of floats to an array of floats. For example, replacing the line by
needed_pos = pos[k:k+1]
The same error message says you are trying to assign an int, and this indicates that pos receives an array of ints. You must pass an array of floats instead.
After those changes, Numba still complains here:
needed_pos = []
needed_pos = np.array(needed_pos, dtype=np.float64)
with the message
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Cannot infer the type of variable 'needed_pos', have imprecise type: list(undefined)<iv=None>.
because it doesn't know the type of the elements that needed_pos will contain.
You can replace those two lines with one that creates an array of size zero with a known type:
needed_pos = np.array((0,), dtype=np.float64)
Now the program compiles and produces the same result with or without Numba.
But a problem remains. Numpy arrays work best when they have a fixed size. If you are continuously adding elements you'd better use lists (Numba lists in this case). This way for example:
#nb.njit
def create_needed_pos(chr_pos, pos):
needed_pos = nb.typed.List.empty_list(nb.float64)
for i in range(len(chr_pos)):
for k in range(len(pos)):
if chr_pos[i] == pos[k]:
if i == k == 1:
needed_pos = nb.typed.List([pos[k]])
else:
needed_pos.append(pos[k])
return needed_pos
I'm experimenting with Numba to try and speed up a union-find algorithm I'm working on. Here's some example code. When I experiment with some sample data I cannot understand the type complaint that Numba appears to be raising.
from numba import jit
import numpy as np
indices = np.arange(8806806, dtype=np.int64)
sizes = np.ones(8806806, dtype=np.int64)
connected_components = 8806806
#jit(npython=True)
def root(p: int) -> int:
while p != indices[p]:
indices[p] = indices[indices[p]]
p = indices[p]
return p
#jit(npython=True)
def connected( p: int, q: int) -> bool:
return root(p) == root(q)
#jit(npython=True)
def union( p: int, q: int) -> None:
root1 = root(p)
root2 = root(q)
if root1 == root2:
return
if (sizes[root1] < sizes[root2]):
indices[root1] = root2
sizes[root2] += sizes[root1]
else:
indices[root2] = root1
sizes[root1] += sizes[root2]
connected_components -= 1
#jit(nopython=True)
def process_values(arr):
for row in arr:
typed_arr = row.astype('int64')
for first, second in zip(arr, arr[1:]):
union(first, second)
process_values(
np.array(
[np.array([8018361, 4645960]),
np.array([1137555, 7763897]),
np.array([7532943, 2248813]),
np.array([5352737, 71466, 3590473, 5352738, 2712260])], dtype='object'))
I cannot understand this error:
TypingError Traceback (most recent call last)
<ipython-input-45-62735e65f581> in <module>
44 np.array([1137555, 7763897]),
45 np.array([7532943, 2248813]),
---> 46 np.array([5352737, 71466, 3590473, 5352738, 2712260])], dtype='object'))
/opt/conda/lib/python3.7/site-packages/numba/core/dispatcher.py in _compile_for_args(self, *args, **kws)
399 e.patch_message(msg)
400
--> 401 error_rewrite(e, 'typing')
402 except errors.UnsupportedError as e:
403 # Something unsupported is present in the user code, add help info
/opt/conda/lib/python3.7/site-packages/numba/core/dispatcher.py in error_rewrite(e, issue_type)
342 raise e
343 else:
--> 344 reraise(type(e), e, None)
345
346 argtypes = []
/opt/conda/lib/python3.7/site-packages/numba/core/utils.py in reraise(tp, value, tb)
78 value = tp()
79 if value.__traceback__ is not tb:
---> 80 raise value.with_traceback(tb)
81 raise value
82
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
non-precise type array(pyobject, 1d, C)
[1] During: typing of argument at <ipython-input-45-62735e65f581> (36)
File "<ipython-input-45-62735e65f581>", line 36:
def process_values(arr):
for row in arr:
^
Does this have anything to do with process_values taking an array of irregularly shaped arrays? Any pointers? Thanks!
the problem is that Numba does not accept arrays of dtype 'object'. You seem to be placing arrays inside arrays, you will have to use lists inside lists. Look for the typed.List class in Numba, https://numba.pydata.org/numba-doc/dev/reference/pysupported.html#typed-list
Alternatively, you can use awkward arrays: https://github.com/scikit-hep/awkward-1.0