I am having trouble compiling a simple function in no-Python mode with Numba:
#njit
def fun(x,y):
points = np.hstack((x,y))
return points
a = 5
b = 2
res = fun(a,b)
While this very simple script works without the #njit decorator, if run with it throws the error:
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<function hstack at 0x7f491475fe50>) found for signature:
>>> hstack(UniTuple(int64 x 2))
There are 4 candidate implementations:
- Of which 4 did not match due to:
Overload in function '_OverloadWrapper._build.<locals>.ol_generated': File: numba/core/overload_glue.py: Line 129.
With argument(s): '(UniTuple(int64 x 2))':
Rejected as the implementation raised a specific error:
TypeError: np.hstack(): expecting a non-empty tuple of arrays, got UniTuple(int64 x 2)
raised from /usr/local/lib/python3.8/dist-packages/numba/core/typing/npydecl.py:748
During: resolving callee type: Function(<function hstack at 0x7f491475fe50>)
During: typing of call at <ipython-input-41-7a0a3bcd4b1a> (28)
File "<ipython-input-41-7a0a3bcd4b1a>", line 28:
def fun(x, y):
points = np.hstack((x, y))
^
If I try to stack one scalar and one array (which it might be the case in the original function from which this problem arised), the behavior doesn't change:
b = np.ones(3)
res = fun(a,b)
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<function hstack at 0x7f491475fe50>) found for signature:
>>> hstack(Tuple(int64, array(float64, 1d, C)))
There are 4 candidate implementations:
- Of which 4 did not match due to:
Overload in function '_OverloadWrapper._build.<locals>.ol_generated': File: numba/core/overload_glue.py: Line 129.
With argument(s): '(Tuple(int64, array(float64, 1d, C)))':
Rejected as the implementation raised a specific error:
TypeError: np.hstack(): expecting a non-empty tuple of arrays, got Tuple(int64, array(float64, 1d, C))
raised from /usr/local/lib/python3.8/dist-packages/numba/core/typing/npydecl.py:748
During: resolving callee type: Function(<function hstack at 0x7f491475fe50>)
During: typing of call at <ipython-input-42-39bffd13df71> (28)
File "<ipython-input-42-39bffd13df71>", line 28:
def fun(x, y):
points = np.hstack((x, y))
This is very puzzling to me. I am using Numba 0.56.4—which should be the last stable release.
A very similar behavior happens with np.concatenate, too.
Any suggestion would much appreciated.
Thank you!
Related
I want to use the numpy.round_ ina method of a class.
Any calculation done by methods of this class I want to accelerate by using numba.
In general, it works fine. But I somehow do not get numpy.round_ running.
When using numpy.round_ numba throws an error.
Here is my code of a reduced example:
from numba import types
from numba.experimental import jitclass
import numpy as np
spec = [
('arr', types.Array(types.uint8, 1, 'C')),
('quot', types.Array(types.float64, 1, 'C')),
]
#jitclass(spec)
class test:
def __init__(self):
self.arr = np.array((130,190,130),dtype=np.uint8)
def rnd_(self):
quot = np.zeros(3, dtype=np.float64)
val = self.arr
quot = np.round(val/3.0)
return quot
t = test()
a = t.rnd_()
It throws the following error:
TypingError: - Resolution failure for literal arguments:
Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<function round_ at 0x0000021B3500D870>) found for signature:
round_(array(float64, 1d, C))
There are 4 candidate implementations:
- Of which 4 did not match due to:
Overload in function '_OverloadWrapper._build.<locals>.ol_generated': File: numba\core\overload_glue.py: Line 131.
With argument(s): '(array(float64, 1d, C))':
Rejected as the implementation raised a specific error:
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<intrinsic stub>) found for signature:
stub(array(float64, 1d, C))
There are 2 candidate implementations:
- Of which 2 did not match due to:
Intrinsic of function 'stub': File: numba\core\overload_glue.py: Line 35.
With argument(s): '(array(float64, 1d, C))':
No match.
During: resolving callee type: Function(<intrinsic stub>)
During: typing of call at <string> (3)
File "<string>", line 3:
<source missing, REPL/exec in use?>
raised from C:\ProgramData\Anaconda3\envs\mybase_conda\lib\site-packages\numba\core\typeinfer.py:1086
During: resolving callee type: Function(<function round_ at 0x0000021B3500D870>)
During: typing of call at .......\python\playground\tmp.py (27)
File "tmp.py", line 27:
def rnd_(self):
<source elided>
val = self.arr
quot = np.round(val/3.0)
^
- Resolution failure for non-literal arguments:
None
During: resolving callee type: BoundFunction((<class 'numba.core.types.misc.ClassInstanceType'>, 'rnd_') for instance.jitclass.test#21b3c031930<arr:array(uint8, 1d, C),quot:array(float64, 1d, C)>)
During: typing of call at <string> (3)
What am I doing wrong?
Seems like you need to pass round's optional arguments as well. I could reproduce the error with an even smaller example:
#nb.jit(nopython=True)
def foo(x):
return np.round(x)
The fix to this is something like:
#nb.jit(nopython=True)
def foo(x):
out = np.empty_like(x)
np.round(x, 0, out)
return out
So for your case, it should be:
def rnd_(self):
quot = np.zeros(3, dtype=np.float64)
np.round(self.arr / 3.0, 0, quot)
return quot
I have a code to calculate skewness over 100 matrices or more. The matrix is actually a directed percolation. I defined 2 functions. the first: doPercolationStep() defines how does this random matrix should be filled. The second: manual(hl) produces this matrix over and over which means calls the doPercolationStep() over and over then calculates skewness of these random matrices. When I run the code with Numba I get This error:
No implementation of function Function(<built-in function setitem>) found for signature:
>>> setitem(array(undefined, 1d, C), int64, array(float64, 1d, C))
There are 16 candidate implementations:
- Of which 16 did not match due to:
Overload of function 'setitem': File: <numerous>: Line N/A.
With argument(s): '(array(undefined, 1d, C), int64, array(float64, 1d, C))':
No match.
During: typing of setitem at <timed exec> (55)
File "<timed exec>", line 55:
<source missing, REPL/exec in use?>
My First function is :
%%time
import numpy as np
import random as rand
from numba import jit , njit , prange
from pylab import *
import matplotlib.pyplot as plt
from numpy import linalg as la
import statistics as stt
#njit(parallel=True)
def doPercolationStep(vector, PROP, time):
even = time%2
vector_copy = np.copy(vector)
WIDTH = len(vector)
for i in range(even, WIDTH, 2):
if vector[i] == 1:
pro1 = np.random.rand()
pro2 = np.random.rand()
if pro1 < PROP:
vector_copy[(i+WIDTH-1)%WIDTH] = 1
if pro2 < PROP:
vector_copy[(i+1)%WIDTH] = 1
vector_copy[i] = 0
return vector_copy
And my main function is :
li=700
#njit(parallel=True)
def manual(hl):
WIDTH = hl
HEIGHT = hl
#PROP = 0.644
L = hl
p = linspace(0.1,0.9,15)
nx = len(p)
N = 100000
sk=[]
ku = []
for ip in range(nx):
w0=[]
for i in range(N):
vector = np.zeros(WIDTH)
vector[WIDTH//2] = 1
PROP=p[ip]
result = []
#result.append(vector)
for i in range(HEIGHT):
vector=doPercolationStep(vector, PROP, i)
result.append(vector)
#np.savetxt('result.dat', result, fmt='%d')
ss=np.array(result)
ss=ss.astype(np.int64)
##ss=np.int(result)
###ss= result
ss = np.where(ss==0, -1, ss)
ww=(ss+(ss.T))/2
re_size=ww/(np.sqrt(L))
w, v = la.eigh(re_size)
w=w.real
w=max(w)
w0.append(w)
w1=np.array(w0)
w1_mean=np.mean(w1)
w1_std=np.std(w1)
w1_std_3=(w1_std)**3
w1_num=N
w1_3=0
for ai in w1:
w1_3+=(ai-w1_mean)**3
w1_skew=(w1_3)/((w1_num)*(w1_std_3))
sk.append(w1_skew)
#kyu=kurtosis(w0)
#ku.append(kyu)
return sk
manual(li)
And finally, I get this error:
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<built-in function setitem>) found for signature:
>>> setitem(array(undefined, 1d, C), int64, array(float64, 1d, C))
There are 16 candidate implementations:
- Of which 16 did not match due to:
Overload of function 'setitem': File: <numerous>: Line N/A.
With argument(s): '(array(undefined, 1d, C), int64, array(float64, 1d, C))':
No match.
During: typing of setitem at <timed exec> (55)
File "<timed exec>", line 55:
<source missing, REPL/exec in use?>
You are not showing the relevant part of the error:
No implementation of function Function(<built-in function setitem>) found for signature:
>>> setitem(array(undefined, 1d, C), int64, array(float64, 1d, C))
There are 16 candidate implementations:
- Of which 16 did not match due to:
Overload of function 'setitem': File: <numerous>: Line N/A.
With argument(s): '(array(undefined, 1d, C), int64, array(float64, 1d, C))':
No match.
During: typing of setitem at /home/jotaele/Devel/codetest/tests/test_numba.py (2452)
File "test_numba.py", line 2452:
def manual(hl):
<source elided>
vector = doPercolationStep(vector, PROP, i)
result.append(vector)
^
This tells you exactly what's happening.
Python's setitem(a,b,c) sets the value of a at index b with the value in c.
In your case, result receives the value, but it is defined as [], so Numba doesn't know its type.
You need to initialize it as an array whose size and type you know in advance. Your code is now shorter and runs with Numba:
...
PROP = p[ip]
ss = np.empty((HEIGHT, WIDTH), dtype=np.int64)
for i in range(HEIGHT):
ss[i] = doPercolationStep(vector, PROP, i)
ss = np.where(ss == 0, -1, ss)
...
What does this error message mean?
TypeError: Could not build a TypeSpec for name: "tf.print/PrintV2"
op: "PrintV2"
input: "tf.print/StringFormat"
attr {
key: "end"
value {
s: "\n"
}
}
attr {
key: "output_stream"
value {
s: "stdout"
}
}
of unsupported type <class 'google3.third_party.tensorflow.python.framework.ops.Operation'>
I'm printing the shape of a tensor. My code "works" without the print, so I'm sure it is this statement, and the tensor is valid. I can print the shape of a tensor in a test colab. I'm clueless how to narrow this down and debug this. My failure is in a big hairy program.
I can't find any information on the web about what might be causing this error.
What does it mean when I get a TypeSpec error from a tf.print?
-- Malcolm
(TF 2.7.0)
I'm sorry for the tardy followup.
Turns out that the output from Keras layers is not a regular tf.tensor. I still don't understand the reason, the error message, or how to give a better message. :-(
Here is a simple example of the problem (and the error message) and an (undocumented) solution.
import tensorflow as tf
keras_input = tf.keras.layers.Input([10])
tf.print(keras_input)
==> TypeError: Could not build a TypeSpec for name: "tf.print_2/PrintV2"
tf.keras.backend.print_tensor(keras_input)
==> <KerasTensor: shape=(None, 10) dtype=float32 (created by layer 'tf.keras.backend.print_tensor')>
So the moral of the story is use tf.keras.backend.print_tensor when working with Keras models.
I gotta say, numba seems to be usable only in extremely simplistic use cases carefully designed to be presented in talks
I can run the following code just fine:
def rt(hi):
for i in hi:
hi_ = i == hi
t = hi[hi_]
return None
rt(np.array(['a','b','c','d'],dtype='U'))
But, when i decorate the above code with njit:
#njit
def rt(hi):
for i in hi:
hi_ = i == hi
t = hi[hi_]
return None
rt(np.array(['a','b','c','d'],dtype='U'))
I get the following error:
---------------------------------------------------------------------------
TypingError Traceback (most recent call last)
<ipython-input-34-eadef1d0ecee> in <module>
5 t = hi[hi_]
6 return None
----> 7 rt(np.array(['a','b','c','d'],dtype='U'))
~/miniconda/envs/IndusInd_credit_cards_collections_scorecard_/lib/python3.8/site-packages/numba/core/dispatcher.py in _compile_for_args(self, *args, **kws)
418 e.patch_message(msg)
419
--> 420 error_rewrite(e, 'typing')
421 except errors.UnsupportedError as e:
422 # Something unsupported is present in the user code, add help info
~/miniconda/envs/IndusInd_credit_cards_collections_scorecard_/lib/python3.8/site-packages/numba/core/dispatcher.py in error_rewrite(e, issue_type)
359 raise e
360 else:
--> 361 raise e.with_traceback(None)
362
363 argtypes = []
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<built-in function getitem>) found for signature:
>>> getitem(array([unichr x 1], 1d, C), Literal[bool](False))
There are 22 candidate implementations:
- Of which 20 did not match due to:
Overload of function 'getitem': File: <numerous>: Line N/A.
With argument(s): '(array([unichr x 1], 1d, C), bool)':
No match.
- Of which 1 did not match due to:
Overload in function 'GetItemBuffer.generic': File: numba/core/typing/arraydecl.py: Line 162.
With argument(s): '(array([unichr x 1], 1d, C), bool)':
Rejected as the implementation raised a specific error:
TypeError: unsupported array index type bool in [bool]
raised from /home/sarthak/miniconda/envs/IndusInd_credit_cards_collections_scorecard_/lib/python3.8/site-packages/numba/core/typing/arraydecl.py:68
- Of which 1 did not match due to:
Overload in function 'GetItemBuffer.generic': File: numba/core/typing/arraydecl.py: Line 162.
With argument(s): '(array([unichr x 1], 1d, C), Literal[bool](False))':
Rejected as the implementation raised a specific error:
TypeError: unsupported array index type Literal[bool](False) in [Literal[bool](False)]
raised from /home/sarthak/miniconda/envs/IndusInd_credit_cards_collections_scorecard_/lib/python3.8/site-packages/numba/core/typing/arraydecl.py:68
During: typing of intrinsic-call at <ipython-input-34-eadef1d0ecee> (5)
File "<ipython-input-34-eadef1d0ecee>", line 5:
def rt(hi):
<source elided>
hi_ = i == hi
t = hi[hi_]
^
How to subset a 1-d array using a boolean 1-d array in numba decorated function?
I made this function, but numba always give me error. Both chr_pos and pos are 1D arrays. What can be the problem?
#nb.njit
def create_needed_pos(chr_pos, pos):
needed_pos=[]
needed_pos=np.array(needed_pos,dtype=np.float64)
for i in range(len(chr_pos)):
for k in range(len(pos)):
if chr_pos[i] == pos[k]:
if i==1 and k==1:
needed_pos=pos[k]
else:
a=pos[k]
needed_pos=np.append(needed_pos,[a])
return needed_pos
needed_pos=create_needed_pos(chr_pos, pos)
The errors:
warnings.warn(errors.NumbaDeprecationWarning(msg,
<input>:1: NumbaWarning:
Compilation is falling back to object mode WITHOUT looplifting enabled because Function "create_needed_pos" failed type inference due to: Cannot unify array(float64, 1d, C) and int32 for 'needed_pos.1', defined at <input> (5)
File "<input>", line 5:
<source missing, REPL/exec in use?>
During: typing of intrinsic-call at <input> (9)
File "<input>", line 9:
<source missing, REPL/exec in use?>
The message
Cannot unify array(float64, 1d, C) and int32 for 'needed_pos.1'
is telling you that you are trying to assign an integer variable to an array. That happens in this line:
needed_pos=pos[k]
You can do that in normal Python, but Numba requires static types. You must assign an array of floats to an array of floats. For example, replacing the line by
needed_pos = pos[k:k+1]
The same error message says you are trying to assign an int, and this indicates that pos receives an array of ints. You must pass an array of floats instead.
After those changes, Numba still complains here:
needed_pos = []
needed_pos = np.array(needed_pos, dtype=np.float64)
with the message
numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Cannot infer the type of variable 'needed_pos', have imprecise type: list(undefined)<iv=None>.
because it doesn't know the type of the elements that needed_pos will contain.
You can replace those two lines with one that creates an array of size zero with a known type:
needed_pos = np.array((0,), dtype=np.float64)
Now the program compiles and produces the same result with or without Numba.
But a problem remains. Numpy arrays work best when they have a fixed size. If you are continuously adding elements you'd better use lists (Numba lists in this case). This way for example:
#nb.njit
def create_needed_pos(chr_pos, pos):
needed_pos = nb.typed.List.empty_list(nb.float64)
for i in range(len(chr_pos)):
for k in range(len(pos)):
if chr_pos[i] == pos[k]:
if i == k == 1:
needed_pos = nb.typed.List([pos[k]])
else:
needed_pos.append(pos[k])
return needed_pos