Why Numba does not work when I'm calling a function over and over into another? - numpy

Question

I have a code to calculate skewness over 100 matrices or more. The matrix is actually a directed percolation. I defined 2 functions. the first: doPercolationStep() defines how does this random matrix should be filled. The second: manual(hl) produces this matrix over and over which means calls the doPercolationStep() over and over then calculates skewness of these random matrices. When I run the code with Numba I get This error:
No implementation of function Function(<built-in function setitem>) found for signature:
>>> setitem(array(undefined, 1d, C), int64, array(float64, 1d, C))
There are 16 candidate implementations:
- Of which 16 did not match due to:
Overload of function 'setitem': File: <numerous>: Line N/A.
With argument(s): '(array(undefined, 1d, C), int64, array(float64, 1d, C))':
No match.
During: typing of setitem at <timed exec> (55)
File "<timed exec>", line 55:
<source missing, REPL/exec in use?>
My First function is :
%%time
import numpy as np
import random as rand
from numba import jit , njit , prange
from pylab import *
import matplotlib.pyplot as plt
from numpy import linalg as la
import statistics as stt
#njit(parallel=True)
def doPercolationStep(vector, PROP, time):
even = time%2
vector_copy = np.copy(vector)
WIDTH = len(vector)
for i in range(even, WIDTH, 2):
if vector[i] == 1:
pro1 = np.random.rand()
pro2 = np.random.rand()
if pro1 < PROP:
vector_copy[(i+WIDTH-1)%WIDTH] = 1
if pro2 < PROP:
vector_copy[(i+1)%WIDTH] = 1
vector_copy[i] = 0
return vector_copy
And my main function is :
li=700
#njit(parallel=True)
def manual(hl):
WIDTH = hl
HEIGHT = hl
#PROP = 0.644
L = hl
p = linspace(0.1,0.9,15)
nx = len(p)
N = 100000
sk=[]
ku = []
for ip in range(nx):
w0=[]
for i in range(N):
vector = np.zeros(WIDTH)
vector[WIDTH//2] = 1
PROP=p[ip]
result = []
#result.append(vector)
for i in range(HEIGHT):
vector=doPercolationStep(vector, PROP, i)
result.append(vector)
#np.savetxt('result.dat', result, fmt='%d')
ss=np.array(result)
ss=ss.astype(np.int64)
##ss=np.int(result)
###ss= result
ss = np.where(ss==0, -1, ss)
ww=(ss+(ss.T))/2
re_size=ww/(np.sqrt(L))
w, v = la.eigh(re_size)
w=w.real
w=max(w)
w0.append(w)
w1=np.array(w0)
w1_mean=np.mean(w1)
w1_std=np.std(w1)
w1_std_3=(w1_std)**3
w1_num=N
w1_3=0
for ai in w1:
w1_3+=(ai-w1_mean)**3
w1_skew=(w1_3)/((w1_num)*(w1_std_3))
sk.append(w1_skew)
#kyu=kurtosis(w0)
#ku.append(kyu)
return sk
manual(li)
And finally, I get this error:
TypingError: Failed in nopython mode pipeline (step: nopython frontend)
No implementation of function Function(<built-in function setitem>) found for signature:
>>> setitem(array(undefined, 1d, C), int64, array(float64, 1d, C))
There are 16 candidate implementations:
- Of which 16 did not match due to:
Overload of function 'setitem': File: <numerous>: Line N/A.
With argument(s): '(array(undefined, 1d, C), int64, array(float64, 1d, C))':
No match.
During: typing of setitem at <timed exec> (55)
File "<timed exec>", line 55:
<source missing, REPL/exec in use?>

Answer 1

You are not showing the relevant part of the error:
No implementation of function Function(<built-in function setitem>) found for signature:
>>> setitem(array(undefined, 1d, C), int64, array(float64, 1d, C))
There are 16 candidate implementations:
- Of which 16 did not match due to:
Overload of function 'setitem': File: <numerous>: Line N/A.
With argument(s): '(array(undefined, 1d, C), int64, array(float64, 1d, C))':
No match.
During: typing of setitem at /home/jotaele/Devel/codetest/tests/test_numba.py (2452)
File "test_numba.py", line 2452:
def manual(hl):
<source elided>
vector = doPercolationStep(vector, PROP, i)
result.append(vector)
^
This tells you exactly what's happening.
Python's setitem(a,b,c) sets the value of a at index b with the value in c.
In your case, result receives the value, but it is defined as [], so Numba doesn't know its type.
You need to initialize it as an array whose size and type you know in advance. Your code is now shorter and runs with Numba:
...
PROP = p[ip]
ss = np.empty((HEIGHT, WIDTH), dtype=np.int64)
for i in range(HEIGHT):
ss[i] = doPercolationStep(vector, PROP, i)
ss = np.where(ss == 0, -1, ss)
...