I have two 1-D of array of float ('Xdata' and 'tdata'). I want to make a new variable named 'ratedata'. The problem is when I run the code, the console showed "IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices". How to encounter this problem? thank you.
the code:
dxdt_a = np.array(pd.read_excel('T50-katalis1-m14.xlsx',index_col=0,header=5))
Xdata = dxdt_a[:,1]
tdata = dxdt_a[:,0]
ratedata = np.zeros(len(Xdata))
for i in ratedata:
ratedata[i] = (Xdata[i+1]-Xdata[i])/(tdata[1]-tdata[0])
Related
I am trying to use the tf.map_fn() , where my elems should be pointing to the channel dimension of my inputs(shape = 100,24,24,6), so my elems should be a list/tuple of tensors, pointing or accessing the values of the channel dimension(6) of the inputs .I am trying to do it by making a for loop in such a way :
#tf.function
def call(self, inputs, training=True):
elems = []
for b in inputs:
for h in b:
for w in h:
for c in w:
elems.append(c)
changed_inputs = tf.map_fn(self.do_mapping, elems)
return changed_inputs
What i am trying to achieve in the self.do_mapping is that it is doing a dictionary look up for the values of a dictionary (vmap) using the keys and the return the values. the dictionary vmap is made by accessing the output of a layer and appending only the similar values of the channel dimension of the output of layer so the keys in dictionary are tuple of 6 (as the size of channel dimension) tf.tensorobjects and values of dictionary is the count which i keep. This is how the dictionary is made :
value = list(self.get_values())
vmap = {}
cnt = 0
for v0 in value:
for v1 in v0:
for v2 in v1:
for v3 in v2:
v = tuple(v3)
if v not in vmap:
vmap[v]=cnt
cnt+=1
the do_mapping function is :
#tf.function
def do_mapping(self,pixel):
if self._compression :
pixel = tuple(pixel)
enumerated_value=self._vmap.get(pixel)
print(enumerated_value)
print(tf.shape(pixel))
exit()
return enumerated_value
If i try to use the tf.map_fn now where i try to point the elems to the channel dimension then i get the following error :(ValueError: elements in elems must be 1+ dimensional Tensors, not scalars ). Please help me to understand how can i use the tf.map_fn for my case ? Thank you in advance
First, instead of doing a for loop (try to avoid for efficiency), you can just reshape that way:
elems = tf.reshape(inputs,-1)
Second, what do you want to do exactly? What do you mean by "it doesn't work"? What is the error message? What is self.do_mapping?
Best,
Keivan
This is my program-
#n= no. of days
def ATR(df , n):
df['H-L'] = abs(df['High'] - df['Low'])
df['H-PC'] = abs(df['High'] - df['Close'].shift(1))
df['L-PC'] = abs(df['Low'] - df['Close'].shift(1))
df['TR']=df[['H-L','H-PC','L-PC']].max(axis=1)
df['ATR'] = np.nan
df.ix[n-1,'ATR']=df['TR'][:n-1].mean()
for i in range(n , len(df)):
df['ATR'][i] = (df['ATR'][i-1]*(n-1) + df['TR'][i])/n
return
A warning shows up
'DataFrame' object has no attribute 'ix
I tried to replace it with iloc:
df.iloc[df.index[n-1],'ATR'] = df['TR'][:n-1].mean()
But this time another error pops up :
only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices
How to fix this?
Converting code is a pain and we have all been there...
df.ix[n-1,'ATR'] = df['TR'][:n-1].mean()
should become
df['ATR'].iloc[n-1] = df['TR'][:n-1].mean()
Hope this fits the bill
I am in the process of converting a Fortran 90 code to python. The Fortran code includes multidimensional array like:
integer fvpair(3,12,2)
integer jpair(3,2)
real*8 aread(3,12)
I wonder if the following numpy array are correct equivalent, assuming zero initialization:
fvpair = np.array([[np.zeros(3)],[np.zeros(12)],[np.zeros(2)]])
jpair = np.array([[np.zeros(3)],[np.zeros(2)]])
aread = np.array([[np.zeros(3)],[np.zeros(12)]])
If you want to preserve the original Fortran array storage order (column-major), do not forget to pass the order='F' flag!
fvpair = np.zeros((3,12,2), dtype=int, order='F')
jpair = np.zeros((3,2), dtype=int, order='F')
aread = np.zeros((3,2), dtype=float64, order='F')
I have been working on link prediction problem in which the data set, which is a numpy array, has to be parsed and stored into another numpy array. I am trying to do the same but at 9th line it is throwing an IndexError: only integers, slices (:), ellipsis (...), numpy.newaxis (None) and integer or boolean arrays are valid indices. I even tried typecasting the indices with int but it seems to not work. What am I missing here ?
1. train_edges, test_edges, = train_test_split(edgeL,test_size=0.3,random_state=16)
2. out_dim = int(W_out.shape[1])
3. in_dim = int(W_in.shape[1])
4. train_x = np.zeros((len(train_edges), (out_dim + in_dim) * 2))
5. train_y = np.zeros((len(train_edges), 1))
6. for i, edge in enumerate(train_edges):
7. u = edge[0]
8. v = edge[1]
9. train_x[int(i), : int(out_dim)] = W_out[u]
10. train_x[int(i), int(out_dim): int(out_dim + in_dim)] = W_in[u]
11. train_x[i, out_dim + in_dim: out_dim * 2 + in_dim] = W_out[v]
12. train_x[i, out_dim * 2 + in_dim:] = W_in[v]
13. if edge[2] > 0:
14. train_y[i] = 1
15. else:
16. train_y[i] = -1
EDIT:
For reference, The W_out is a 64-dimensional tuple which looks like this
print(W_out[0])
type(W_out.shape[1])
Output:
[[0.10160154 0. 0.70414263 0.6772633 0.07685234 0.75205046
0.421092 0.1776721 0.8622188 0.15669271 0. 0.40653425
0.5768579 0.75861764 0.6745151 0.37883565 0.18074909 0.73928916
0.6289512 0. 0.33160248 0.7441727 0. 0.8810399
0.1110919 0.53732747 0. 0.33330196 0.36220717 0.298112
0.10643011 0.8997948 0.53510064 0.6845873 0.03440218 0.23005858
0.8097505 0.7108275 0.38826624 0.28532124 0.37821335 0.3566149
0.42527163 0.71940386 0.8075657 0.5775364 0.01444144 0.21734199
0.47439903 0.21176265 0.32279345 0.00187511 0.43511534 0.4302601
0.39407462 0.20941389 0.199842 0.8710182 0.2160332 0.30246672
0.27159846 0.19009161 0.32349357 0.08938174]]
int
And edge is a tuple which is from training data set which has source, destination, sign. It looks like this...
train_edges, test_edges, = train_test_split(edgeL,test_size=0.3,random_state=16)
for i, edge in enumerate(train_edges):
print(edge)
print(i)
type(i)
type(edge)
Output:
Streaming output truncated to the last 5000 lines.
2936
['16936', '17031', '1']
2937
['15307', '14904', '1']
2938
['22852', '13045', '1']
2939
['14291', '96703', '1']
2940
Any help/suggestion is highly appreciated.
Your syntax is causing the error.
Looks like accessing the edge object may be the issue. Debug using type() and len() of edge and see what the index error is.
implicitly specifying int(i) is not needed, so the issue will be in the assignment of train_index[x] or your enumeration logic is not right.
As mentioned by #indigo_4_alpha, The error is caused by the 'edge[0]` element which is a string.
Code for checking the train_edges
train_edges, test_edges, = train_test_split(edgeL,test_size=0.3,random_state=16)
for i, edge in enumerate(train_edges):
print(edge)
print(i)
print(edge[0], edge[1],edge[2])
print(type(edge[0]))
Output
['11635' '22046' '1']
2608
11635 22046 1
<class 'str'>
After observing the output, I noticed that individually edge[0] is a string. Then I realized that int(W_out[u] is of no-effect when u itself is a string.
So, I type-casted u=edge[0] to u=int(edge[0]) in the lines 7 and 8 of the code, as shown below.
Master code for Train and test data split
1. train_edges, test_edges, = train_test_split(edgeL,test_size=0.3,random_state=16)
2. out_dim = int(W_out.shape[1])
3. in_dim = int(W_in.shape[1])
4. train_x = np.zeros((len(train_edges), (out_dim + in_dim) * 2))
5. train_y = np.zeros((len(train_edges), 1))
6. for i, edge in enumerate(train_edges):
7. u = int(edge[0])
8. v = int(edge[1])
Thank you one and all for sparing your time and giving me your valuable suggestions.
I'm trying to calculate the average Luminance of an RGB image. To do this, I find the luminance of each pixel i.e.
L(r,g,b) = X*r + Y*g + Z*b (some linear combination).
And then find the average by summing up luminance of all pixels and dividing by width*height.
To speed this up, I'm using pyopencl.reduction.ReductionKernel
The array I pass to it is a Single Dimension Numpy Array so it works just like the example given.
import Image
import numpy as np
im = Image.open('image_00000001.bmp')
data = np.asarray(im).reshape(-1) # so data is a single dimension list
# data.dtype is uint8, data.shape is (w*h*3, )
I want to incorporate the following code from the example into it . i.e. I would make changes to datatype and the type of arrays I'm passing. This is the example:
a = pyopencl.array.arange(queue, 400, dtype=numpy.float32)
b = pyopencl.array.arange(queue, 400, dtype=numpy.float32)
krnl = ReductionKernel(ctx, numpy.float32, neutral="0",
reduce_expr="a+b", map_expr="x[i]*y[i]",
arguments="__global float *x, __global float *y")
my_dot_prod = krnl(a, b).get()
Except, my map_expr will work on each pixel and convert each pixel to its luminance value.
And reduce expr remains the same.
The problem is, it works on each element in the array, and I need it to work on each pixel which is 3 consecutive elements at a time (RGB ).
One solution is to have three different arrays, one for R, one for G and one for B ,which would work, but is there another way ?
Edit: I changed the program to illustrate the char4 usage instead of float4:
import numpy as np
import pyopencl as cl
import pyopencl.array as cl_array
deviceID = 0
platformID = 0
workGroup=(1,1)
N = 10
testData = np.zeros(N, dtype=cl_array.vec.char4)
dev = cl.get_platforms()[platformID].get_devices()[deviceID]
ctx = cl.Context([dev])
queue = cl.CommandQueue(ctx)
mf = cl.mem_flags
Data_In = cl.Buffer(ctx, mf.READ_WRITE, testData.nbytes)
prg = cl.Program(ctx, """
__kernel void Pack_Cmplx( __global char4* Data_In, int N)
{
int gid = get_global_id(0);
//Data_In[gid] = 1; // This would change all components to one
Data_In[gid].x = 1; // changing single component
Data_In[gid].y = 2;
Data_In[gid].z = 3;
Data_In[gid].w = 4;
}
""").build()
prg.Pack_Cmplx(queue, (N,1), workGroup, Data_In, np.int32(N))
cl.enqueue_copy(queue, testData, Data_In)
print testData
I hope it helps.