I have a matrix of zeros with shape (8,30)
matrix
Out[312]:
array([[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])
Then i have another matrix with shape (4,30) like this:
matrix2
Out[313]:
array([[0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,
0, 1, 0, 0, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
1, 0, 0, 0, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,
0, 1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 1]])
What is the quickest way, using numpy, to add to matrix the part corresponding to matrix2 ?
My desired output would be
array([[0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,
0, 1, 0, 0, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
1, 0, 0, 0, 0, 0, 0, 0],
[0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0,
0, 1, 0, 0, 0, 0, 0, 0],
[0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 1],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.],
[0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.]])
still with shape (8,30).
You can use:
matrix[:matrix2.shape[0]] += matrix2
Output:
[[0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0]
[0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0]
[0 1 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0]
[0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]]
Related
I have data predictor in 2D array like this below
array([[ 0, 0, 0, ..., 10, 6, 1],
[ 0, 0, 0, ..., 12, 6, 1],
[ 0, 0, 0, ..., 8, 6, 1],
...,
[ 0, 0, 0, ..., 54, 30, 60],
[ 0, 0, 0, ..., 1472, 5, 348],
[ 0, 0, 0, ..., 58, 45, 60]])
and label data with 3D shape like this below
[array([[0., 0., 1., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[0., 1., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]], dtype=float32),
array([[0., 0., 1., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
...
...,
[0., 1., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.]], dtype=float32),
...]
I would build the model with TensorFlow
model = Sequential()
model.add(Embedding(total_words, 10, input_length=max_sequence_len))
model.add(LSTM(100))
model.add(Dropout(0.1))
model.add(Dense(total_words_label, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer='adam')
But, I get error in latest Dense
ValueError: Shapes (None, 10, 4991) and (None, 4991) are incompatible
It looks like from latest Dense shape because I have 3D label. How the way I can adjust it? I'm learning about TensorFlow.
And, how to convert the output to the 3D label when I have predict it?
--- update (adding the process) --
the example data
input output/label
Hartford Avenue, Bellingham, MA, 2019 {'address': 'Hartford Avenue', 'city': 'Bellingham', 'state': 'MA','zip': '2019'}
Oak Street, Brockton, MA, 2301 {'address': 'Oak Street', 'city': 'Brockton', 'state': 'MA', 'zip': '2301'}
the input is as predictor and the output/label is as label
The process of creating label
tokenizer_label = Tokenizer()
l = list(all['input'])
tokenizer_label.fit_on_texts(l)
total_words_label = len(tokenizer_label.word_index) + 1
labels = []
for i in range(len(l)):
labels.append(keras_utils.to_categorical(tokenizer_label.texts_to_sequences(l[i].split()), num_classes=total_words_label))
I have the first 3D array of size (50,250,250) that includes data points (1,2,3,4,5). I set up a threshold that is 3, where the data points above should equal to 1 and below it equal to 0. the only exception is when the data points are equal to 3, it has to test the second threshold (threshold1=50) that is based on the second 3D array of size (50,250,250). my equation is how to include the two thresholds in my code! In other words, the for loop will check every datapoint in array 1 and perform the first threshold testing, if the datapoint is equal to 3, the for loop should check the counterpart of that datapoint in the second array for the second threshold testing! I have tried the below code, but the results did not make sense
res1=[]
f1=numpy.ones((250, 250))
threshold=3
threshold1=30
for i in array1:
i = i.data
ii= f1*i
ii[ii < threshold] = 0
ii[ii > threshold] = 1
res1.append(ii)
if ii[ii == threshold]:
for j in array2:
j = j.data
jj[jj < threshold1] = 0
jj[jj > threshold1] = 1
res1.append(jj)
Array1:
array([[[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[3., 3., 3., ..., 0., 0., 0.],
[3., 3., 3., ..., 0., 0., 0.],
[3., 3., 3., ..., 0., 0., 0.]],
[[0., 0., 0., ..., 0., 0., 1.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[3., 3., 3., ..., 0., 0., 0.],
[3., 3., 3., ..., 0., 0., 0.],
[3., 3., 3., ..., 0., 0., 0.]],
Array2:[[ nan, nan, nan, ..., nan,
0.9839769, 1.7042577],
[ nan, nan, nan, ..., nan,
nan, nan],
[ nan, nan, nan, ..., 3.2351596,
2.0924768, 1.7604152],
...,
[ nan, nan, nan, ..., 158.48865 ,
158.48865 , 125.888 ],
[ nan, nan, nan, ..., 158.48865 ,
158.48865 , 158.48865 ],
[ nan, nan, nan, ..., 125.88556 ,
158.48865 , 158.48865 ]],
the produced list (rest1)
`[array([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[1., 1., 1., ..., 0., 0., 0.],
[1., 1., 1., ..., 0., 0., 0.],
[1., 1., 1., ..., 0., 0., 0.]]),
array([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
...,
[1., 1., 1., ..., 0., 0., 0.],
[1., 1., 1., ..., 0., 0., 0.],
[1., 1., 1., ..., 0., 0., 0.]]),
array([[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],
[0., 0., 0., ..., 0., 0., 0.],`
IIUC, for your second if condition, you are trying to see whether there is at least a 3 value in that array1, and then you will choose that 2D array of the same position. In that case, you should use in operator.
for i in range(len(array1)):
if threshold in array1[i]:
array2[i][array2[i] < threshold1] = 0
array2[i][array2[i] > threshold1] = 1
res1.append(array2[i])
else:
array1[i][array1[i] < threshold] = 0
array1[i][array1[i] > threshold] = 1
res1.append(array1[i])
The above method is a bit lengthy for numpy. There's a numpy way to do this, too.
array1[array1 < threshold] = 0
array1[array1 > threshold] = 1
array2_condition = np.unique(np.argwhere(array1 == 3)[:,0]) # return the index of array1 if 3 in array1
chosen_array2 = array2[array2_condition]
chosen_array2[chosen_array2 < threshold1] = 0
chosen_array2[chosen_array2 > threshold1] = 1
array2[array2_condition] = chosen_array2 # if you still want array2 values to be changed
res1 = array1
res1[array2_condition] = chosen_array2 # Final result
Update
As was mentioned by the OP, every 2D array has at least a 3 in it. So, the array2_condition is not applicable. Instead, we will modify the array2_condition and use a for loop to change the elements.
res1 = array1
res1[res1 < threshold] = 0
res1[res1 > threshold] = 1
array2_condition = np.argwhere(array1 == 3)
for data in array2_condition:
if array2[tuple(data)] > threshold1:
res1[tuple(data)] = 1
elif array2[tuple(data)] < threshold1:
res12[tuple(data)] = 0
I have this data:
s_result = [{'time': array([ 0. , 0.1, 0.2, ..., 299.7, 299.8, 299.9]), 'I': array([10., 10., 10., ..., 0., 0., 0.]), 'S': array([60., 60., 60., ..., 0., 0., 0.]), 'M': array([40., 40., 40., ..., 0., 0., 0.]), 'R': array([0., 0., 0., ..., 1., 1., 1.]), 'P1': array([ 0., 0., 0., ..., 19., 19., 19.]), 'D1': array([ 0., 0., 0., ..., 81., 81., 81.]), 'P2': array([0., 0., 0., ..., 0., 0., 0.]), 'D2': array([0., 0., 0., ..., 0., 0., 0.]), 'P3': array([0., 0., 0., ..., 0., 0., 0.]), 'D3': array([0., 0., 0., ..., 0., 0., 0.]), 'P4': array([0., 0., 0., ..., 0., 0., 0.]), 'D4': array([0., 0., 0., ..., 0., 0., 0.]), 'P5': array([0., 0., 0., ..., 0., 0., 0.]), 'D5': array([0., 0., 0., ..., 0., 0., 0.]), 'P6': array([0., 0., 0., ..., 0., 0., 0.]), 'D6': array([0., 0., 0., ..., 0., 0., 0.]), 'P7': array([0., 0., 0., ..., 0., 0., 0.]), 'D7': array([0., 0., 0., ..., 0., 0., 0.]), 'P8': array([0., 0., 0., ..., 0., 0., 0.]), 'D8': array([0., 0., 0., ..., 0., 0., 0.]), 'P9': array([0., 0., 0., ..., 0., 0., 0.]), 'D9': array([0., 0., 0., ..., 0., 0., 0.])}, {'time': array([ 0. , 0.1, 0.2, ..., 299.7, 299.8, 299.9]), 'I': array([10., 10., 10., ..., 0., 0., 0.]), 'S': array([60., 60., 60., ..., 0., 0., 0.]), 'M': array([40., 40., 40., ..., 0., 0., 0.]), 'R': array([0., 0., 0., ..., 0., 0., 0.]), 'P1': array([ 0., 0., 0., ..., 20., 20., 20.]), 'D1': array([ 0., 0., 0., ..., 80., 80., 80.]), 'P2': array([0., 0., 0., ..., 0., 0., 0.]), 'D2': array([0., 0., 0., ..., 0., 0., 0.]), 'P3': array([0., 0., 0., ..., 0., 0., 0.]), 'D3': array([0., 0., 0., ..., 0., 0., 0.]), 'P4': array([0., 0., 0., ..., 0., 0., 0.]), 'D4': array([0., 0., 0., ..., 0., 0., 0.]), 'P5': array([0., 0., 0., ..., 0., 0., 0.]), 'D5': array([0., 0., 0., ..., 0., 0., 0.]), 'P6': array([0., 0., 0., ..., 0., 0., 0.]), 'D6': array([0., 0., 0., ..., 0., 0., 0.]), 'P7': array([0., 0., 0., ..., 0., 0., 0.]), 'D7': array([0., 0., 0., ..., 0., 0., 0.]), 'P8': array([0., 0., 0., ..., 0., 0., 0.]), 'D8': array([0., 0., 0., ..., 0., 0., 0.]), 'P9': array([0., 0., 0., ..., 0., 0., 0.]), 'D9': array([0., 0., 0., ..., 0., 0., 0.])}, {'time': array([ 0. , 0.1, 0.2, ..., 299.7, 299.8, 299.9]), 'I': array([10., 10., 10., ..., 0., 0., 0.]), 'S': array([60., 60., 60., ..., 0., 0., 0.]), 'M': array([40., 40., 40., ..., 0., 0., 0.]), 'R': array([0., 0., 0., ..., 0., 0., 0.]), 'P1': array([ 0., 0., 0., ..., 20., 20., 20.]), 'D1': array([ 0., 0., 0., ..., 80., 80., 80.]), 'P2': array([0., 0., 0., ..., 0., 0., 0.]), 'D2': array([0., 0., 0., ..., 0., 0., 0.]), 'P3': array([0., 0., 0., ..., 0., 0., 0.]), 'D3': array([0., 0., 0., ..., 0., 0., 0.]), 'P4': array([0., 0., 0., ..., 0., 0., 0.]), 'D4': array([0., 0., 0., ..., 0., 0., 0.]), 'P5': array([0., 0., 0., ..., 0., 0., 0.]), 'D5': array([0., 0., 0., ..., 0., 0., 0.]), 'P6': array([0., 0., 0., ..., 0., 0., 0.]), 'D6': array([0., 0., 0., ..., 0., 0., 0.]), 'P7': array([0., 0., 0., ..., 0., 0., 0.]), 'D7': array([0., 0., 0., ..., 0., 0., 0.]), 'P8': array([0., 0., 0., ..., 0., 0., 0.]), 'D8': array([0., 0., 0., ..., 0., 0., 0.]), 'P9': array([0., 0., 0., ..., 0., 0., 0.]), 'D9': array([0., 0., 0., ..., 0., 0., 0.])}]
I intend to work on only M as follows:
for index in range(0, 3):
x_stochastic = s_result[index]['M']
x_stochastic=((s_result['M'][0]-s_result['M'][:])/s_result['M'][0])
plt.plot(s_trajectory['time'],x_stochastic , 'r')
plt.xlabel('Time')
plt.ylabel('Monomer Conversion,X')
The expected outcome is 3 different trajectories as shown in the above data. But this is what got graphically:
I couldn't get your code as provided to run. Furthermore in the data snipped you posted all three M Value series are indistinguishable.
I've taken the liberty to modify the data to
s_result = [{'time': np.array([0,100,200,300]),
'M': np.array([40,40,30,0])},
{'time': np.array([0,100,200,300]),
'M': np.array([40,10,5,0])},
]
The corrected loop
for index in range(0, len(s_result)):
x_stochastic = s_result[index]
x_stochastic=((x_stochastic['M'][0]-x_stochastic['M'][:])/x_stochastic['M'][0])
plt.plot(s_result[index]['time'],x_stochastic , 'r')
plt.xlabel('Time')
plt.ylabel('Monomer Conversion,X')
then produces two distinct traces as desired.
Please explain what tokenizer.texts_to_matrix does and what the result is?
from tensorflow.keras.preprocessing.text import Tokenizer
tokenizer = Tokenizer(oov_token="<OOV>")
sentences = [text]
print(sentences)
tokenizer.fit_on_texts(sentences)
word_index = tokenizer.word_index
sequences = tokenizer.texts_to_sequences(sentences)
matrix = tokenizer.texts_to_matrix(sentences)
print(word_index)
print(sequences)
print(matrix)
---
['The fool doth think he is wise, but the wise man knows himself to be a fool.']
# word_index
{'<OOV>': 1, 'the': 2, 'fool': 3, 'wise': 4, 'doth': 5, 'think': 6, 'he': 7, 'is': 8, 'but': 9, 'man': 10, 'knows': 11, 'himself': 12, 'to': 13, 'be': 14, 'a': 15}
# sequences
[[2, 3, 5, 6, 7, 8, 4, 9, 2, 4, 10, 11, 12, 13, 14, 15, 3]]
# matrix
[[0. 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.]]
In the binary mode (default mode), it indicates which words from learnt vocabulary are in the input texts. You have trained your tokenizer on
['The fool doth think he is wise, but the wise man knows himself to be a fool.']
So when you convert the same text to a matrix, it will have all words (indicated by 1) except OOV - because all words are known - hence position at 1 of result vector is 0 (see word_index, and 0 is always 0 since words are enumerated starting from 1
Some examples
tokenizer.texts_to_matrix(['foo'])
# only OOV in this one text
array([[0., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0.]])
tokenizer.texts_to_matrix(['he he'])
# known word, twice (does not matter how often)
array([[0., 0., 0., 0., 0., 0., 0., 1., 0., 0., 0., 0., 0., 0., 0., 0.,
0.]])
tokenizer.texts_to_matrix(['the fool'])
array([[0., 0., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
0.]])
Other mods
Other mods are more clear
count - How many times a word from vocabulary was in the text
tokenizer.texts_to_matrix(['He, he the fool'], mode="count")
array([[0., 0., 1., 1., 0., 0., 0., 2., 0., 0., 0., 0., 0., 0., 0., 0.,
0.]])
freq - count with sum normalized to 1.0
tokenizer.texts_to_matrix(['he he the fool'], mode="freq")
array([[0. , 0. , 0.25, 0.25, 0. , 0. , 0. , 0.5 , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. , 0. ]])
tfidf
tokenizer.texts_to_matrix(['he he the fool'], mode="tfidf")
array([[0. , 0. , 0.84729786, 0.84729786, 0. ,
0. , 0. , 1.43459998, 0. , 0. ,
0. , 0. , 0. , 0. , 0. ,
0. , 0. ]])
I have an array in numpy:
A=np.zeros((5,10))
and I want to change one value in each row with ones according to another array N=np.array([7, 2, 9, 4, 5])
like: A[:,N]=1;
0 0 0 0 0 1 0 0 0 0
0 0 1 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 1
0 0 0 0 1 0 0 0 0 0
0 0 0 0 0 1 0 0 0 0
How can I do that?
Since you want to set a single element per row, you need to fancy-index the first axis using arange(5). this can be thought of as setting indices (I0[0], N[0])=(0,7), (I0[1],N[1])=(1,2), ...
I0 = np.arange(A.shape[0])
A[I0, N] = 1
A
=>
array([[ 0., 0., 0., 0., 0., 0., 0., 1., 0., 0.],
[ 0., 0., 1., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0., 0., 1.],
[ 0., 0., 0., 0., 1., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 1., 0., 0., 0., 0.]])
A.nonzero()
=> (array([0, 1, 2, 3, 4]), array([7, 2, 9, 4, 5]))