Related
I'm working on a logistic regression assignment and my professor has this code example.
What is the new_x variable and why are we transforming it as a matrix?
data = pd.DataFrame( {’id’: [ 1,2,3,4,5,6,7,8], ’Label’: [’green’, ’green’, ’green’, ’green’,
’red’, ’red’, ’red’, ’red’],
’Height’: [5, 5.5, 5.33, 5.75, 6.00, 5.92, 5.58, 5.92],
’Weight’: [100, 150, 130, 150, 180, 190, 170, 165], ’Foot’: [6, 8, 7, 9, 13, 11, 12, 10]},
columns = [’id’, ’Height’, ’Weight’, ’Foot’, ’Label’] )
X = data[[’Height’, ’Weight’]].values
scaler = StandardScaler()
scaler.fit(X)
X = scaler.transform(X)
Y = data[’Label’].values
log_reg_classifier = LogisticRegression()
log_reg_classifier.fit(X,Y)
new_x = scaler.transform(np.asmatrix([6, 160]))
predicted = log_reg_classifier.predict(new_x)
accuracy = log_reg_classifier.score(X, Y)
Let's take it step by step.
data = pd.DataFrame( {’id’: [ 1,2,3,4,5,6,7,8], ’Label’: [’green’, ’green’, ’green’, ’green’,
’red’, ’red’, ’red’, ’red’],
’Height’: [5, 5.5, 5.33, 5.75, 6.00, 5.92, 5.58, 5.92],
’Weight’: [100, 150, 130, 150, 180, 190, 170, 165], ’Foot’: [6, 8, 7, 9, 13, 11, 12, 10]},
columns = [’id’, ’Height’, ’Weight’, ’Foot’, ’Label’] )
You create an initial feature matrix that contains the columns [’id’, ’Height’, ’Weight’, ’Foot’, ’Label’].
scaler = StandardScaler()
scaler.fit(X)
X = scaler.transform(X)
Y = data[’Label’].values
You than obtain a np.array, that contains only weight and height using data[[’Height’, ’Weight’]].values. See pandas docs on slicing for more info. You can obtain the size of the feature matrix with X.shape i. e., [n,2].
X = data[[’Height’, ’Weight’]].values
scaler = StandardScaler()
scaler.fit(X)
X = scaler.transform(X)
Y = data[’Label’].values
log_reg_classifier = LogisticRegression()
log_reg_classifier.fit(X,Y)
You use those two features only to train the logistic regression after standardization.
That is your classifier is learned on two features (i. e., height and weight) only, but mutliple samples. Every classifier in sklearn implements the fit() method to fit the classifier to the training data.
As your model is trained on a feature matrix with two features, your sample that you want to predict (new_x) also needs two features. Thus, you first create a np.asmatrix([6, 160] with shape [1,2] and elements [height=6,weight=160], scale it and pass it to your trained model. log_reg_classifier.predict(new_x) returns the prediction. You assess the performance of the classifier by comparing the prediction with the true label and calculating the (mean) accuracy. Et voila.
new_x = scaler.transform(np.asmatrix([6, 160]))
predicted = log_reg_classifier.predict(new_x)
accuracy = log_reg_classifier.score(X, Y)
I'm trying to create a CNN to solve a problem. The input_shape for first layer I provided was (20, 196, 1).
However, when I do model.summary() I get dimensions as (None, 20, 196, 1) and my X is a list of features of dimensions (20, 196, 1). While performing model.fit() I get this error -
Error when checking input: expected input_1 to have 4 dimensions, but
got array with shape (20, 196, 1).
Can anyone point out what I'm doing wrong? Also, if I wanted to increase the dimension from (20, 196, 1) to (None, 20, 196, 1), what do I do?
The first axis should always correspond to the batch size.
For example, consider the case where you can want N elements in your batch. Each element consists of input features with dimension (20, 196, 1). Now, your batch would have a size of (N, 20, 196, 1).
An option would be stacking the samples on the first axis: first, create a list of samples, then assign this to the input data. For example:
# list of samples with size (20, 196, 1)
list_of_samples = [x1, x2, x3, .. xn]
# your input data would be:
input_batch = np.array(list_of_samples)
Otherwise, if your samples xi are already tensors, another possibility is to stack them on the first axis:
# given xi = tensor with shape (20, 196, 1), for i = 1, 2,..., N
input_batch = tf.stack([x1, x2, x3, ..., xn], axis=0)
# input_batch has now shape (N, 20, 196, 1)
I have tensor (None, 196) and after reshaping, it becomes (None, 14, 14).
And now, I want to copy it to channel axis, so that the shape should be (None, 14, 14, 512). Lastly, I want to copy to timestep axis, so it becomes (None, 10, 14, 14, 512). I accomplish those steps using this snippet code:
def replicate(tensor, input_target):
batch_size = K.shape(tensor)[0]
nf, h, w, c = input_target
x = K.reshape(tensor, [batch_size, 1, h, w, 1])
# Replicate to channel dimension
x = K.tile(x, [batch_size, 1, 1, 1, c])
# Replicate to timesteps dimension
x = K.tile(x, [batch_size, nf, 1, 1, 1])
return x
x = ...
x = Lambda(replicate, arguments={'input_target':input_shape})(x)
another_x = Input(shape=input_shape) # shape (10, 14, 14, 512)
x = layers.multiply([x, another_x])
x = ...
I plot the model and the output shape is just like I want it to be. But, the problem arises in model training. I set the batch size to 2. This the the error message:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [8,10,14,14,512] vs. [2,10,14,14,512]
[[{{node multiply_1/mul}} = Mul[T=DT_FLOAT, _class=["loc:#training/Adam/gradients/multiply_1/mul_grad/Sum"], _device="/job:localhost/replica:0/task:0/device:GPU:0"](Lambda_2/Tile_1, _arg_another_x_0_0/_189)]]
[[{{node metrics/top_k_categorical_accuracy/Mean_1/_265}} = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_6346_metrics/top_k_categorical_accuracy/Mean_1", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
Looks like, K.tile() increases the batch size from 2 to 8. When I set the batch size to 10, it becomes 1000.
So, my question is how to achieve the result as I want? Is it good way to use tile()? Or, should I use repeat_elements()? Thanks!
I am using Tensorflow 1.12.0 and Keras 2.2.4.
As a rule of thumb, try to avoid bringing batch size to the transformations happening in the Lambda layer.
When you use tile operation, you only set only the dimension that needs to change (for example you had batch_size value in your tile operation which is wrong). Also I am using tf.tile instead of K.tile (TF 1.12 doesn't have tile in the Keras backend it seems).
def replicate(tensor, input_target):
_, nf, h, w, c = input_target
x = K.reshape(tensor, [-1, 1, h, w, 1])
# Replicate to channel dimension
# You can combine below lines to tf.tile(x, [1, nf, 1, 1, c]) as well
x = tf.tile(x, [1, 1, 1, 1, c])
# Replicate to timesteps dimension
x = tf.tile(x, [1, nf, 1, 1, 1])
return x
Simple example
input_shape= [None, 10, 14, 14, 512]
x = Input(shape=(196,))
x = Lambda(replicate, arguments={'input_target':input_shape})(x)
print(x.shape)
Which gives
>>> (?, 10, 14, 14, 512)
I have a 5-d numpy array, the shape is (5, 1000, 32, 32, 3), which means there are 3 channels of 32*32 pixels, and 1000 samples, 5 different timestamps. How do I print specific 32*32 data, for example, I want to print the 32*32 data from 16th sample, 2nd timestamp, 1st channel?
With a mix of inedexing and slicing this can be done like this:
arr = np.random.randint(1000, size=(5, 1000, 32, 32, 3))
result = arr[1, 15, :, :, 0]
print(result.shape)
This will output the shape of the result:
(32, 32)
I am learning the TensorFlow, building a multilayer_perceptron model. I am looking into some examples like the one at: https://github.com/aymericdamien/TensorFlow-Examples/blob/master/notebooks/3_NeuralNetworks/multilayer_perceptron.ipynb
I then have some questions in the code below:
def multilayer_perceptron(x, weights, biases):
:
:
pred = multilayer_perceptron(x, weights, biases)
:
:
with tf.Session() as sess:
sess.run(init)
:
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
print ("Accuracy:", accuracy.eval({x: X_test, y: y_test_onehot}))
I am wondering what do tf.argmax(prod,1) and tf.argmax(y,1) mean and return (type and value) exactly? And is correct_prediction a variable instead of real values?
Finally, how do we get the y_test_prediction array (the prediction result when the input data is X_test) from the tf session? Thanks a lot!
tf.argmax(input, axis=None, name=None, dimension=None)
Returns the index with the largest value across axis of a tensor.
input is a Tensor and axis describes which axis of the input Tensor to reduce across. For vectors, use axis = 0.
For your specific case let's use two arrays and demonstrate this
pred = np.array([[31, 23, 4, 24, 27, 34],
[18, 3, 25, 0, 6, 35],
[28, 14, 33, 22, 20, 8],
[13, 30, 21, 19, 7, 9],
[16, 1, 26, 32, 2, 29],
[17, 12, 5, 11, 10, 15]])
y = np.array([[31, 23, 4, 24, 27, 34],
[18, 3, 25, 0, 6, 35],
[28, 14, 33, 22, 20, 8],
[13, 30, 21, 19, 7, 9],
[16, 1, 26, 32, 2, 29],
[17, 12, 5, 11, 10, 15]])
Evaluating tf.argmax(pred, 1) gives a tensor whose evaluation will give array([5, 5, 2, 1, 3, 0])
Evaluating tf.argmax(y, 1) gives a tensor whose evaluation will give array([5, 5, 2, 1, 3, 0])
tf.equal(x, y, name=None) takes two tensors(x and y) as inputs and returns the truth value of (x == y) element-wise.
Following our example, tf.equal(tf.argmax(pred, 1),tf.argmax(y, 1)) returns a tensor whose evaluation will givearray(1,1,1,1,1,1).
correct_prediction is a tensor whose evaluation will give a 1-D array of 0's and 1's
y_test_prediction can be obtained by executing pred = tf.argmax(logits, 1)
The documentation for tf.argmax and tf.equal can be accessed by following the links below.
tf.argmax() https://www.tensorflow.org/api_docs/python/math_ops/sequence_comparison_and_indexing#argmax
tf.equal() https://www.tensorflow.org/versions/master/api_docs/python/control_flow_ops/comparison_operators#equal
Reading the documentation:
tf.argmax
Returns the index with the largest value across axes of a tensor.
tf.equal
Returns the truth value of (x == y) element-wise.
tf.cast
Casts a tensor to a new type.
tf.reduce_mean
Computes the mean of elements across dimensions of a tensor.
Now you can easily explain what it does. Your y is one-hot encoded, so it has one 1 and all other are zero. Your pred represents probabilities of classes. So argmax finds the positions of best prediction and correct value. After that you check whether they are the same.
So now your correct_prediction is a vector of True/False values with the size equal to the number of instances you want to predict. You convert it to floats and take the average.
Actually this part is nicely explained in TF tutorial in the Evaluate the Model part
tf.argmax(input, axis=None, name=None, dimension=None)
Returns the index with the largest value across axis of a tensor.
For the case in specific, it receives pred as argument for it's input and 1 as axis. The axis describes which axis of the input Tensor to reduce across. For vectors, use axis = 0.
Example: Given the list [2.11,1.0021,3.99,4.32] argmax will return 3 which is the index of the highest value.
correct_prediction is a tensor that will be evaluated later. It is not a regular python variable. It contains the necessary information to compute the value later.
For this specific case, it will be part of another tensor accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float")) and will be evaluated by eval on accuracy.eval({x: X_test, y: y_test_onehot}).
y_test_prediction should be your correct_prediction tensor.
For those who do not have much time to understand tf.argmax:
x = np.array([[1, 9, 3],[4, 5, 6]])
tf.argmax(x, axis = 0)
output:
[array([1, 0, 1], dtype=int64)]
tf.argmax(x, axis = 1)
Output:
[array([1, 2], dtype=int64)]
source