My tensor training data set structure is as follows:
[[1 3 0 99 4 ... 9 0 9],
...
[3 2 3 1 3 ... 9 9 8]]
In which there are 798 "rows" or individual arrays and each array is of length 9000, or has 9000 "columns".
The label data is like this: [ [1 0 1 0 0 ... 0 0 0 1]]. It is one array with 798 columns.
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.LSTM(100,batch_input_shape=(798,1000,9000)))
model.compile(loss='mean_absolute_error',optimizer='adam',metrics=['accuracy'])
model.fit(training_data,training_labels,batch_size=150,epochs=10,validation_data=(validation_data,validation_labels))
But I kept getting the error:
ValueError: Input 0 of layer sequential_2 is incompatible with the layer: expected ndim=3, found ndim=2. Full shape received: [None, 9000]
I'm not sure what I'm doing wrong. Should I specify the batch_input_shape? Should I specify the input_shape? What do samples, batch size, time step, and features mean in relation to my data?
Related
Is it possible to calculate the internal node predictions of an xgboost model? The R package, gbm, provides a prediction for internal nodes of each tree.
The xgboost output, however only shows predictions for the final leaves of the model.
xgboost output:
Notice that the Quality column has the final prediction for the leaf node in row 6. I would like that value for each of the internal nodes as well.
Tree Node ID Feature Split Yes No Missing Quality Cover
1: 0 0 0-0 Sex=female 0.50000 0-1 0-2 0-1 246.6042790 222.75
2: 0 1 0-1 Age 13.00000 0-3 0-4 0-4 22.3424225 144.25
3: 0 2 0-2 Pclass=3 0.50000 0-5 0-6 0-5 60.1275253 78.50
4: 0 3 0-3 SibSp 2.50000 0-7 0-8 0-7 23.6302433 9.25
5: 0 4 0-4 Fare 26.26875 0-9 0-10 0-9 21.4425507 135.00
6: 0 5 0-5 Leaf NA <NA> <NA> <NA> 0.1747126 42.50
R gbm output:
In the R gbm package output, the prediction column contains values for both leaf nodes (SplitVar == -1) and the internal nodes. I would like access to these values from the xgboost model
SplitVar SplitCodePred LeftNode RightNode MissingNode ErrorReduction Weight Prediction
0 1 0.000000000 1 8 15 32.564591 445 0.001132514
1 2 9.500000000 2 3 7 3.844470 282 -0.085827382
2 -1 0.119585850 -1 -1 -1 0.000000 15 0.119585850
3 0 1.000000000 4 5 6 3.047926 207 -0.092846157
4 -1 -0.118731665 -1 -1 -1 0.000000 165 -0.118731665
5 -1 0.008846912 -1 -1 -1 0.000000 42 0.008846912
6 -1 -0.092846157 -1 -1 -1 0.000000 207 -0.092846157
Question:
How do I access or calculate predictions for the internal nodes of an xgboost model? I would like to use them for a greedy, poor man's version of SHAP scores.
The solution to this problem is to dump the xgboost json object with all_stats=True. That adds the cover statistic to the output which can be used to distribute the leaf points through the internal nodes:
def _calculate_contribution(node: AnyNode) -> float32:
if isinstance(node, Leaf):
return node.contrib
else:
return (
node.left.cover * Node._calculate_contribution(node.left)
+ node.right.cover * Node._calculate_contribution(node.right)
) / node.cover
The internal contribution is the weighted average of the child contributions. Using this method, the generated results exactly match those returned when calling the predict method with pred_contribs=True and approx_contribs=True.
Problem: How can i convert a .tflite (serialised flat buffer) to .pb (frozen model)? The documentation only talks about one way conversion.
Use-case is: I have a model that is trained on converted to .tflite but unfortunately, i do not have details of the model and i would like to inspect the graph, how can i do that?
I found the answer here
We can use Interpreter to analysis the model and the same code looks like following:
import numpy as np
import tensorflow as tf
# Load TFLite model and allocate tensors.
interpreter = tf.lite.Interpreter(model_path="converted_model.tflite")
interpreter.allocate_tensors()
# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# Test model on random input data.
input_shape = input_details[0]['shape']
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()
output_data = interpreter.get_tensor(output_details[0]['index'])
print(output_data)
Netron is the best analysis/visualising tool i found, it can understand lot of formats including .tflite.
I don't think there is a way to restore tflite back to pb as some information are lost after conversion. I found an indirect way to have a glimpse on what is inside tflite model is to read back each of the tensor.
interpreter = tf.contrib.lite.Interpreter(model_path=model_path)
interpreter.allocate_tensors()
# trial some arbitrary numbers to find out the num of tensors
num_layer = 89
for i in range(num_layer):
detail = interpreter._get_tensor_details(i)
print(i, detail['name'], detail['shape'])
and you would see something like below. As there are only limited of operations that are currently supported, it is not too difficult to reverse engineer the network architecture. I have put some tutorials too on my Github
0 MobilenetV1/Logits/AvgPool_1a/AvgPool [ 1 1 1 1024]
1 MobilenetV1/Logits/Conv2d_1c_1x1/BiasAdd [ 1 1 1 1001]
2 MobilenetV1/Logits/Conv2d_1c_1x1/Conv2D_bias [1001]
3 MobilenetV1/Logits/Conv2d_1c_1x1/weights_quant/FakeQuantWithMinMaxVars [1001 1 1 1024]
4 MobilenetV1/Logits/SpatialSqueeze [ 1 1001]
5 MobilenetV1/Logits/SpatialSqueeze_shape [2]
6 MobilenetV1/MobilenetV1/Conv2d_0/Conv2D_Fold_bias [32]
7 MobilenetV1/MobilenetV1/Conv2d_0/Relu6 [ 1 112 112 32]
8 MobilenetV1/MobilenetV1/Conv2d_0/weights_quant/FakeQuantWithMinMaxVars [32 3 3 3]
9 MobilenetV1/MobilenetV1/Conv2d_10_depthwise/Relu6 [ 1 14 14 512]
10 MobilenetV1/MobilenetV1/Conv2d_10_depthwise/depthwise_Fold_bias [512]
11 MobilenetV1/MobilenetV1/Conv2d_10_depthwise/weights_quant/FakeQuantWithMinMaxVars [ 1 3 3 512]
12 MobilenetV1/MobilenetV1/Conv2d_10_pointwise/Conv2D_Fold_bias [512]
13 MobilenetV1/MobilenetV1/Conv2d_10_pointwise/Relu6 [ 1 14 14 512]
14 MobilenetV1/MobilenetV1/Conv2d_10_pointwise/weights_quant/FakeQuantWithMinMaxVars [512 1 1 512]
15 MobilenetV1/MobilenetV1/Conv2d_11_depthwise/Relu6 [ 1 14 14 512]
16 MobilenetV1/MobilenetV1/Conv2d_11_depthwise/depthwise_Fold_bias [512]
17 MobilenetV1/MobilenetV1/Conv2d_11_depthwise/weights_quant/FakeQuantWithMinMaxVars [ 1 3 3 512]
18 MobilenetV1/MobilenetV1/Conv2d_11_pointwise/Conv2D_Fold_bias [512]
19 MobilenetV1/MobilenetV1/Conv2d_11_pointwise/Relu6 [ 1 14 14 512]
20 MobilenetV1/MobilenetV1/Conv2d_11_pointwise/weights_quant/FakeQuantWithMinMaxVars [512 1 1 512]
I have done this with TOCO, using tf 1.12
tensorflow_1.12/tensorflow/bazel-bin/tensorflow/contrib/lite/toco/toco --
output_file=coco_ssd_mobilenet_v1_1.0_quant_2018_06_29.pb --
output_format=TENSORFLOW_GRAPHDEF --input_format=TFLITE --
input_file=coco_ssd_mobilenet_v1_1.0_quant_2018_06_29.tflite --
inference_type=FLOAT --input_type=FLOAT --input_array="" --output_array="" --
input_shape=1,450,450,3 --dump_grapHviz=./
(you can remove the dump_graphviz option)
I have 6 classes and I used tf-slim in Tensorflow to obtained the confusion matrix such as
[[41 2 0 0 0 0]
[ 1 11 4 1 0 0]
[ 0 1 12 0 0 0]
[ 0 0 0 22 1 0]
[ 0 0 0 0 7 0]
[ 0 0 0 0 0 20]]
My question is that what is confusion matrix order of the above table? Is it right if I said that the columns represent the prediction label, while the rows represent the true label? Some reference said on opposite side.
Did you use tf.confusion_matrix(labels,predictions)?
If so, the columns represent the predicton labels, whereas the rows represent the real labels.
The usual representation is
PREDICTED
[[41 2 0 0 0 0]
T [ 1 11 4 1 0 0]
R [ 0 1 12 0 0 0]
U [ 0 0 0 22 1 0]
E [ 0 0 0 0 7 0]
[ 0 0 0 0 0 20]]
As pointed out by M. Rath (+1), this is also what Tensorflow does. This means for 41 samples you correctly predicted class 0. For 2 samples, you predicted class 1, but it actually was class 0.
Please note that you can also manipulate the order for visualizations. So instead of
class 0, class 1, class 2
you could have (for both, prediction and true value) the order
class 0, class 2, class 1
This contains the same information, but a visualization might convey a different story. See my masters thesis Analysis and Optimization of
Convolutional Neural Network Architectures page 48 (Confusion Matrix Ordering), especially figure 5.12 and 5.13.
An implementation can be found in the tool clana
I coded some of the neural networks like image classifier,mnist and NLP I
got an accuracy of 98 percent on my GPU (NVIDIA GT 610). How can I feed new data (not training data) to my neural network and get the predictions?
Let's suppose:
Inputs Output
0 0 1 0
1 1 1 1
1 0 1 1
0 1 1 0
I got an accuracy of 98.7 how to give an input like [ 1 1 0] and predict the output. Is there any method in Tensorflow to do this?
If your output variable is y and your input placeholder is x:
sess.run(y, feed_dict={x: mnist.test.images})
See https://www.tensorflow.org/get_started/mnist/beginners
I have a byte tensor of integer class labels, e.g. from the MNIST data set.
1
7
5
[torch.ByteTensor of size 3]
How do use it to create a tensor of 1-hot vectors?
1 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 0 0 0
0 0 0 0 1 0 0 0 0 0
[torch.DoubleTensor of size 3x10]
I know I could do this with a loop, but I'm wondering if there's any clever Torch indexing that will get it for me in a single line.
indices = torch.LongTensor{1,7,5}:view(-1,1)
one_hot = torch.zeros(3, 10)
one_hot:scatter(2, indices, 1)
You can find the documentation for scatter in the torch/torch7 github readme (in the master branch).
An alternate method is to shuffle rows from an identity matrix:
indicies = torch.LongTensor{1,7,5}
one_hot = torch.eye(10):index(1, indicies)
This was not my idea, I found it in karpathy/char-rnn.