Working on simple softmax tensorflow classification with 4 features/inputs and 3 outputs.
To save results : input vs predicted output vs actual output , doing tf.concat across all tensors and print(Tensor.eval()) to review results.
Wondering in real use case what are the best ways to review results which are not scalar(So tensorboard scalar not an option) , but N-D type results. Any tool to plot graph with tensor outputs? Dump to file(Data could be huge , hard to analyze)? Dump to database? Depends on real time requirements , but looking for any tools to plug in output of tensorflow.
Please share thoughts. Thanks!
You can of course request any tensor and get it back in numpy format, from there you can do what you like with it.
Tensorboard is the main visualization tool in tensorflow. It allows you to display scalar and histogram data types, as well as images and audio clips as you train.
Here are a few images from tensorboard of scalar values, distribution, and histogram
Scalar values are showing error rates
Distribution shows the distribution of gradient magnitudes
Histogram shows a 3D histogram of gradient magnitudes over time
Related
I have been trying to code a model that looks at an image with a grid and returns a matrix with the contents of that grid.
Here is an example of the input image:
Input
And this should be the output:
[30202133333,
12022320321,
23103100322,
13103110301,
22221301212,
33100210001,
11012010320,
21230233011,
00330223230,
02121221220,
23133103321,
23110110330]
With 0: Blue, 1: Pink, 2: Lavender, 3: Green
I have a hard time finding resources on how to do this. What would be the simpelst way?
Thanks in advance!
There could be multiple design choices to generate this type of output. I suggest using Autoencoders.
Here is some information about Autoencoders taken from Wikipedia -
An autoencoder is a type of artificial neural network used to learn
efficient codings of unlabeled data (unsupervised learning).1 The
encoding is validated and refined by attempting to regenerate the
input from the encoding. The autoencoder learns a representation
(encoding) for a set of data, typically for dimensionality reduction,
by training the network to ignore insignificant data (“noise”).
While autoencoders are typically used to reconstruct the input, you have a slightly different problem of mapping the input to a specific matrix.
You'd want to set up the architecture by providing images as input and the corresponding matrices as your "labels." The architecture can be further optimized by using Convolutional layers instead of MLP layers.
I am trying pixelcnn, which is auto-regressive generative model. After training, the model receive an all-zero tensor and generate the next pixel form the left top coner. Now that the model parameters are fixed, does the model only can produce the same outputs starting from the same zero tensor? How to produce different samples?
Yes, you always provide an all-zero tensor. However, for PixelCNN each pixel location is represented by a distribution. So when you do the forward pass you then sample from a random distribution at the end. That is how the pixel values are different each run.
This is of course because PixelCNN is a probabilistic neural network. So the pixels, as mentioned before, are all represented by conditional probability distributions of all the layers below, not just point estimates.
I was trying to visualize the output of all the activation functions in each layers of my fully-connected network and I was surprised when I checked the last layer. It is a very simple regression model and the output layer has therefore one neuron. I know it would be better to visualize it as a scalar, but I was just trying to visualize it using histograms and I realized that the value is somehow split. Does this make any sense? I would rather expect that it cannot be visualized at all or that the histogram would consist of a single point.
Attention vector for sequence 2 sequence model is basically a array of shape [batch_size, time_step,1], which indicates the weighs of a particular time step.
But if I use tf.summary.histogram to show it on tensorboard, tensorflow will only show the distributions of weights, I can't tell the which time step is more important. I can use tf.summary.scalar, but length of my source sequence is 128 , it is too much plots. The most nature way of show this kind of data it a picture like this, but how can I do it in tensorboad?
Tensorboard does not currently support visualizing tensor summaries. There is a summary op for it, but Tensorboard will just skip it when reading summaries from disk at the moment. I am also not aware of any third party plugins that support this, though it is very much doable.
In your plot, it seems like there are only 19 time steps. One way is to create 19 scalar summaries. Alternatively, you can use tf.summary.tensor_summary op, but process the tensorboard event file (containing this data) with your own script.
I'm trying to use TensorFlow to train output servo commands given an input image.
I plan on using a file as #mrry suggested in this question, with the images like so:
../some/path/some_img.JPG *some_label*
My question is, what are the label formats I can provide to TensorFlow and what structures are suggested?
My data is basically n servo commands from 0-10 seconds. A vector would work great:
[0,2,4,3]
or similarly:
[0,.25,.4,.3]
I couldn't find much about labels in the docs. Can anyone shed any light on TensorFlow labels?
And a very related question is what is the best way to structure these for TensorFlow to properly learn from them?
In Tensorflow Labels are just generic tensor. You can use any kind of tensor to store your labels. In your case a 1-D tensor with shape (4,) seems to be desired.
Labels do only differ from the rest of the data by its use in the computational graph. (Usually) labels should only be used inside the loss function while you propagate the other data through the whole network. For your problem a 4-d regression function should work.
Also, look at my newest comment to the (old) question. Using the slice_input_producer seems to be preferable in your case.