I have a tensor with shape (k,1) and I want to replicate the first column n times. That is, the resulting tensor should have the shape (k,n) with matrix values copied from the first column. How to achieve this using tf.tile?
You can use tf.tile() operation. For example:
a = tf.constant([[1], [2], [3]], dtype = tf.float32)
c = tf.tile(a, [1,3])
with tf.Session() as sess:
print(sess.run(c))
returns:
[[1. 1. 1.]
[2. 2. 2.]
[3. 3. 3.]]
Related
As a tensorflow newbie, I'm trying to concatenate two tensor, t1 and t2, together during graph construction. t1, t2 have different ranks: [B, T, feat_dim1] and [B, feat_dim2]. But T can only be known during runtime, so in graph construction the shapes of t1, t2 are actually [B, None, feat_dim1] and [B, feat_dim2]. What I wanted is to append t2 to t1 to get a tensor with the shape: [B, None, feat1+feat2].
The first thing I thought of using is tf.stack([t2, t2, ...], axis=1) to expand the rank, but since T=None during graph construction, I cannot build the list for tf.stack(). I also checked tf.while_loop for building the list with tf.Tensor object, but couldn't get the gist of using function.
Currently the code I am working on doesn't support eager mode, so could someone give me some hint about how to concatenate t1 and t2? or how to expand t2 to [B, T, feat2] given T=None during graph construction? Thanks a lot for any suggestions.
Add another dimension to tensor t2: (B, feat_dim2) --> (B, 1, feat_dim2).
Tile tensor t2 None times along the previously added second dimension, where None is the dynamic second dimension of tensor t1.
Concatenate t1 and t2 along the last dimension.
import tensorflow as tf
import numpy as np
B = 5
feat_dim1 = 3
feat_dim2 = 4
t1 = tf.placeholder(tf.float32, shape=(B, None, feat_dim1)) # [5, None, 3]
t2 = 2.*tf.ones(shape=(B, feat_dim2)) # [5, 4]
def concat_tensors(t1, t2):
t2 = t2[:, None, :] # 1. `t1`: [5, 4]` --> `[5, 1, 4]`
tiled = tf.tile(t2, [1, tf.shape(t1)[1], 1]) # 2. `[5, 1, 4]` --> `[5, None, 4]`
res = tf.concat([t1, tiled], axis=-1) # 3. concatenate `t1`, `t2` --> `[5, None, 7]`
return res
res = concat_tensors(t1, t2)
with tf.Session() as sess:
print(res.eval({t1: np.ones((B, 2, feat_dim1))}))
# [[[1. 1. 1. 2. 2. 2. 2.]
# [1. 1. 1. 2. 2. 2. 2.]]
#
# [[1. 1. 1. 2. 2. 2. 2.]
# [1. 1. 1. 2. 2. 2. 2.]]
#
# [[1. 1. 1. 2. 2. 2. 2.]
# [1. 1. 1. 2. 2. 2. 2.]]
#
# [[1. 1. 1. 2. 2. 2. 2.]
# [1. 1. 1. 2. 2. 2. 2.]]
#
# [[1. 1. 1. 2. 2. 2. 2.]
# [1. 1. 1. 2. 2. 2. 2.]]]
For simplicity, assume the image has only 1 channel, we have one filter of size [2,2] which could be written as W=[[a, b],[c,d]], I want to use this set of parameters to do 2 convolutions, first is just using W itself:
output = tf.nn.conv2d(input, W)
Second convolution I would like to use filter W_2=[[d,c],[b,a]] which is basically some kind of 'transpose' of W. It needs to share parameters with W. How should I write this in tensorflow?
Many thanks!
You can just reverse the weights:
import tensorflow as tf
w = tf.Variable([[1, 2.], [3, 4]])
w_transposed = w[::-1, ::-1]
grad = tf.gradients(w_transposed, w)[0]
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print('Values')
print(w.eval())
print(w_transposed.eval())
print('Gradient')
print(grad.eval())
prints
Values
[[1. 2.]
[3. 4.]]
[[4. 3.]
[2. 1.]]
Gradient
[[1. 1.]
[1. 1.]]
I want to create an M*N tensor where all elements are all zeros except one random element per row which shall be one but I don't know how.
This is one way to do that:
import tensorflow as tf
m = 4
n = 6
dt = tf.float32
random_idx = tf.random_uniform((m, 1), maxval=n, dtype=tf.int32)
result = tf.cast(tf.equal(tf.range(n)[tf.newaxis], random_idx), dtype=dt)
with tf.Session() as sess:
print(sess.run(result))
Output:
[[ 0. 0. 0. 0. 0. 1.]
[ 0. 0. 1. 0. 0. 0.]
[ 0. 1. 0. 0. 0. 0.]
[ 0. 1. 0. 0. 0. 0.]]
i'm trying to create a convolutional neural network, which predicts whether or not to sell for a hydropower dam, the issue i am having is the output. I input two inputs, price(a normalized float) and waterinflow (either 1 or 0 at this point).
My issue is that running this and trying to get the answer as a set of actions 0/1, gives me floats which do not make any sense other than if the output is set as the corresponding number instead of the set of actions. This is fine when the amount of actions are small, but will be horrible later on when the number of actions are extended.
Does anyone know how i can make it so that it outputs the actions as either 0 or 1, instead of the floats which seem to be certainty of the prediction.
Meaning if there are 4 actions, and the correct answer is 0, 1, 0, 1, then the predictions should be in the same form(4 actions either 0 or 1)
Any help would be much appreciated
Binary output from Normalized Probability
What you are looking for is a method of converting your normalized probability output to a binary one.
This is very straight forward in Tensorflow and involves added a tf.round function. The trick is to make sure you do not use the output tf.round in training. This is best demonstrated using a working code example.
Working code example
This code calculates the XOR function using a neural net. The outputs are y_out (the probability output) and y_binary (the casting of the probability output to binary)
### imports
import tensorflow as tf
import numpy as np
### constant data
x = [[0.,0.],[1.,1.],[1.,0.],[0.,1.]]
y_ = [[1.,0.],[1.,0.],[0.,1.],[0.,1.]]
### induction
# 1x2 input -> 2x3 hidden sigmoid -> 3x1 sigmoid output
# Layer 0 = the x2 inputs
x0 = tf.placeholder( dtype=tf.float32 , shape=[None,2] )
y0 = tf.placeholder( dtype=tf.float32 , shape=[None,2] )
# Layer 1 = the 2x3 hidden sigmoid
m1 = tf.Variable( tf.random_uniform( [2,3] , minval=0.1 , maxval=0.9 , dtype=tf.float32 ))
b1 = tf.Variable( tf.random_uniform( [3] , minval=0.1 , maxval=0.9 , dtype=tf.float32 ))
h1 = tf.sigmoid( tf.matmul( x0,m1 ) + b1 )
# Layer 2 = the 3x2 softmax output
m2 = tf.Variable( tf.random_uniform( [3,2] , minval=0.1 , maxval=0.9 , dtype=tf.float32 ))
b2 = tf.Variable( tf.random_uniform( [2] , minval=0.1 , maxval=0.9 , dtype=tf.float32 ))
y_logit = tf.matmul( h1,m2 ) + b2
y_out = tf.nn.softmax( y_logit )
y_binary = tf.round( y_out )
### loss
# loss : a loss function that uses y_logit or y_out , but NOT y_binary
loss = tf.reduce_sum( tf.square( y0 - y_out ) )
# training step
train = tf.train.GradientDescentOptimizer(1.0).minimize(loss)
### training
# run 500 times using all the X and Y
# print out the loss and any other interesting info
with tf.Session() as sess:
sess.run( tf.global_variables_initializer() )
print "\nloss"
for step in range(500) :
sess.run(train, feed_dict={x0:x,y0:y_})
if (step + 1) % 100 == 0 :
print sess.run(loss, feed_dict={x0:x,y0:y_})
y_out_value , y_binary_value = sess.run([y_out,y_binary], feed_dict={x0:x,y0:y_})
print "\nThe expected output is :"
print np.array(y_)
print "\nThe softmax output is :"
print np.array(y_out_value)
print "\nThe binary output is :"
print np.array(y_binary_value)
print ""
Output
The expected output is :
[[ 1. 0.]
[ 1. 0.]
[ 0. 1.]
[ 0. 1.]]
The softmax output is :
[[ 0.96538627 0.03461381]
[ 0.81609273 0.18390732]
[ 0.11534476 0.88465524]
[ 0.0978259 0.90217412]]
The binary output is :
[[ 1. 0.]
[ 1. 0.]
[ 0. 1.]
[ 0. 1.]]
As you can see, you can retrieve the probability outputs OR the probabilities cast as binary and still have all the benefits of classic logits.
Cheers.
I guess it is important to note that the output of neural nets are actually posterior probability computed on each element of the classes present---for a typical classification problem.
The figures returned tells you how likely is the ouput to be of class A, B, C given the input x. So that you can not expect to get 0 or 1 always.
#An example would be if I get
Output = [0.5,0.2,0.3] given input x.
#I predict the class should be A because it has posterior of 0.5
(the highest value of the 3 values returned).
Class = A (0.5)
# Or I might as well round it up. Tensor flow can do this for you
So I guess you should get the output and apply probabilistic assumptions thats fit your model like say the highest value in the returned predictions gives the class it belongs.
It might not be easy to wait for absolute one or zero prediction.
Be careful of this fact I wrote above. Its a common mistake. And please do read the paper below. Once you have posteriors, you can add and build models on them. There is no limitation to what you can achieve!
For example you can apply Gaussian Mixture models/ Markov models/Build decision Tress/Combine expert systems on the output, those are the elegant and scientific approach.
Read this paper for more info.
http://www.ee.iisc.ac.in/people/faculty/prasantg/downloads/NeuralNetworksPosteriors_Lippmann1991.pdf
Hope it helps!
I am trying to apply DBSCAN on a dataset of (Lan,Lat) .. The algorithm is very sensitive for the parameter; EPS & MinPts.
I would like to have a look through a Histogram over the data, to determine the proper values. Unfortunately, Matplotlib Hist() take only 1D array.
Passing a 2D matrix as argument, Hist() treats each column as a separate input.
Scatter plot and histograms:
Does anyone has a way to solve this,
If you follow the DBSCAN article, you only need the 4-nearest-neighbor distance for each object, not all pairwise distances. I.e., a 1 dimensional array.
Instead of doing a histogram, they sort the values, and try to choose a knee in this plot.
find the 4 nearest neighbor of each object
collect all 4NN distances in one array
sort this array in descending order
plot the resulting curve
look for a knee, often best at around 5%-10% of your x axis (so 95%-90% of objects are core points).
For details, see the original DBSCAN publication!
You could use numpy.histogram2d:
import numpy as np
np.random.seed(2016)
N = 100
arr = np.random.random((N, 2))
xedges = np.linspace(0, 1, 10)
yedges = np.linspace(0, 1, 10)
lat = arr[:, 0]
lng = arr[:, 1]
hist, xedges, yedges = np.histogram2d(lat, lng, (xedges, yedges))
print(hist)
yields
[[ 0. 0. 5. 0. 3. 0. 0. 0. 3.]
[ 0. 3. 0. 3. 0. 0. 4. 0. 2.]
[ 2. 2. 1. 1. 1. 1. 3. 0. 1.]
[ 2. 1. 0. 3. 1. 2. 1. 1. 3.]
[ 3. 0. 3. 2. 0. 1. 0. 2. 0.]
[ 3. 2. 3. 1. 1. 2. 1. 1. 0.]
[ 2. 3. 0. 1. 0. 1. 3. 0. 0.]
[ 1. 1. 1. 1. 2. 0. 2. 1. 1.]
[ 0. 1. 1. 0. 1. 1. 2. 0. 0.]]
Or to visualize the histogram:
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.imshow(hist)
plt.show()