I am implementing a simple linear regression code in google collab and trying to visualize the results with tensorboard with the following command
%tensorboard --logdir=/tmp/lr-train.
However, when I run this command, the tensorboard just simply does not show up. Instead I just see the following message Reusing TensorBoard on port 6012 (pid 3219), started 0:07:06 ago. (Use '!kill 3219' to kill it.)
How to launch the tensorboard in my case? Here is the code I am trying to run:
%tensorflow_version 1.x
import tensorflow as tf
import numpy as np
N = 100
x_zeros = np.random.multivariate_normal(
mean=np.array((-1, -1)), cov = 0.1 * np.eye(2), size = (N//2))
y_zeros = np.zeros((N//2,))
x_ones = np.random.multivariate_normal(mean=np.array((1, 1)), cov = 0.1 * np.eye(2), size=(N//2))
y_ones = np.zeros((N//2))
x_np = np.vstack([x_zeros, x_ones])
y_np = np.concatenate([y_zeros, y_ones])
with tf.name_scope("placeholders"):
x = tf.placeholder(tf.float32, (N, 2))
y = tf.placeholder(tf.float32, (N, 1))
with tf.name_scope("weights"):
W = tf.Variable(tf.random_normal((2, 1)))
b = tf.Variable(tf.random_normal((1, )))
with tf.name_scope("prediction"):
y_pred = tf.matmul(x, W) + b
with tf.name_scope("loss"):
l = tf.reduce_sum((y - y_pred)**2)
with tf.name_scope("optim"):
train_op = tf.train.AdamOptimizer(0.05).minimize(l)
with tf.name_scope("summaries"):
tf.summary.scalar("loss", l)
merged = tf.summary.merge_all()
train_writer = tf.summary.FileWriter('/tmp/lr-train', tf.get_default_graph())
n_steps = 100
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
# Train model
for i in range(n_steps):
feed_dict = {x: x_np, y: y_np.reshape(-1,1)}
_, summary, loss = sess.run([train_op, merged, l], feed_dict=feed_dict)
if i%10 == 0:
print("step %d, loss: %f" % (i, loss))
I tried an example in tf-2, and tensorboard launched without any issues with the same command.
I tried your code in Colab and was able to reproduce what you mentioned and found a solution that worked as described below.
Use a ”space” between —logdir and /tmp/lr-train instead of a =.
What did not work as mentioned in the question:
%load_ext tensorboard
%tensorboard --logdir=/tmp/lr-train
What did work:
%load_ext tensorboard
%tensorboard --logdir /tmp/lr-train
You can also terminate your active session and then rerun all cells afterward. But maybe you even found a way of killing a tensorboard within a session.
Related
Using TensorFlow 1.9, I want to train a neural network in one Python file, and then restore the network using a different Python file. I have tried to do this using a simple example, but when I try to load my "prediction" operation, I receive an error. Specifically, the error is: KeyError: "The name 'prediction' refers to an Operation not in the graph.".
Below is my Python file to train and save the network. It generates some example data and trains a simple neural network, then saves the network every epoch.
import numpy as np
import tensorflow as tf
input_data = np.zeros([100, 10])
label_data = np.zeros([100, 1])
for i in range(100):
for j in range(10):
input_data[i, j] = i * j / 1000
label_data[i] = 2 * input_data[i, 0] + np.random.uniform(0.01)
input_placeholder = tf.placeholder(tf.float32, shape=[None, 10], name='input_placeholder')
label_placeholder = tf.placeholder(tf.float32, shape=[None, 1], name='label_placeholder')
x = tf.layers.dense(inputs=input_placeholder, units=10, activation=tf.nn.relu)
x = tf.layers.dense(inputs=x, units=10, activation=tf.nn.relu)
prediction = tf.layers.dense(inputs=x, units=1, name='prediction')
loss_op = tf.reduce_mean(tf.square(prediction - label_placeholder))
train_op = tf.train.AdamOptimizer(learning_rate=0.001).minimize(loss_op)
saver = tf.train.Saver()
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for epoch_num in range(100):
_, loss = sess.run([train_op, loss_op], feed_dict={input_placeholder: input_data, label_placeholder: label_data})
print('epoch ' + str(epoch_num) + ', loss = ' + str(loss))
saver.save(sess, '../Models/model', global_step=epoch_num + 1)
And below is my Python file to restore the network. It loads the input and output placeholders, together with the operation required for making predictions. However, even though I have named an operation as prediction in the training code above, the code below cannot seem to find this operation in the loaded graph.
import tensorflow as tf
import numpy as np
input_data = np.zeros([100, 10])
for i in range(100):
for j in range(10):
input_data[i, j] = i * j / 1000
with tf.Session() as sess:
saver = tf.train.import_meta_graph('../Models/model-99.meta')
saver.restore(sess, '../Models/model-99')
graph = tf.get_default_graph()
input_placeholder = graph.get_tensor_by_name('input_placeholder:0')
label_placeholder = graph.get_tensor_by_name('label_placeholder:0')
prediction = graph.get_operation_by_name('prediction')
pred = sess.run([prediction], feed_dict={input_placeholder: input_data})
Why can this code not find this operation, and what should I do to correct my code?
You have to modify a single line in your loading script (tested with tf 1.8):
prediction = graph.get_tensor_by_name('prediction/BiasAdd:0')
You have to specify which tensor you want to access, as prediction is only the namespace for the dense layer. You can check the exact name during saving with prediction.name. And when restoring, use tf.get_tensor_by_name as you are interested in the value, not the operation producing it.
I'm new to tensorboard. I have faced some problem while using it.
Problem 1 :
I'm writing an adversarial learning model. For visualizing the loss of the model I have the following loss,
actor loss
critic loss
for the learning algorithm provided in this paper,
in one(or K) batch I have to feed actor and critic both. Then I need to only feed value to the critic. This time there is no actor. I think, to show value in tensorboard I need to do following,
def model():
...
actor_loss = ...
tf.summary.scalar('actor', actor_loss)
...
critic_loss = ...
tf.summary.scalar('critic', critic_loss)
my_graph = tf.Graph()
with my_graph.as_default():
tf.reset_default_graph()
sess = tf.Session()
with sess.as_default():
model()
merged = tf.summary.merge_all()
writer = tf.summary.FileWriter(address+ '/train',
sess.graph)
init = tf.global_variables_initializer()
sess.run(init)
Now while giving input to inner_loop (where actor and critic both participated) there's no problem, we get the result by following,
a,b,c,d,summary = sess.run( [actor_train_step, critic_train_step, actor_loss, critic_loss, merged], feed_dict = feed_dict )
writer.add_summary(summary, batch)
but when we want to give input only to the critic, then the code becomes following,
a,b,summary = sess.run( [critic_train_step, critic_loss, merged], feed_dict = feed_dict )
writer.add_summary(summary, batch)
But as merged have dependency over actor_loss it cannot run. On the other side, I can't just feed value to of actor to the model. How how to solve this issue?
Problem 2
I am not evaluating (calculating the score value) the model by tensor operation. Actually, I generate the output and fed the output to another script and got the score value from there. So after each of the batch/epoch I am evaluating my model and got a score value from the script. How can I save this value to tensorboard?
I can not formalize a tf.summary.merge_all() operation before the session initialization as I am calculating the evaluation score value at the training time from outside script.
Where should I put the tf.summary.merge_all() operation?
Now if I want to combine the Problem 1 and Problem 2 to in a single project is there anything new I have to do.
Note: I'm new to tensorboard. So it will be better if you can give a detailed explanation.
Problem #1
If you only want to summary only the critic op, you should only run the summary op for the critic part instead of using tf.summary.merge_all()
For example:
def model():
...
actor_loss = ...
tf.summary.scalar('actor', actor_loss)
...
critic_loss = ...
summary_critic = tf.summary.scalar('critic', critic_loss)
a,b,summary = sess.run( [critic_train_step, critic_loss, summary_critic], feed_dict = feed_dict )
writer.add_summary(summary, batch)
Problem #2
To visualize the values you got after running the outside script. You can convert those values to tensor using tf.convert_to_tensor(), which is documented here. Then serializing that tensor to visualize it on tensorboard.
For example:
vals = output_from_outside_script()
vals_tensor = sess.run(tf.convert_to_tensor(vals))
tf.summary.scalar('evaluation', vals_tensor)
Every tf.summary operations will create a Summary protobuf which serializing your tensor to an events file. And instead of running all the summary ops, Tensorflow provide tf.summary.merger_all() to run all the summary ops in your graph.
I tried to do it in your case.
Outside script:
import numpy as np
def output_from_outside_script(var):
return np.sum(var)
Code in adversarial training:
import tensorflow as tf
import numpy as np
from outside_evaluation import *
sess = tf.Session()
x = sess.run(tf.constant([[1,2,3,4]], dtype=tf.float32))
X = tf.placeholder(dtype=tf.float32, shape=[1, 4])
W = tf.Variable(tf.truncated_normal([4, 10], stddev=0.1))
sess.run(tf.global_variables_initializer())
val = tf.matmul(a=X, b=W, name='matmul')
tf.summary.scalar('matmul_mean', tf.reduce_mean(val))
y = sess.run(val, feed_dict={X: x})
print('y = ', y)
vals = output_from_outside_script(y)
print('vals = ', vals)
vals_tensor = tf.convert_to_tensor(vals, name='vals_tensor')
tf.summary.scalar('evaluation', vals_tensor)
writer = tf.summary.FileWriter(os.path.join('test_log'), sess.graph)
merged = tf.summary.merge_all()
summary = sess.run(merged, feed_dict={X: x})
writer.add_summary(summary)
writer.close()
Output:
('y = ', array([[-0.51137048, -0.16054343, -0.03827953, 0.1124011 , 0.09200752,
-0.22235785, 0.41357356, 1.04061067, -0.08877556, -0.86647421]],
('vals = ', -0.22920817)
Tensorboard log:
Scalar:
Should there be any problem, please let me know.
I need help in luanching tensorboard from tensorflow running on the datalab,
My code is the followings (everything is on the datalab):
import tensorflow as tf
with tf.name_scope('input'):
print ("X_np")
X_np = tf.placeholder(tf.float32, shape=[None, num_of_features],name="input")
with tf.name_scope('weights'):
print ("W is for weights & - 15 number of diseases")
W = tf.Variable(tf.zeros([num_of_features,15]),name="W")
with tf.name_scope('biases'):
print ("b")
#todo:authemate for more diseases
b = tf.Variable(tf.zeros([15]),name="biases")
with tf.name_scope('layer'):
print ("y_train_np")
y_train_np = tf.nn.softmax(tf.matmul(X_np,W) + b)
with tf.name_scope('correct'):
print ("y_ - placeholder for correct answer")
y_ = tf.placeholder(tf.float32, shape=[None, 15],name="correct_answer")
with tf.name_scope('loss'):
print ("cross entrpy")
cross_entropy = -tf.reduce_sum(y_*tf.log(y_train_np))
# % of correct answers found in batch
print("is correct")
is_correct = tf.equal(tf.argmax(y_train_np,1),tf.argmax(y_,1))
print("accuracy")
accuracy = tf.reduce_mean(tf.cast(is_correct,tf.float32))
print("train step")
train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cross_entropy)
# train data and get results for batches
print("initialize all varaible")
init = tf.global_variables_initializer()
print("session")
sess = tf.Session()
writer = tf.summary.FileWriter("logs/", sess.graph)
init = tf.global_variables_initializer()
sess.run(init)
!tensorboard --logdir=/logs
the output is:
Starting TensorBoard 41 on port 6006
(You can navigate to http://172.17.0.2:6006)
However, when I click on the link, the webpage is empty
Please let me know what I am missing. I am expecting to see the graph. later i would like to generate more data. Any suggestion is appreciated.
Many thanks!
If you are using datalab, you can use tensorboard as below:
from google.datalab.ml import TensorBoard as tb
tb.start('./logs')
http://googledatalab.github.io/pydatalab/google.datalab.ml.html
You can also create a Cloud AI Platform Notebook instance with TensorBoard support by entering the following command into the Cloud Shell. Afterwards you can simply launch tensorboard when you want from launcher (File->New Launcher-> Tensorboard)
export IMAGE_FAMILY="tf-1-14-cpu"
export ZONE="us-west1-b"
export INSTANCE_NAME="tf-tensorboard-1"
export INSTANCE_TYPE="n1-standard-4"
gcloud compute instances create "${INSTANCE_NAME}" \
--zone="${ZONE}" \
--image-family="${IMAGE_FAMILY}" \
--image-project=deeplearning-platform-release \
--machine-type="${INSTANCE_TYPE}" \
--boot-disk-size=200GB \
--scopes=https://www.googleapis.com/auth/cloud-platform \
--metadata="proxy-mode=project_editors
I just started to use Tensorflow in python for optimisation problems. And I just gave it a try with really simple regression model. But the results (both slope and constant) I obtain seemed to be quite far off from what I expect, can anyone point out what I have done wrong (the code runs, but I am not sure if I use Tensorflow properly).
What I did:
1 import module:
import matplotlib.pyplot as plt
import numpy as np
import random as ran
import tensorflow as tf
2 create data based on a linear structure (y = 3 X + 4 + error):
train_X = np.array(range(-20,20,1))
b = 3; c = 4; sd = 0.5;
error = np.random.normal(loc=0.0, scale=sd, size=40);
deterministic = b* train_X + c;
train_Y = np.add(deter,error)
3 setting up for optimisation:
X = tf.placeholder(tf.float32,[40])
Y = tf.placeholder(tf.float32,[40])
beta = tf.Variable(np.random.randn(), name="beta")
alpha = tf.Variable(np.random.randn(), name="alpha")
n_samples = 40
learning_rate = 0.01
pred_full = tf.add(tf.scalar_mul(beta, X),alpha)
cost = tf.reduce_mean(tf.pow(tf.subtract(Y, pred_full),2))
optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(cost)
4 Running it:
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
sess.run(optimizer, feed_dict={X: train_X, Y: train_Y})
result = sess.run(cost, feed_dict={X: train_X, Y: train_Y})
result_beta = sess.run(beta, feed_dict={X: train_X, Y: train_Y})
result_alpha = sess.run(alpha, feed_dict={X: train_X, Y: train_Y})
print('result:', result, ';', 'result_beta:', result_beta, ';', 'result_alpha:',result_alpha)
The result I obtained is:
result: 1912.99 ; result_beta: 6.75786 ; result_alpha: -0.209623
obviously beta is supposed to be close to 3 and alhpa should be close to 4. I am wondering what went wrong in my code?
Thanks
You have to call the optimizer multiple times for multiple iterations of gradient descent. As, #dv3 noted, try
init = tf.global_variables_initializer()
with tf.Session() as sess:
sess.run(init)
for i in range(50):
opt, result_alpha, result_beta = sess.run([optimizer, alpha, beta], feed_dict={X: train_X, Y: train_Y})
print('beta =', result_beta, 'alpha =', result_alpha)
NB: It isn't necessary to access multiple tensor values each with individual calls to run(). You can do that with a list of values to fetch.
I am experimenting (learning) TensorBoard and use the following code I got from the internet (simple regression function)
import tensorflow as tf
import numpy as np
#sess = tf.InteractiveSession() #define a session
# Create 100 phony x, y data points in NumPy, y = x * 0.1 + 0.3
x_data = np.random.rand(100).astype("float32")
y_data = x_data * 0.1 + 0.3
# Try to find values for W and b that compute y_data = W * x_data + b
# (We know that W should be 0.1 and b 0.3, but Tensorflow will
# figure that out for us.)
with tf.name_scope("calculatematmul") as scope:
W = tf.Variable(tf.random_uniform([1], -1.0, 1.0))
b = tf.Variable(tf.zeros([1]))
y = W * x_data + b
# Minimize the mean squared errors.
loss = tf.reduce_mean(tf.square(y - y_data))
optimizer = tf.train.GradientDescentOptimizer(0.5)
train = optimizer.minimize(loss)
# Before starting, initialize the variables. We will 'run' this first.
init = tf.initialize_all_variables()
# Launch the graph.
sess = tf.Session()
sess.run(init)
#### ----> ADD THIS LINE <---- ####
writer = tf.train.SummaryWriter('mnist_logs', sess.graph_def)
# Fit the line.
for step in xrange(201):
sess.run(train)
if step % 20 == 0:
print(step, sess.run(W), sess.run(b))
The code runs fine when I create a python file and run the file with
python test.py
also it runs fine in the jupyter notebook
However, while Tensorboard gets the information from running the python file (that is to say, it creates the xyz....home file), the interactive version does not create any info usable for Tensorboard.
Can somebody explain to me why, please!
Thanks
Peter
Be sure that you use the full path when starting tensorboard.
tensorboard --logdir='./somedirectory/mnist_logs'