I have checked the website but as always it was not clear for me. Can anyone describes all of the steps (from very beginning) to run any tensorflow program on GPU's?
From Tensorflow official site:
https://www.tensorflow.org/tutorials/using_gpu
# Creates a graph.
c = []
for d in ['/device:GPU:2', '/device:GPU:3']:
with tf.device(d):
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3])
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2])
c.append(tf.matmul(a, b))
with tf.device('/cpu:0'):
sum = tf.add_n(c)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print(sess.run(sum))
Related
Sorry if this is something it was just asked, but I searched for it without success. I am thinking about to apply for the tensorflow certificate exam. My first question is if, during the exam, the custom activation functions are allowed.
For example: Imagine a question about a regression when the data is:
features = np.array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0], dtype=float)
targets = np.array([0.0, 1.0, 4.0, 9.0, 16.0, 25.0, 36.0], dtype=float)
This is clearly a x^2 problem. Could I do something like this?
tf_lpow = lambda x: tf.math.pow(x, 2)
model = tf.keras.models.Sequential([
tf.keras.layers.Dense(units=1, activation=tf_lpow, input_shape=(1,)),
])
Considering that maybe this could not be allowed, I was thinking about another solution:
lr_scheduler = ReduceLROnPlateau(monitor='loss', factor=0.75, patience=50, min_lr=3e-80)
callbacks = [lr_scheduler]
model = tf.keras.models.Sequential([
tf.keras.layers.Dense(units=6, activation='sigmoid', kernel_regularizer=tf.keras.regularizers.l2(0.01), kernel_initializer=tf.keras.initializers.RandomNormal(stddev=0.01), input_shape=(1,)),
tf.keras.layers.Dense(units=1, activation='linear')
])
But even in the case the loss is decreasing, the accuracy is stack in 0.2857, not reaching the goal. In this case, what could I do?
Thanks in advance.
I am trying to run a simple program on TPU:
import tensorflow as tf
tpu = tf.distribute.cluster_resolver.TPUClusterResolver()
print("Device:", tpu.master())
tf.config.experimental_connect_to_cluster(tpu)
tf.tpu.experimental.initialize_tpu_system(tpu)
strategy = tf.distribute.experimental.TPUStrategy(tpu)
a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
with strategy.scope():
c = tf.matmul(a, b)
print("c device: ", c.device)
with tf.Session(config=tf.ConfigProto(log_device_placement=True)) as sess:
print(c.eval())
When I run this, it looks like the TPU is being found. However, none of the logged devices have 'TPU' in the name -- it is all on the CPU.
What am I doing wrong?
strategy.scope() is for model training.
If you want to run tf.matmul on a TPU you could use either this:
with tf.device('/TPU:0'):
c = tf.matmul(a, b)
Or
#tf.function
def matmul_fn(x, y):
z = tf.matmul(x, y)
return z
z = strategy.run(matmul_fn, args=(a, b))
print(z)
Details are here.
model = tf.keras.Sequential([
tf.keras.layers.Dense(units=1, input_shape=[1])
])
model.compile(optimizer='sgd', loss='mean_squared_error')
xs = np.array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0], dtype=float)
ys = np.array([.5, 1.0, 1.5, 2.0, 2.5, 3.0, 3.5], dtype=float)
model.fit(xs, ys, epochs=1000)
return (model.predict(y_new))
this code giving error:
type numpy.ndarray does not define round method in model.predict()
Link for the tf tutorial
# Creates a graph.
with tf.device('/cpu:0'):
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print sess.run(c)
In above example cpu:0 has assigned to the execution process. With the log_device_placement true. So this is solution for the above code that the have mentioned
Device mapping:
/job:localhost/replica:0/task:0/gpu:0 -> device: 0, name: Tesla K40c, pci bus
id: 0000:05:00.0
b: /job:localhost/replica:0/task:0/cpu:0
a: /job:localhost/replica:0/task:0/cpu:0
MatMul: /job:localhost/replica:0/task:0/gpu:0
[[ 22. 28.]
[ 49. 64.]]
Now here place holders a, b and the matmul operation c which runs inside a session is inside the device log cpu:o but in the log device description why only MatMul has been executed in gpu:0 ?
That seems like a bug in the documentation, the MatMul operation will be placed on CPU in this case.
Indeed, running the code sample does show this:
import tensorflow as tf
# Creates a graph.
with tf.device('/cpu:0'):
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print sess.run(c)
# And to prove that GPUs exist and can be used
with tf.device('/gpu:0'):
d = tf.random_normal([])
print sess.run(d)
Will show the following placements:
MatMul: (MatMul): /job:localhost/replica:0/task:0/cpu:0
b: (Const): /job:localhost/replica:0/task:0/cpu:0
a: (Const): /job:localhost/replica:0/task:0/cpu:0
random_normal/RandomStandardNormal: (RandomStandardNormal): /job:localhost/replica:0/task:0/gpu:0
I think the documentation bug is that the c = tf.matmul(a, b) statement was supposed to be outside the with tf.device('/cpu:0') scope.
How does it start the device in tensorflow？And where can I found the details in source code. I am searching for a long time on net. But no use. Please help or try to give some ideas how to achieve this.
You can assign a task to a device like this:
# Creates a graph.
with tf.device('/cpu:0'):
a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')
c = tf.matmul(a, b)
# Creates a session with log_device_placement set to True.
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
# Runs the op.
print sess.run(c)
taken from here