What's the difference between tf.cond and if-else? - tensorflow

Question

What difference between tf.cond and if-else?
Scenario 1
import tensorflow as tf
x = 'x'
y = tf.cond(tf.equal(x, 'x'), lambda: 1, lambda: 0)
with tf.Session() as sess:
print(sess.run(y))
x = 'y'
with tf.Session() as sess:
print(sess.run(y))
Scenario 2
import tensorflow as tf
x = tf.Variable('x')
y = tf.cond(tf.equal(x, 'x'), lambda: 1, lambda: 0)
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
print(sess.run(y))
tf.assign(x, 'y')
with tf.Session() as sess:
init.run()
print(sess.run(y))
The outputs are both 1.
Does it mean only tf.placeholder can work, and not all the tensor, such as tf.variable? When should I choose if-else condition and when to use tf.cond? What are the diffences between them?

Answer 1

tf.cond is evaluated at the runtime, whereas if-else is evaluated at the graph construction time.
If you want to evaluate your condition depending on the value of the tensor at the runtime, tf.cond is the best option.

Answer 2

Did you mean if ... else in Python vs. tf.cond?
You can use if ... else for creating different graph for different external conditions. For example you can make one python script for graphs with 1, 2, 3 hidden layers, and use command line parameters for select which one use.
tf.cond is for add condition block to the graph. For example, you can define Huber function by code like this:
import tensorflow as tf
delta = tf.constant(1.)
x = tf.placeholder(tf.float32, shape=())
def left(x):
return tf.multiply(x, x) / 2.
def right(x):
return tf.multiply(delta, tf.abs(x) - delta / 2.)
hubber = tf.cond(tf.abs(x) <= delta, lambda: left(x), lambda: right(x))
and calculation in Graph will go by different branch for different input data.
sess = tf.Session()
with sess.as_default():
sess.run(tf.global_variables_initializer())
print(sess.run(hubber, feed_dict = {x: 0.5}))
print(sess.run(hubber, feed_dict = {x: 1.0}))
print(sess.run(hubber, feed_dict = {x: 2.0}))
> 0.125
> 0.5
> 1.5

Answer 3

Since the graph in TensorFlow is static, you cannot modify it once built. Thus you can use if-else outside of the graph at anytime for example while preparing batches and etc., but you can also employ it while constructing the graph. That is, if the condition doesn't depend on the value of any tensor, for example the dimention(having been set) of the tensor or the shape of any tensor. In such scenarios the graph will not be changed due to the condition while excuting the graph. The graph has been fixed after you finished drawing the graph and the if-else condition would not affect the graph while excuting the graph.
But if the condition depends on the value of the tensor in it that condition should be included in the graph and hence tf.cond should be applied.

Answer 4

Simply put: if else is how you do switch in Python, while tf.cond is how you do switch in Tensorflow. During running, if else is fixed in the compiled Python program, while tf.cond is fixed in the constructed Tensorflow graph.
You can think of tf.cond as the Tensorflow's internal way of doing if else.