How to explain the result of this tensorflow program? - tensorflow

Question

import tensorflow as tf
x = tf.Variable(1)
x = x + 1
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
print(sess.run(x))
print(sess.run(x))
The output is
2
2
But I think the output should be
2
3
The first run, x has been update to 2, and the second run, x should be 3.
Who can tell me why the second run of x is 2 either? If x can't be update by the first run, how is the parameter of neural network update?
Update
x = tf.Variable(1)
x.assign(x+1)
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
#print(x)
print(sess.run(x))
print(sess.run(x))
The output is
1
1
It is amazing.

Answer 1

Here's an analysis of your and Ishant Mrinal examples, it should help you understand what's going on here.
Example 1
x = tf.Variable(1)
Creation of a python variable x. Creation of a Tensorflow node variable_1. The python variable x holds a logical pointer to the node variable_1.
x = x + 1
Python assignment, destructive operation.
x now holds a pointer to the operation sum(variable_1, constant(1)).
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
usual code of initialization of variables.
print(sess.run(x))
Execution of x = execution of sum(variable_1, constant(1)) = 2
print(sess.run(x))
Execution of x = execution of sum(variable_1, constant(1)) = 2
Example 2
x = tf.Variable(1)
Creation of a python variable x. Creation of a Tensorflow node variable_1. The python variable x holds a logical pointer to the node variable_1.
init = tf.global_variables_initializer()
initialization of variable_1.
with tf.Session() as sess:
init.run()
execution of the initialization.
# here x point to the variable
print(sess.run(x))
evaluation of x = 1.
x = x + 1
Definition of a new node, exactly as in the previous example.
print(sess.run(x))
evaluation of the sum operation, thus 2.
Example 3
x = tf.Variable(1)
usual creation.
as_op = x.assign(x+1)
definition of a sum node followed by the definition of an assignment node, held by the python variable as_op.
This operation forces the order of execution between these 2 nodes. First executes the sum node, then use the result to assign it to the node variable variable_1.
with tf.Session() as sess:
init.run()
usual init ops
# here x point to the variable
print(sess.run(x))
evaluation of variable_1, thus 1.
sess.run(as_op)
execution of sum and assigment, thus temp = variable_1 + 1; variable_1 = temp;
print(sess.run(x))
extraction of the value pointed to x, thus 2.

Answer 2

tensorflow add op returns a updated tensor
x = x + 1
with tf.Session() as sess:
init.run()
print(x)
# here x is a addition op; hence the result same for both run
> result Tensor("add:0", shape=(), dtype=int32)
for both run x is just the same add op
print(sess.run(x)) # result: 2
print(sess.run(x)) # result: 2
if you change the location of the addition op then values will be different; since you will accessing initial value and updated value from the add op
x = tf.Variable(1)
init = tf.global_variables_initializer()
with tf.Session() as sess:
init.run()
# here x point to the variable
print(sess.run(x)) # result: 1
x = x + 1
# here x point to the add op
print(sess.run(x)) # result: 2
Using assign op to get the updated value
x = tf.Variable(1)
as_op = x.assign(x+1)
with tf.Session() as sess:
init.run()
# here x point to the variable
print(sess.run(x)) # result: 1
sess.run(as_op) # variable updated
print(sess.run(x)) # result: 2