I am able to perform addition using constant but not able to perform through tf.Variable
Below code is working fine when I am using constant for addition.
import tensorflow as tf
a = tf.constant(5)
b = tf.constant(6)
sess = tf.Session()
result = sess.run(a + b)
print(result)
But when I tried with tf.Variable it is not working, Here is mine code
import tensorflow as tf
a = tf.Variable(5)
b = tf.Variable(6)
sess = tf.Session()
result = sess.run(a + b)
print(result)
Can somebody help me ? Thanks !!!
You need to initialize the variables firstly:
import tensorflow as tf
a = tf.Variable(5)
b = tf.Variable(6)
sess = tf.Session()
Initialize variables:
sess.run(tf.global_variables_initializer())
result = sess.run(a + b)
print(result)
11
You can read more about variable initialization here, which says Unlike tf.Tensor objects, a tf.Variable exists outside the context of a single session.run call. So Before you can use a variable, it must be initialized. The initialization is Session specific, which means whenever you start a new session, and want to use these variables, you'll have to initialize them firstly.
Related
I recently found a paper where they used a CNN with complex 2D-feature-maps as an input. However, there Network also outputs a complex output vector. They used Keras with tensorflow backend.
Here is the link: https://arxiv.org/pdf/1802.04479.pdf
I asked myself if it is possible to build complex Deep Neural Networks like CNNs with tensorflow. As far as i know it is not possible. Did i missed something?
There are other related questions which adresses the same problem with no answer: Complex convolution in tensorflow
when building a realy senseless model with real number in and output all works correct:
import tensorflow as tf
from numpy import random, empty
n = 10
feature_vec_real = random.rand(1,n)
X_real = tf.placeholder(tf.float64,feature_vec_real.shape)
def model(x):
out = tf.layers.dense(
inputs=x,
units=2
)
return out
model_output = model(X_real)
sess = tf.Session()
init = tf.global_variables_initializer()
sess.run(init)
output = sess.run(model_output,feed_dict={X_real:feature_vec_real})
but when using complex inputs:
import tensorflow as tf
from numpy import random, empty
n = 10
feature_vec_complex = empty(shape=(1,n),dtype=complex)
feature_vec_complex.real = random.rand(1,n)
feature_vec_complex.imag = random.rand(1,n)
X_complex = tf.placeholder(tf.complex128,feature_vec_complex.shape)
def complex_model(x):
out = tf.layers.dense(
inputs=x,
units=2
)
return out
model_output = complex_model(X_complex)
sess = tf.Session()
init = tf.global_variables_initializer()
sess.run(init)
output = sess.run(model_output,feed_dict={X_complex:feature_vec_complex})
i get the following error:
ValueError: An initializer for variable dense_7/kernel of is required
So what is the correct way to initialize the weights of the dense kernel when having complex inputs?
I know there is the possibility to handle complex numbers as two different layers in the network. But this is not what i want.
Thanks for your help!
I would like to show my example below:
x = tf.placeholder(dtype=...)
a = numpy.asarray([784, 10])
z = slim.fully_connected(x, 10, weights_initializer=?)
I have tried weights_initializer = lambda x1:a, it reports the error: TypeError: () got an unexpected keyword argument 'dtype'
I also found another post here:https://github.com/tensorflow/tensorflow/issues/4016
However, I still don't know the answer. Thank you very much.
Sorry, I don't really understand what you're trying to do.
If your fully connected layer has 10 hidden neurons then your initializer must have the shape (input_shape, 10), what you're giving is a (2,) shape. Secondly, to initialize weights with a constant matrix you should use tf.constant_initializer(..) function.
Are you trying to do the following: (you can change the init function used with numpy)
import tensorflow as tf
import numpy as np
slim = tf.contrib.slim
input_size = ?
x = tf.placeholder(dtype=tf.float32, shape=[input_size])
a = np.random.normal((input_size, 10))
z = slim.fully_connected(x, 10,
weights_initializer=tf.constant_initializer(a))
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
I need to loop in graph for different times. I am unable to pass the integer variable through feed_dict.
rough1 = tf.Graph()
with rough1.as_default():
st = tf.placeholder(tf.float32,shape = ())
d = tf.Variable(0)
for i in range(st):
d = tf.add(d,1)
with tf.Session(graph = rough1) as sess:
sess.run(tf.initialize_all_variables())
s = sess.run([d], feed_dict={st:3})
print s
When you build a Tensorflow graph, the graph isn't actually evaluated until you call tf.Session.run. A consequence of this is that Python constructs like range cannot inspect the value of a Tensor except by calling run(). Here, range(st) requires that the value of st be an integer known to Python, but the value of st is not known until the graph actually executes.
Put another way: the code that constructs the graph must not depend on the evaluation of that graph.
But here, you can just use a Python integer rather than a Tensor and everything works fine:
rough1 = tf.Graph()
with rough1.as_default():
d = tf.Variable(0)
for i in range(3):
d = tf.add(d,1)
with tf.Session(graph = rough1) as sess:
sess.run(tf.initialize_all_variables())
s = sess.run([d])
print s
Hope that helps!
Try this and see if it is what you are looking for.
st=1
rough1 = tf.Graph()
with rough1.as_default():
d = tf.Variable(0)
for i in range(st):
d = tf.add(d,1)
with tf.Session(graph = rough1) as sess:
sess.run(tf.initialize_all_variables())
s = sess.run([d])
print s
I'm having trouble using the Defun decorator in tensorflow. Namely, Defun can't close over any TF ops created outside. Below is a self-contained example showing what I'd like to do. Note that the tensor x belongs to different graphs inside and outside the call to custom_op. The Defun code creates a temporary graph, translates the graph into a function proto, and then merges this into the original graph. The code crashes in the first step, since the tensors that we close over are not in the new temporary graph. Is there a way around this? Being able to close over things would be very helpful.
import tensorflow as tf
from tensorflow.python.framework import function
w = tf.Variable(1.0)
function_factory = lambda x: x*w
#function.Defun(x=tf.float32)
def custom_op(x):
print('graph for x inside custom_op: ', x.graph)
return function_factory(x)
x = tf.constant(2.0)
print('graph for x outside custom_op: ', x.graph)
y = custom_op(x)
with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
sess.run(y)
No, the Defun decorator does not capture everything. You need to pass in the w explicitly, as following in your example:
import tensorflow as tf
from tensorflow.python.framework import function
w = tf.Variable(1.0)
#function.Defun(tf.float32, tf.float32)
def custom_op(x, w):
print('graph for x inside custom_op: ', x.graph)
return x * w
x = tf.constant(2.0)
print('graph for x outside custom_op: ', x.graph)
y = custom_op(x, tf.identity(w))
with tf.Session() as sess:
sess.run(tf.initialize_all_variables())
sess.run(y)
(we may add more complete support for capturing if the need is high.)
I am trying to initialize a tensorflow Variable with pre-trained word2vec embeddings.
I have the following code:
import tensorflow as tf
from gensim import models
model = models.Word2Vec.load_word2vec_format('GoogleNews-vectors-negative300.bin', binary=True)
X = model.syn0
embeddings = tf.Variable(tf.random_uniform(X.shape, minval=-0.1, maxval=0.1), trainable=False)
sess.run(tf.initialize_all_variables())
sess.run(embeddings.assign(X))
And I am receiving the following error:
ValueError: Cannot create an Operation with a NodeDef larger than 2GB.
The array (X) I am trying to assign is of shape (3000000, 300) and its size is 3.6GB.
I am getting the same error if I try tf.convert_to_tensor(X) as well.
I know that it fails due to the fact that the array is larger than 2GB. However, I do not know how to assign an array larger than 2GB to a tensorflow Variable
It seems like the only option is to use a placeholder. The cleanest way I can find is to initialize to a placeholder directly:
X_init = tf.placeholder(tf.float32, shape=(3000000, 300))
X = tf.Variable(X_init)
# The rest of the setup...
sess.run(tf.initialize_all_variables(), feed_dict={X_init: model.syn0})
The easiest solution is to feed_dict'ing it into a placeholder node that you use to tf.assign to the variable.
X = tf.Variable([0.0])
place = tf.placeholder(tf.float32, shape=(3000000, 300))
set_x = X.assign(place)
# set up your session here....
sess.run(set_x, feed_dict={place: model.syn0})
As Joshua Little noted in a separate answer, you can also use it in the initializer:
X = tf.Variable(place) # place as defined above
...
init = tf.initialize_all_variables()
... create sess ...
sess.run(init, feed_dict={place: model.syn0})
try this:
import tensorflow as tf
from gensim import models
model = models.KeyedVectors.load_word2vec_format('./GoogleNews-vectors-negative300.bin', binary=True)
X = model.syn0
embeddings = tf.Variable(tf.random_uniform(X.shape, minval=-0.1, maxval=0.1), trainable=False)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
embeddings.load(model.syn0, sess)