What's wrong with `tf.zeros_initializer` in `get_variable()`? - tensorflow

I want to train some adversarial examples using CW algorithm, and I used an example from here and a CW implementation from here. But I encountered an error about tf.zeros_initializer:
ValueError: The initializer passed is not valid. It should be a callable with no arguments and the shape should not be provided or an instance of
'tf.keras.initializers.*' and `shape` should be fully defined.
Edit: It seems that non-fully defined shape conflicts with using initializers. How can I fix it?
Here's a piece of code:
# ... omitted
with tf.variable_scope('model', reuse=tf.AUTO_REUSE):
# CW
_, env.adv_cw, _ = cw.cw(model, env.x)
Here's env.x:
env.x = tf.placeholder(tf.float32, (None, width, height, channels), name='x')
When I run the code, I get the error message:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-39-712c8b007d37> in <module>()
8 with tf.variable_scope('model', reuse=tf.AUTO_REUSE):
9 # CW
---> 10 _, env.adv_cw, _ = cw.cw(model, env.x)
5 frames
/content/cw.py in cw(model, x, y, eps, ord_, T, optimizer, alpha, min_prob, clip)
50 """
51 xshape = x.get_shape().as_list()
---> 52 noise = tf.get_variable('noise', shape=xshape, dtype=tf.float32,
53 initializer=tf.zeros_initializer)
54
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variable_scope.py in get_variable(name, shape, dtype, initializer, regularizer, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint, synchronization, aggregation)
1494 constraint=constraint,
1495 synchronization=synchronization,
-> 1496 aggregation=aggregation)
1497
1498
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variable_scope.py in get_variable(self, var_store, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint, synchronization, aggregation)
1237 constraint=constraint,
1238 synchronization=synchronization,
-> 1239 aggregation=aggregation)
1240
1241 def _get_partitioned_variable(self,
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variable_scope.py in get_variable(self, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter, constraint, synchronization, aggregation)
560 constraint=constraint,
561 synchronization=synchronization,
--> 562 aggregation=aggregation)
563
564 def _get_partitioned_variable(self,
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variable_scope.py in _true_getter(name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, constraint, synchronization, aggregation)
512 constraint=constraint,
513 synchronization=synchronization,
--> 514 aggregation=aggregation)
515
516 synchronization, aggregation, trainable = (
/usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/variable_scope.py in _get_single_variable(self, name, shape, dtype, initializer, regularizer, partition_info, reuse, trainable, collections, caching_device, validate_shape, use_resource, constraint, synchronization, aggregation)
906 variable_dtype = None
907 else:
--> 908 raise ValueError("The initializer passed is not valid. It should "
909 "be a callable with no arguments and the "
910 "shape should not be provided or an instance of "
ValueError: The initializer passed is not valid. It should be a callable with no arguments and the shape should not be provided or an instance of `tf.keras.initializers.*' and `shape` should be fully defined.
But Google's TensorFlow Guide gives an example of the usage of get_variable:
my_int_variable = tf.get_variable("my_int_variable", [1, 2, 3], dtype=tf.int32,
initializer=tf.zeros_initializer)
Environment: Google Colab, TensorFlow 1.14.0-rc1, Python 3.6

Just make changes according to your placeholder dimension, let me take example of your placeholder variable.
** x = placeholder(t f. float 32, (None, width, height, channels), name='x')**.
It has 4 dimension : [None, width, height, channels], but width, height, channels is not defined, means, for an image width =6, height=6, channels=3 is defined so the tensor dimension is [6 x 6 x 3].
What you can do is, the image you read, get the all three dimension value in different variable and pass it to your placeholder variable.
Ex.
Image A = 32 x 32 x 3
width = A.shape[0]
height = A.shape[1]
channels =A.shape[2]
Or you can directly provide the value to width, height, channels (If you know your input data shape) by this way the placeholder will be defined.

Related

Using gen_nn_ops.max_pool_grad_v2 for layerwise relevance propagation not working: Keras symbolic input/output

I am trying to apply layerwise relevance propagation (LRP) following some examples and tutorials, in particular this one and this one. However, I am struggling to get the relevances for the max_pooling layers. In particular, the problem appears when using gen_nn_ops.max_pool_grad_v2:
gen_nn_ops.max_pool_grad_v2(orig_input=a, orig_output=z, grad=s,
ksize=ksize,
strides=[1, 2, 2, 1], padding='VALID')
where the inputs are:
a : <KerasTensor: shape=(None, 6, 8, 512) dtype=float32 (created by layer 'activation_23')>
z : <KerasTensor: shape=(None, 3, 4, 512) dtype=float32 (created by layer 'tf.compat.v1.nn.max_pool_1')>
s: <KerasTensor: shape=(None, 3, 4, 512) dtype=float32 (created by layer 'tf.math.truediv_23')>
The error I got is:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/scratch/local/26230143/ipykernel_15814/2576584237.py in <module>
----> 1 gen_nn_ops.max_pool_grad_v2(orig_input=a, orig_output=z, grad=s,
2 ksize=kk,
3 strides=[1, 2, 2, 1], padding='VALID')
~/.local/lib/python3.9/site-packages/tensorflow/python/ops/gen_nn_ops.py in max_pool_grad_v2(orig_input, orig_output, grad, ksize, strides, padding, data_format, name)
6251 pass
6252 try:
-> 6253 return max_pool_grad_v2_eager_fallback(
6254 orig_input, orig_output, grad, ksize, strides, padding=padding,
6255 data_format=data_format, name=name, ctx=_ctx)
~/.local/lib/python3.9/site-packages/tensorflow/python/ops/gen_nn_ops.py in max_pool_grad_v2_eager_fallback(orig_input, orig_output, grad, ksize, strides, padding, data_format, name, ctx)
6283 data_format = "NHWC"
6284 data_format = _execute.make_str(data_format, "data_format")
-> 6285 _attr_T, _inputs_T = _execute.args_to_matching_eager([orig_input, orig_output, grad], ctx, [_dtypes.float32, _dtypes.float64, _dtypes.int32, _dtypes.uint8, _dtypes.int16, _dtypes.int8, _dtypes.int64, _dtypes.bfloat16, _dtypes.uint16, _dtypes.half, _dtypes.uint32, _dtypes.uint64, ], _dtypes.float32)
6286 (orig_input, orig_output, grad) = _inputs_T
6287 ksize = _ops.convert_to_tensor(ksize, _dtypes.int32)
~/.local/lib/python3.9/site-packages/tensorflow/python/eager/execute.py in args_to_matching_eager(l, ctx, allowed_dtypes, default_dtype)
254 # not list allowed dtypes, in which case we should skip this.
255 if dtype is None and allowed_dtypes:
--> 256 tensor = ops.convert_to_tensor(t, ctx=ctx)
257 # If we did not match an allowed dtype, try again with the default
258 # dtype. This could be because we have an empty tensor and thus we
~/.local/lib/python3.9/site-packages/tensorflow/python/profiler/trace.py in wrapped(*args, **kwargs)
181 with Trace(trace_name, **trace_kwargs):
182 return func(*args, **kwargs)
--> 183 return func(*args, **kwargs)
184
185 return wrapped
~/.local/lib/python3.9/site-packages/tensorflow/python/framework/ops.py in convert_to_tensor(value, dtype, name, as_ref, preferred_dtype, dtype_hint, ctx, accepted_result_types)
1693
1694 if ret is None:
-> 1695 ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
1696
1697 if ret is NotImplemented:
~/.local/lib/python3.9/site-packages/tensorflow/python/framework/constant_op.py in _constant_tensor_conversion_function(v, dtype, name, as_ref)
341 as_ref=False):
342 _ = as_ref
--> 343 return constant(v, dtype=dtype, name=name)
344
345
~/.local/lib/python3.9/site-packages/tensorflow/python/framework/constant_op.py in constant(value, dtype, shape, name)
265 ValueError: if called on a symbolic tensor.
266 """
--> 267 return _constant_impl(value, dtype, shape, name, verify_shape=False,
268 allow_broadcast=True)
269
~/.local/lib/python3.9/site-packages/tensorflow/python/framework/constant_op.py in _constant_impl(value, dtype, shape, name, verify_shape, allow_broadcast)
277 with trace.Trace("tf.constant"):
278 return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
--> 279 return _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
280
281 g = ops.get_default_graph()
~/.local/lib/python3.9/site-packages/tensorflow/python/framework/constant_op.py in _constant_eager_impl(ctx, value, dtype, shape, verify_shape)
302 def _constant_eager_impl(ctx, value, dtype, shape, verify_shape):
303 """Creates a constant on the current device."""
--> 304 t = convert_to_eager_tensor(value, ctx, dtype)
305 if shape is None:
306 return t
~/.local/lib/python3.9/site-packages/tensorflow/python/framework/constant_op.py in convert_to_eager_tensor(value, ctx, dtype)
100 dtype = dtypes.as_dtype(dtype).as_datatype_enum
101 ctx.ensure_initialized()
--> 102 return ops.EagerTensor(value, ctx.device_name, dtype)
103
104
~/.local/lib/python3.9/site-packages/keras/engine/keras_tensor.py in __array__(self, dtype)
252
253 def __array__(self, dtype=None):
--> 254 raise TypeError(
255 f'You are passing {self}, an intermediate Keras symbolic input/output, '
256 'to a TF API that does not allow registering custom dispatchers, such '
TypeError: You are passing KerasTensor(type_spec=TensorSpec(shape=(None, 6, 8, 512),
dtype=tf.float32, name=None), name='activation_23/Relu:0', description="created by layer
'activation_23'"), an intermediate Keras symbolic input/output, to a TF API that does not allow
registering custom dispatchers, such as `tf.cond`, `tf.function`, gradient tapes, or
`tf.map_fn`. Keras Functional model construction only supports TF API calls that *do* support
dispatching, such as `tf.math.add` or `tf.reshape`. Other APIs cannot be called directly on
symbolic Kerasinputs/outputs. You can work around this limitation by putting the operation in a
custom Keras layer `call` and calling that layer on this symbolic input/output.
I am assuming that somehow a, z and s must be tf.Tensor (am I wrong)? (instead of KerasTensor) , but I don't really know and neither how to get around this issue. I read that using:
from tensorflow.python.framework.ops import disable_eager_execution
disable_eager_execution()
might work, but it is not working for me. I'd really appreaciate any idea, suggestion here!
Thank you very much in advance,

Adding custom base64-string conversion layer to existing Keras model

I am trying to configure a model that I previously trained to classify images in a such a way that it accepts images as base64-strings (instead of a NumPy array), converts them to a NumPy array and then performs the prediction. How do I add a layer on top of my regular input layer that accepts strings and outputs a NumPy array?
So I've already pre-trained a model that predicts images based on the ResNet architecture. Having looked at this and this answer, I am trying to create a Lambda layer that converts strings to RGB jpeg images. I have done this as shown in the sample code below:
image = tf.placeholder(shape=[], dtype=tf.string)
input_tensor = keras.layers.Input(shape = (1,), tensor = image, dtype=tf.string)
x = keras.layers.Lambda(lambda image: tf.image.decode_jpeg(image))(input_tensor)
output_tensor = model(x)
new_model = Model(input_tensor, output_tensor)
Where model() is the Keras keras.models.Model model that I have pre-trained.
I am expecting new_model() to be the new Keras model that has 1 extra layer on top of my previous model, which accepts base64-string and outputs a NumPy array into the next layer.
However, the third line of my code raises the following error:
TypeError: Input 'contents' of 'DecodeJpeg' Op has type float32 that does not match expected type of string.
My understanding of this is that the 'image' in the Lambda layer that uses the decode_jpeg() is a float32 instead of a string, which seems odd to me as I have set the dtype of both the placeholder as well as the Input layer to tf.string.
I have searched all over stackoverflow for this but can't find a solution for this error. It appears this question also has not been able to find a solution for this specific issue.
EDIT 1: corrected typo and added full error message
The full error message is show below:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
509 as_ref=input_arg.is_ref,
--> 510 preferred_dtype=default_dtype)
511 except TypeError as err:
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in internal_convert_to_tensor(value, dtype, name, as_ref, preferred_dtype, ctx)
1103 if ret is None:
-> 1104 ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
1105
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py in _TensorTensorConversionFunction(t, dtype, name, as_ref)
946 "Tensor conversion requested dtype %s for Tensor with dtype %s: %r" %
--> 947 (dtype.name, t.dtype.name, str(t)))
948 return t
ValueError: Tensor conversion requested dtype string for Tensor with dtype float32: 'Tensor("lambda_28/Placeholder:0", shape=(?, 1), dtype=float32)'
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
<ipython-input-47-5793b0703860> in <module>
1 image = tf.placeholder(shape=[], dtype=tf.string)
2 input_tensor = Input(shape = (1,), tensor = image, dtype=tf.string)
----> 3 x = Lambda(lambda image: tf.image.decode_jpeg(image))(input_tensor)
4 output_tensor = model(x)
5
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/keras/engine/base_layer.py in __call__(self, inputs, **kwargs)
472 if all([s is not None
473 for s in to_list(input_shape)]):
--> 474 output_shape = self.compute_output_shape(input_shape)
475 else:
476 if isinstance(input_shape, list):
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/keras/layers/core.py in compute_output_shape(self, input_shape)
650 else:
651 x = K.placeholder(shape=input_shape)
--> 652 x = self.call(x)
653 if isinstance(x, list):
654 return [K.int_shape(x_elem) for x_elem in x]
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/keras/layers/core.py in call(self, inputs, mask)
685 if has_arg(self.function, 'mask'):
686 arguments['mask'] = mask
--> 687 return self.function(inputs, **arguments)
688
689 def compute_mask(self, inputs, mask=None):
<ipython-input-47-5793b0703860> in <lambda>(image)
1 image = tf.placeholder(shape=[], dtype=tf.string)
2 input_tensor = Input(shape = (1,), tensor = image, dtype=tf.string)
----> 3 x = Lambda(lambda image: tf.image.decode_jpeg(image))(input_tensor)
4 output_tensor = model(x)
5
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/gen_image_ops.py in decode_jpeg(contents, channels, ratio, fancy_upscaling, try_recover_truncated, acceptable_fraction, dct_method, name)
946 try_recover_truncated=try_recover_truncated,
947 acceptable_fraction=acceptable_fraction, dct_method=dct_method,
--> 948 name=name)
949 _result = _op.outputs[:]
950 _inputs_flat = _op.inputs
~/anaconda3/envs/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py in _apply_op_helper(self, op_type_name, name, **keywords)
531 if input_arg.type != types_pb2.DT_INVALID:
532 raise TypeError("%s expected type of %s." %
--> 533 (prefix, dtypes.as_dtype(input_arg.type).name))
534 else:
535 # Update the maps with the default, if needed.
TypeError: Input 'contents' of 'DecodeJpeg' Op has type float32 that does not match expected type of string.

Initializing RNNCell

I'm having issues with using any child class of tensorflows RNNCell. According to the tensorflow source, the state for any RNNCell should be a tuple, however when I give it a tuple, it throws an error saying that it's trying to ask for the ndims attribute of the state tuple, which doesn't exist.
I am trying to create an LSTM where I can control each individual input.
This is the simplest code I can make, and even that has the same issue, so I hope that I am doing something wrong that can easily be fixed. This is the simple code:
lstm_layer = tf.contrib.rnn.LSTMCell(num_units = 64)
initial_state = lstm_layer.zero_state(batch_size=1,dtype=tf.float32)
initial_input = np.expand_dims(np.array([1,2,3,4,5,6,7,8]),0)
output_single, state_single = lstm_layer(inputs=initial_input,state=initial_state)
Here is the error I get:
AttributeError Traceback (most recent call last)
<ipython-input-22-1dcce10906e5> in <module>
2 initial_state = lstm_layer.zero_state(batch_size=1,dtype=tf.float32)
3 initial_input = np.expand_dims(np.array([1,2,3,4,5,6,7,8]),0)
----> 4 output_single, state_single = lstm_layer(inputs=initial_input,state=initial_state)
~/anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py in __call__(self, inputs, state, scope, *args, **kwargs)
369 # method. See the class docstring for more details.
370 return base_layer.Layer.__call__(self, inputs, state, scope=scope,
--> 371 *args, **kwargs)
372
373
~/anaconda3/lib/python3.7/site-packages/tensorflow/python/layers/base.py in __call__(self, inputs, *args, **kwargs)
528
529 # Actually call layer
--> 530 outputs = super(Layer, self).__call__(inputs, *args, **kwargs)
531
532 if not context.executing_eagerly():
~/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in __call__(self, inputs, *args, **kwargs)
536 if not self.built:
537 # Build layer if applicable (if the `build` method has been overridden).
--> 538 self._maybe_build(inputs)
539 # We must set self.built since user defined build functions are not
540 # constrained to set self.built.
~/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/base_layer.py in _maybe_build(self, inputs)
1589 # Check input assumptions set before layer building, e.g. input rank.
1590 input_spec.assert_input_compatibility(
-> 1591 self.input_spec, inputs, self.name)
1592 input_list = nest.flatten(inputs)
1593 if input_list and self._dtype is None:
~/anaconda3/lib/python3.7/site-packages/tensorflow/python/keras/engine/input_spec.py in assert_input_compatibility(input_spec, inputs, layer_name)
107 spec.min_ndim is not None or
108 spec.max_ndim is not None):
--> 109 if x.shape.ndims is None:
110 raise ValueError('Input ' + str(input_index) + ' of layer ' +
111 layer_name + ' is incompatible with the layer: '
AttributeError: 'tuple' object has no attribute 'ndims'
It looks as though there is some base method that gets called for all layers in the keras API, but it doesn't work with tuples. However, I find it strange that this would be an issue that nobody has ever come against before. So I hope its just me making a mistake
I figured out the issue. Tensorflow wasn't playing nicely with numpy in this case. Instead of
initial_input = np.expand_dims(np.array([1,2,3,4,5,6,7,8]),0)
I needed to give it
initial_input = tf.expand_dims(np.array([1,2,3,4,5,6,7,8],dtype=np.float32),0)

GridSearchCV: "TypeError: 'StratifiedKFold' object is not iterable"

I want to perform GridSearchCV in a RandomForestClassifier, but data is not balanced, so I use StratifiedKFold:
from sklearn.model_selection import StratifiedKFold
from sklearn.grid_search import GridSearchCV
from sklearn.ensemble import RandomForestClassifier
param_grid = {'n_estimators':[10, 30, 100, 300], "max_depth": [3, None],
"max_features": [1, 5, 10], "min_samples_leaf": [1, 10, 25, 50], "criterion": ["gini", "entropy"]}
rfc = RandomForestClassifier()
clf = GridSearchCV(rfc, param_grid=param_grid, cv=StratifiedKFold()).fit(X_train, y_train)
But I get an error:
TypeError Traceback (most recent call last)
<ipython-input-597-b08e92c33165> in <module>()
9 rfc = RandomForestClassifier()
10
---> 11 clf = GridSearchCV(rfc, param_grid=param_grid, cv=StratifiedKFold()).fit(X_train, y_train)
c:\python34\lib\site-packages\sklearn\grid_search.py in fit(self, X, y)
811
812 """
--> 813 return self._fit(X, y, ParameterGrid(self.param_grid))
c:\python34\lib\site-packages\sklearn\grid_search.py in _fit(self, X, y, parameter_iterable)
559 self.fit_params, return_parameters=True,
560 error_score=self.error_score)
--> 561 for parameters in parameter_iterable
562 for train, test in cv)
c:\python34\lib\site-packages\sklearn\externals\joblib\parallel.py in __call__(self, iterable)
756 # was dispatched. In particular this covers the edge
757 # case of Parallel used with an exhausted iterator.
--> 758 while self.dispatch_one_batch(iterator):
759 self._iterating = True
760 else:
c:\python34\lib\site-packages\sklearn\externals\joblib\parallel.py in dispatch_one_batch(self, iterator)
601
602 with self._lock:
--> 603 tasks = BatchedCalls(itertools.islice(iterator, batch_size))
604 if len(tasks) == 0:
605 # No more tasks available in the iterator: tell caller to stop.
c:\python34\lib\site-packages\sklearn\externals\joblib\parallel.py in __init__(self, iterator_slice)
125
126 def __init__(self, iterator_slice):
--> 127 self.items = list(iterator_slice)
128 self._size = len(self.items)
c:\python34\lib\site-packages\sklearn\grid_search.py in <genexpr>(.0)
560 error_score=self.error_score)
561 for parameters in parameter_iterable
--> 562 for train, test in cv)
563
564 # Out is a list of triplet: score, estimator, n_test_samples
TypeError: 'StratifiedKFold' object is not iterable
When I write cv=StratifiedKFold(y_train) I have ValueError: The number of folds must be of Integral type. But when I write `cv=5, it works.
I don't understand what is wrong with StratifiedKFold
I had exactly the same problem. The solution that worked for me is to replace:
from sklearn.grid_search import GridSearchCV
with
from sklearn.model_selection import GridSearchCV
Then it should work fine.
The problem here is an API change as mentioned in other answers, however the answers could be more explicit.
The cv parameter documentation states:
cv : int, cross-validation generator or an iterable, optional
Determines the cross-validation splitting strategy. Possible inputs
for cv are:
None, to use the default 3-fold cross-validation, integer,
to specify the number of folds.
An object to be used as a
cross-validation generator.
An iterable yielding train/test splits.
For integer/None inputs, if y is binary or multiclass, StratifiedKFold
used. If the estimator is a classifier or if y is neither binary nor
multiclass, KFold is used.
So, whatever the cross validation strategy used, all that is needed is to provide the generator using the function split, as suggested:
kfolds = StratifiedKFold(5)
clf = GridSearchCV(estimator, parameters, scoring=qwk, cv=kfolds.split(xtrain,ytrain))
clf.fit(xtrain, ytrain)
It seems that cv=StratifiedKFold()).fit(X_train, y_train) should be changed to cv=StratifiedKFold()).split(X_train, y_train).
The api changed in the latest version. You used to pass y and now you pass just the number when you create the stratifiedKFold object. You pass the y later.

Cannot do simple initialization of variable - 'data type not understood'

I was running through the CIFAR-10 tutorial on Tensorflow, but I cannot get any of my variable declarations to work. Even something simple as:
biases = tf.get_variable('biases', [64], tf.constant_initializer(0.0))
gives the error:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-23-86228512ca30> in <module>()
----> 1 biases = tf.get_variable('biases', [64], tf.constant_initializer(0.0))
/home/mmm/programs/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.pyc in get_variable(name, shape, dtype, initializer, regularizer, trainable, collections, caching_device, partitioner, validate_shape)
730 initializer=initializer, regularizer=regularizer, trainable=trainable,
731 collections=collections, caching_device=caching_device,
--> 732 partitioner=partitioner, validate_shape=validate_shape)
733
734
/home/mmm/programs/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.pyc in get_variable(self, var_store, name, shape, dtype, initializer, regularizer, trainable, collections, caching_device, partitioner, validate_shape)
594 regularizer=regularizer, reuse=self.reuse, trainable=trainable,
595 collections=collections, caching_device=caching_device,
--> 596 partitioner=partitioner, validate_shape=validate_shape)
597
598 def _get_partitioned_variable(
/home/mmm/programs/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.pyc in get_variable(self, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape)
159 initializer=initializer, regularizer=regularizer, reuse=reuse,
160 trainable=trainable, collections=collections,
--> 161 caching_device=caching_device, validate_shape=validate_shape)
162
163 def _get_partitioned_variable(
/home/mmm/programs/anaconda/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.pyc in _get_single_variable(self, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, validate_shape)
423
424 should_check = reuse is not None
--> 425 dtype = dtypes.as_dtype(dtype)
426 shape = tensor_shape.as_shape(shape)
427
/home/mmm/programs/anaconda/lib/python2.7/site-packages/tensorflow/python/framework/dtypes.pyc in as_dtype(type_value)
534
535 for key, val in _NP_TO_TF:
--> 536 if key == type_value:
537 return val
538
TypeError: data type not understood
I am desperate to find out what is wrong and where it went wrong.
Thanks in advance!
I'm not familiar with the tutorial but it looks like you provided tf.constant_initializer(0.0) as your data type which returns an initializer to generate constants. The third parameter of tf.get_variable() should be the data type of your variable which for a biases variable is usually something like tf.float32 or tf.float64.
The documentation also put me off. I just want to be more explicit for future readers.
The tutorial has
`tf.get_variable(<name>, <shape>, <initializer>)`: Creates or returns a variable with a given name.
which gave the suggestion that maybe just passing 3 things would work. Wrong. Need to be explicit on the key word argument. So the following won't work:
def get_mdl_get_var(x):
# variables for parameters
W = tf.get_variable('W', [784, 10], tf.random_normal_initializer(mean=0.0,stddev=0.1))
b = tf.get_variable('b', [10], tf.constant_initializer(value=0.1))
Wx_b = tf.matmul(x, W)+b
y = tf.nn.softmax(Wx_b)
return y
but the following code works now:
def get_mdl_get_var(x):
# variables for parameters
W = tf.get_variable(name='W', shape=[784, 10], initializer=tf.random_normal_initializer(mean=0.0,stddev=0.1))
b = tf.get_variable(name='b', shape=[10], initializer=tf.constant_initializer(value=0.1))
Wx_b = tf.matmul(x, W)+b
y = tf.nn.softmax(Wx_b)
return y
hope it helps.