I am getting the following error when the code comes to the dense layer -> second line (y_est =...)
Tensorflow error: An initializer for variable dense_1/kernel of dtype:'complex64' is required
My variable y_in has a complex value and it seems I have to initialize my dense layer with the same variable type (complex64) but I don't know how to do it.
Any ideas?
y_in = tf.reshape(input, shape=[-1,self.n])
y_est = tf.layers.dense(y_in, 20, activation= tf.nn.tanh) # line with error
h_hat = tf.layers.dense(y_est, 2, activation= None)
Thank you very much.
You have not specified your own custom kernel_initializer, and the standard initializers in TensorFlow do not support complex weights yet. See this ticket for the details and possible solutions.
Related
I am running Tensorflow 2.1 with keras API. I am following the following coding style:
model = tf.keras.Sequential()
...
model.fit(..., callbacks=callbacks)
Now, I would like to save some intermediate layer tensor value as image summary (as a sample what is happening at n-th training step). In order to do this, I've implemented my own callback class. I've also learned how keras.callbacks.TensorBoard is implemented, since it can save layer weights as image summaries.
I do the following in my on_epoch_end:
tensor = self.model.get_layer(layer_name).output
with context.eager_mode():
with ops.init_scope():
tensor = tf.keras.backend.get_value(tensor)
tf.summary.image(layer_name, tensor, step=step, max_outputs=1)
Unfortunately, I am still getting issue related to eager/graph modes:
tensor = tf.keras.backend.get_value(tensor)
File "/home/matwey/lab/venv/lib/python3.6/site-packages/tensorflow_core/python/keras/backend.py", line 3241, in get_value
return x.numpy()
AttributeError: 'Tensor' object has no attribute 'numpy'
Unfortunately, there is a little to no documentation on how to correctly combine keras callbacks and tf.summary.image. How could I overcome this issue?
upd: tf_nightly-2.2.0.dev20200427 has the same behaviour.
I'm in the process of converting my pytorch models into tensorflow 2.0, so I'm still getting used to it. I have mostly gone off the API, I made a custom model, and defined it's call method with argument inputs:
class CustomModel(tf.keras.Model):
<... init...>
def call(self, inputs):
print("inputs: ", inputs)
self.sequential_convolution(inputs)
The sequential_convolution is a keras.Sequential of multiple convolution related layers. I can create the model object, compile it. It is variable length on both the output and input
model = CustomModel(inputs=tf.keras.Input(shape=(None, vdim)))
model.compile(optimizer=optimizer, loss=loss_func, metrics=[calc_accuracy])
for x, y in dataset:
print("x.shape: ", x.shape)
print("y.shape: ", y.shape)
model.fit(x, y, batch_size=1)
Where the shapes are x.shape: (244, 161) and y.shape: (40,). Both are Tensorflow tensors created from numpy arrays with tf.convert_to_tensor().
But when the model's call method prints the inputs, I get the following:
Tensor("input_1_1:0", shape=(None, 161), dtype=float32)
Which I should point out is not the Input defined on the model, this input is calculated from the actual input provided in the model.fit(), I manually changed the numbers to see what the causes were...
Which then ultimately leads to the stack trace:
x = self.sequential_conv(inputs)
/usr/local/lib/python3.6/dist-packages/tensorflow_core/python/autograph/impl/api.py:396 converted_call
return py_builtins.overload_of(f)(*args)
TypeError: 'NoneType' object is not callable
This error occurs in a function deemed internal use only, but not able to ascertain what the cause of my problem is.
As I can't find much information on the matter, I feel that it's most likely something simple I haven't done, but I'm not sure. Any help would be great...
I'm using Keras/TF with the following model:
conv = Conv2D(4, 3, activation = None, use_bias=True)(inputs)
conv = Conv2D(2, 1, activation = None, use_bias=True)(conv)
model = Model(input = inputs, output = conv)
model.compile(optimizer=Adam(lr=1e-4), loss=keras.losses.mean_absolute_error)
In model.fit, I get an error saying:
ValueError: Error when checking target: expected conv2d_2 to have
shape (300, 320, 2) but got array with shape (300, 320, 1)
This is as expected because the targets are single channel images whereas the last layer in the model has 2 channels.
What I don't understand is why when I use a custom loss function:
def my_loss2(y_true, y_pred):
return keras.losses.mean_absolute_error(y_true, y_pred)
and compile the model:
model.compile(optimizer = Adam(lr=1e-4), loss=my_loss2)
it does work (or at least, not giving the error). Is there any kind of automatic conversion/truncation going on?
I'm using TF (CPU) 1.12.0, and Keras 2.2.2
Sincerely,
Elad
Why is the behavior different for built-in and custom losses?
It turns out that Keras is performing an upfront shape check for built-in functions that are defined in the losses module.
In the source code of Model._standardize_user_data, which is called by fit, I found this comment:
# If `loss_fn` is not a function (e.g. callable class)
# or if it not in the `losses` module, then
# it is a user-defined loss and we make no assumptions
# about it.
In the code around that comment you can see that indeed, depending on the type of loss function (built-in or custom), the output shape is either passed to an inner call of standardize_input_data or not. If the output shape is passed, standardize_input_data is raising the error message you are getting.
And I think this behavior makes some sense: Without knowing the implementation of a loss function, you cannot know its shape requirements. Someone may invent some loss function that needs different shapes. On the other hand, the docs clearly say that the loss function's parameters must have the same shape:
y_true: True labels. TensorFlow/Theano tensor.
y_pred: Predictions. TensorFlow/Theano tensor of the same shape as y_true.
So I find this a little inconsistent...
Why does your custom loss function work with incompatible shapes?
If you provide a custom loss, it may still work, even if the shapes do not perfectly match. In your case, where only the last dimension is different, I'm quite sure that broadcasting is what is happening. The last dimension of your targets will just be duplicated.
In many cases broadcasting is quite useful. Here, however, it is likely not since it hides a logical error.
I'm trying to initialize the parameters of an RNN cell. For that I think I have to build the layer first. So then I tried to build the layer by using the build method in BasicLSTMCell, but it gave an error int object has no attribute value. Please help me or give me another method to initialize the weights and biases.
import tensorflow as tf
my_cell = tf.contrib.rnn.BasicLSTMCell(num_units = 128) # 128 is the hidden dimension which is a hyperparameter to be choosen by you
# Note you have just declared the LSTM cell till now, they haven't been initialized.
# Do something with the LSTM
with tf.Session as sess:
sess.run(tf.global_variables_initializer()) # This line is necessary for the LSTM's parameters (The Matrices and Bias vectors) to be initialized
I am trying to create a CNN with keras to process 20x20 patches from a larger image of 600x600.
When I attempt the run the code below I receive an error AttributeError: 'Tensor' object has no attribute '_keras_history'
The below code is only intended to look at the first 20 x 20 patch out of an total of 900, I am trying to get this functioning before attempting to loop through the entire input image.
I don't understand why it is returning the error as each layer is generated with an keras layer and I haven't applied any other operations to the tensor?
I am using tensorflow 1.3 and keras 2.0.6.
nb_filters=16
input_image=Input(shape=(600,600,3))
Input_1R=Reshape((900,20,20,3))(input_image)
conv1=Convolution2D(nb_filters,(5,5),activation='relu',padding='valid')(Input_1R[:,0])
conv4=Convolution2D(1,(6,6),activation='hard_sigmoid',padding='same')(conv1)
dense6=Dense(1)(conv4)
output_dense=dense6
model = Model(inputs=input_image, outputs=output_dense)
The error occurs because the slicing operation Input_1R[:,0] is not performed in a Keras layer.
You can wrap it into a Lambda layer:
sliced = Lambda(lambda x: x[:, 0])(Input_1R)
conv1 = Convolution2D(nb_filters, (5,5), activation='relu', padding='valid')(sliced)