I'm trying to use a MobileNetV2 model with a TimeDistributed layer in TensorFlow 2.0 to process a sequence of images. Here is my code:
`import tensorflow as tf
mobilenet = tf.keras.applications.MobileNetV2(input_shape=(224, 224, 3), include_top=False)
mobilenet = tf.keras.Sequential([mobilenet, tf.keras.layers.GlobalAveragePooling2D()])
td_layer = tf.keras.layers.TimeDistributed(mobilenet, input_shape=(15, 224, 224, 3))
However, when I run this code, I get the following error:`
Copy code
NotImplementedError: Exception encountered when calling layer "time_distributed_12" (type TimeDistributed).
Please run in eager mode or implement the `compute_output_shape` method on your layer (KerasLayer).
Call arguments received by layer "time_distributed_12" (type TimeDistributed):
• inputs=tf.Tensor(shape=(None, 6, 224, 224, 3), dtype=float32)
• training=None
• mask=None
I understand that I need to implement the compute_output_shape method for the MobileNetV2 model, but I'm not sure how to do that. Can anyone provide an example of how to implement this method, or suggest an alternative approach to using the MobileNetV2 model in a TimeDistributed layer?
Thank you in advance for any help you can provide.
i have tried,
`class MobileNetWithShape(tf.keras.applications.MobileNetV2):
def __init__(self, *args, **kwargs):
super(MobileNetWithShape, self).__init__(*args, **kwargs)
def compute_output_shape(self, input_shape):
shape = tf.TensorShape(input_shape).as_list()
shape[1] = self.output.shape[1]
return tf.TensorShape(shape)
Create an instance of the MobileNetWithShape model
mobilenet = MobileNetWithShape(input_shape=(224, 224, 3), include_top=False)
result = mobilenet
td_layer = tf.keras.layers.TimeDistributed(mobilenet, input_shape=(15, 224, 224, 3))`
This was an output:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Input In [32], in <cell line: 1>()
----> 1 class MobileNetWithShape(tf.keras.applications.MobileNetV2):
2 def __init__(self, *args, **kwargs):
3 super(MobileNetWithShape, self).__init__(*args, **kwargs)
TypeError: function() argument 'code' must be code, not str
Related
I am working on tensorflow federated (tff). The problem arises when I call the iterative process and pass on the instance of the model created.
I have declared a keras model which is used at both the server and at the client. I basically want to modify the client model's weights and biases in order to perform mathematical functions on them. These mathematical functions are performed on the models' weights and biases by using function named 'processed'.
The processed function can return weights and layers both in tf or numpy format (the funtion 'processed' just uses tf.convert_to_tensor command to convert the numpy returned weights/biases from the get/set_weights command used)
def create_keras_model():
return tf.keras.models.Sequential([
tf.keras.layers.Conv2D(filters=32, kernel_size=[5, 5],name='conv2d_1',activation=tf.nn.relu, use_bias=True, bias_initializer =tf.initializers.lecun_normal(seed=137), input_shape=(28 ,28 ,1 )),
tf.keras.layers.MaxPool2D(pool_size=[2, 2], strides=2),
tf.keras.layers.Conv2D(filters=32, kernel_size=[5 5],name='conv2d_2',activation=tf.nn.relu, use_bias = True, bias_initializer=tf.initializers.lecun_normal(seed=137)),
tf.keras.layers.MaxPool2D(pool_size=[2 2], strides=2),
tf.keras.layers.Reshape(target_shape=(4 * 3 * 3)),
tf.keras.layers.Dense(units= 50, activation=tf.nn.relu, use_bias=True, bias_initializer=tf.initializers.lecun_normal(seed=137), name='dense_1'),
tf.keras.layers.Dense(units=10 , use_bias=True, bias_initializer=tf.initializers.lecun_normal(seed=137), activation=tf.nn.softmax, name='dense_2' ),
])
I create an instant of the above Keras model named as
net_1 = create_keras_model()
Now I am accessing the weights and biases by using the following function (proccessed) and assigning them new weights and biases after some mathmatical operations
def processed(net_1,L_norm):
a=net_1.get_layer('conv2d_1').get_weights()[0]
b=net_1.get_layer('conv2d_1').get_weights()[1]
c= net_1.get_layer('conv2d_2').get_weights()[0]
d=net_1.get_layer('conv2d_2').get_weights()[1]
e= net_1.get_layer('dense_1').get_weights()[0]
f= net_1.get_layer('dense_1').get_weights()[1]
g=net_1.get_layer('dense_2').get_weights()[0]
h= net_1.get_layer('dense_2').get_weights()[1]
L1,B1,L2,B2,L3,B3,L4,B4 = processing_work(a,b,c,d,e,f,g,h,L_norm)
L1=tf.convert_to_tensor(L1)
B1=tf.convert_to_tensor(B1)
L2=tf.convert_to_tensor(L2)
B2=tf.convert_to_tensor(B2)
L3=tf.convert_to_tensor(L3)
B3=tf.convert_to_tensor(B3)
L4=tf.convert_to_tensor(L4)
B4=tf.convert_to_tensor(B4)
net_1.get_layer('conv2d_1').set_weights([L1,B1])
net_1.get_layer('conv2d_2').set_weights([L2,B2])
net_1.get_layer('dense_1').set_weights([L3,B3])
net_1.get_layer('dense_2').set_weights([L4,B4])
return net_1
Following that, I use the following function
def model_fn_of():
# We _must_ create a new model here, and _not_ capture it from an external
# scope. TFF will call this within different graph contexts.
local_model = processed(create_keras_model(),L_norm)
return tff.learning.from_keras_model(
local_model,
input_spec=preprocessed_example_dataset.element_spec,
loss=tf.keras.losses.SparseCategoricalCrossentropy(),
metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])
Then I call the iterator of tff.
iterative_process = tff.learning.algorithms.build_weighted_fed_avg(
model_fn_of,
client_optimizer_fn=lambda: tf.keras.optimizers.SGD(learning_rate=0.02),
server_optimizer_fn=lambda: tf.keras.optimizers.SGD(learning_rate=1.00))
However I get the following error:
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-32-651e903cc722> in <module>
2 model_fn_of,
3 client_optimizer_fn=lambda: tf.keras.optimizers.SGD(learning_rate=0.02),
----> 4 server_optimizer_fn=lambda: tf.keras.optimizers.SGD(learning_rate=1.00))
8 frames
/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py in __getattr__(self, name)
511 from tensorflow.python.ops.numpy_ops import np_config
512 np_config.enable_numpy_behavior()""".format(type(self).__name__, name))
--> 513 self.__getattribute__(name)
514
515 #staticmethod
AttributeError: 'Tensor' object has no attribute 'numpy'
Can someone please help me with this. I dont know what is wrong and from which part of the code the error is even popping out. Please note that the dtype of weights/biases of model layers before and after using the 'processed' function are same.
Thanks in Advance
I need to build a transformer-based architecture in Tensorflow following the encoder-decoder approach where the encoder is a preexisting Huggingface Distilbert model and the decoder is a CNN.
Inputs: a text containing texts with several phrases in a row. Outputs: codes according to taxonomic criteria. My data file has 7387 pairs text-label in TSV format:
text \t code
This is example text number one. It might contain some other phrases. \t C21
This is example text number two. It might contain some other phrases. \t J45.1
This is example text number three. It might contain some other phrases. \t A27
The remainder of the code is this:
text_file = "data/datafile.tsv"
with open(text_file) as f:
lines = f.read().split("\n")[:-1]
text_and_code_pairs = []
for line in lines:
text, code = line.split("\t")
text_and_code_pairs.append((text, code))
random.shuffle(text_and_code_pairs)
num_val_samples = int(0.10 * len(text_and_code_pairs))
num_train_samples = len(text_and_code_pairs) - 3 * num_val_samples
train_pairs = text_and_code_pairs[:num_train_samples]
val_pairs = text_and_code_pairs[num_train_samples : num_train_samples + num_val_samples]
test_pairs = text_and_code_pairs[num_train_samples + num_val_samples :]
train_texts = [fst for (fst,snd) in train_pairs]
train_labels = [snd for (fst,snd) in train_pairs]
val_texts = [fst for (fst,snd) in val_pairs]
val_labels = [snd for (fst,snd) in val_pairs]
test_texts = [fst for (fst,snd) in test_pairs]
test_labels = [snd for (fst,snd) in test_pairs]
distilbert_encoder = TFDistilBertModel.from_pretrained("distilbert-base-multilingual-cased")
tokenizer = DistilBertTokenizerFast.from_pretrained("distilbert-base-multilingual-cased")
train_encodings = tokenizer(train_texts, truncation=True, padding=True)
val_encodings = tokenizer(val_texts, truncation=True, padding=True)
test_encodings = tokenizer(test_texts, truncation=True, padding=True)
train_dataset = tf.data.Dataset.from_tensor_slices((
dict(train_encodings),
train_labels
))
val_dataset = tf.data.Dataset.from_tensor_slices((
dict(val_encodings),
val_labels
))
test_dataset = tf.data.Dataset.from_tensor_slices((
dict(test_encodings),
test_labels
))
model = build_model(distilbert_encoder)
model.fit(train_dataset.batch(64), validation_data=val_dataset, epochs=3, batch_size=64)
model.predict(test_dataset, verbose=1)
Lastly, the build_model function:
def build_model(transformer, max_len=512):
model = tf.keras.models.Sequential()
# Encoder
inputs = layers.Input(shape=(max_len,), dtype=tf.int32)
distilbert = transformer(inputs)
# LAYER - something missing here?
# Decoder
conv1D = tf.keras.layers.Conv1D(filters=5, kernel_size=10)(distilbert)
pooling = tf.keras.layers.MaxPooling1D(pool_size=2)(conv1D)
flat = tf.keras.layers.Flatten()(pooling)
fc = tf.keras.layers.Dense(1255, activation='relu')(flat)
softmax = tf.keras.layers.Dense(1255, activation='softmax')(fc)
model = tf.keras.models.Model(inputs = inputs, outputs = softmax)
model.compile(tf.keras.optimizers.Adam(learning_rate=5e-5), loss="categorical_crossentropy", metrics=['accuracy'])
print(model.summary())
return model
I managed to narrow down the possible locations of my problem. After changing from sequential to functional Keras API, I get the following error:
Traceback (most recent call last):
File "keras_transformer.py", line 99, in <module>
main()
File "keras_transformer.py", line 94, in main
model = build_model(distilbert_encoder)
File "keras_transformer.py", line 23, in build_model
conv1D = tf.keras.layers.Conv1D(filters=5, kernel_size=10)(distilbert)
File "/home/users/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 897, in __call__
self._maybe_build(inputs)
File "/home/users/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 2416, in _maybe_build
self.build(input_shapes) # pylint:disable=not-callable
File "/home/users/user/.local/lib/python3.6/site-packages/tensorflow/python/keras/layers/convolutional.py", line 152, in build
input_shape = tensor_shape.TensorShape(input_shape)
File "/home/users/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 771, in __init__
self._dims = [as_dimension(d) for d in dims_iter]
File "/home/users/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 771, in <listcomp>
self._dims = [as_dimension(d) for d in dims_iter]
File "/home/users/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 716, in as_dimension
return Dimension(value)
File "/home/users/user/.local/lib/python3.6/site-packages/tensorflow/python/framework/tensor_shape.py", line 200, in __init__
None)
File "<string>", line 3, in raise_from
TypeError: Dimension value must be integer or None or have an __index__ method, got 'last_hidden_state'
It seems that the error lies in the connection between the output of the transformer and the input of the convolutional layer. Am I supposed to include another layer between them so as to adapt the output of the transformer? If so, what would be the best option?I'm using tensorflow==2.2.0, transformers==4.5.1 and Python 3.6.9
I think the problem is to call the right tensor for the tensorflow layer after the dilbert instance. Because distilbert = transformer(inputs) returns an instance rather than a tensor like in tensorflow, e.g., pooling = tf.keras.layers.MaxPooling1D(pool_size=2)(conv1D). pooling is the output tensor of the MaxPooling1D layer.
I fix your problem by calling the last_hidden_state variable of the distilbert instance (i.e. output of the dilbert model), and this will be your input to the next Conv1D layer.
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' # suppress Tensorflow messages
from transformers import TFDistilBertModel, DistilBertModel
import tensorflow as tf
distilbert_encoder = TFDistilBertModel.from_pretrained("distilbert-base-multilingual-cased")
def build_model(transformer, max_len=512):
# model = tf.keras.models.Sequential()
# Encoder
inputs = tf.keras.layers.Input(shape=(max_len,), dtype=tf.int32)
distilbert = transformer(inputs)
# Decoder
###### !!!!!! #########
conv1D = tf.keras.layers.Conv1D(filters=5, kernel_size=10)(distilbert.last_hidden_state)
###### !!!!!! #########
pooling = tf.keras.layers.MaxPooling1D(pool_size=2)(conv1D)
flat = tf.keras.layers.Flatten()(pooling)
fc = tf.keras.layers.Dense(1255, activation='relu')(flat)
softmax = tf.keras.layers.Dense(1255, activation='softmax')(fc)
model = tf.keras.models.Model(inputs = inputs, outputs = softmax)
model.compile(tf.keras.optimizers.Adam(learning_rate=5e-5), loss="categorical_crossentropy", metrics=['accuracy'])
print(model.summary())
return model
model = build_model(distilbert_encoder)
This returns,
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 512)] 0
_________________________________________________________________
tf_distil_bert_model (TFDist TFBaseModelOutput(last_hi 134734080
_________________________________________________________________
conv1d (Conv1D) (None, 503, 5) 38405
_________________________________________________________________
max_pooling1d (MaxPooling1D) (None, 251, 5) 0
_________________________________________________________________
flatten (Flatten) (None, 1255) 0
_________________________________________________________________
dense (Dense) (None, 1255) 1576280
_________________________________________________________________
dense_1 (Dense) (None, 1255) 1576280
=================================================================
Total params: 137,925,045
Trainable params: 137,925,045
Non-trainable params: 0
Note: I assume you mean tf.keras.layers.Input by layers.Input in your build_model function.
I think you are right. The problem seems to be the input to the Conv1D layer.
According to the documentation the outputs.last_hidden_state has a shape of (batch_size, sequence_length, hidden_size).
Conv1D is expecting an input of shape (batch_size, sequence_length).
Maybe you could resolve the problem by either changing your Conv1D to Conv2D or adding a Conv2D layer in between.
I have an autoencoder for image compression, where the encoded tensor has the shape: (batch_size, 12, 64, 48).
batch_size is the number of images being fed in a batch,
12 is the number of channels of this last encoder layer, which has a
64x48 width/height.
I want to input this to a ConvLSTM2D layer, and i would like the output of the ConvLSTM2D to have the same dimension as the input of the ConvLSTM2D.
The intention is to see image reconstruction on a video sequence, rather than unordered images from a dataset.
Placing a ConvLSTM2d between an encoder/decoder in a autoencoder architecture has been difficult, especially because most examples use the Sequential API, and i want to use the functional API in Keras.
I tried reshaping the input but the error persists
import tensorflow as tf
import tensorflow.keras.backend as K
def LSTM_layer(input):
input = tf.keras.backend.expand_dims(input, axis=-1)
lstm1 = tf.keras.layers.ConvLSTM2D(filters=12, kernel_size=(3, 3), strides=(1, 1), data_format="channels_first",
input_shape=(None, 12, 64, 48),
padding='same', return_sequences=True)(input)
return lstm1
def build_model(input_shape):
#create an input with input_shape as the size
input_ = tf.keras.Input(shape=input_shape, name="input_node")
lstm_features = LSTM_layer(input_)
model = tf.keras.Model(inputs=input_, outputs=[lstm_features])
return model
def main():
input_shape = (12, 64, 48) #this is the size of the tensor which is outputted by my encoder, with channels_first assumed
model = build_model(input_shape)
if __name__ == '__main__':
main()
Unfortunately, this is throwing this error:
Traceback (most recent call last):
File "lstm.py", line 29, in <module>
main()
File "lstm.py", line 26, in main
model = build_model(input_shape)
File "lstm.py", line 20, in build_model
model = tf.keras.Model(inputs=input_, outputs=[lstm_features])
File "/home/hallab/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/training.py", line 121, in __init__
super(Model, self).__init__(*args, **kwargs)
File "/home/hallab/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/network.py", line 80, in __init__
self._init_graph_network(*args, **kwargs)
File "/home/hallab/.local/lib/python3.5/site-packages/tensorflow/python/training/checkpointable/base.py", line 474, in _method_wrapper
method(self, *args, **kwargs)
File "/home/hallab/.local/lib/python3.5/site-packages/tensorflow/python/keras/engine/network.py", line 224, in _init_graph_network
'(thus holding past layer metadata). Found: ' + str(x))
ValueError: Output tensors to a Model must be the output of a TensorFlow `Layer` (thus holding past layer metadata). Found: Tensor("conv_lst_m2d/transpose_1:0", shape=(?, 12, 12, 48, 1), dtype=float32)
Most posts about this error instruct to wrap the operation in a lambda.. but i am not implementing a custom operation here, this should be a keras tf layer... right?
Also, in my implementation, i want the output tensor from the LSTM unit to be the same as the input, can i get some feedback about that as well?
Thank you.
You could use Lambda to wrap the output form K.expand_dims before input it to next layer like this:
import tensorflow as tf
import tensorflow.keras.backend as K
from tensorflow.keras.layers import Lambda
def expand_dims(x):
return K.expand_dims(x, 1)
def expand_dims_output_shape(input_shape):
return (input_shape[0], 1, input_shape[1])
def LSTM_layer(input_):
lstm1 = Lambda(expand_dims, expand_dims_output_shape)(input_)
lstm1 = tf.keras.layers.ConvLSTM2D(filters=12, kernel_size=(3, 3), strides=(1, 1), data_format="channels_first", padding='same', return_sequences=False)(lstm1)
return lstm1
I'm trying to reduce the GPU memory cost of DenseNet. I found someone use tf.contrib.layers.recompute_grad to complete this work in tensorflow. I have a Keras model and I want to use tf.contrib.layers.recompute_grad in Keras. I tried to use a Keras custom layer to wrap it up.
class Back_Recompute(Layer):
def __init__(self, filters, kernel_size, w_decay, **kwargs):
super(Back_Recompute, self).__init__(**kwargs)
self.n_filters = filters
self.we_decay = w_decay
self.ks = kernel_size
def call(self, ip):
global brcount
with tf.variable_scope('BR_{}'.format(brcount), use_resource=True):
def _x(inner_ip):
x = Conv2D(self.n_filters, self.ks, kernel_initializer='he_normal', padding='same', use_bias=False,
kernel_regularizer=l2(self.we_decay))(inner_ip)
return x
brcount = brcount + 1
_x = tf.contrib.layers.recompute_grad(_x)
return _x(ip)
def compute_output_shape(self, input_shape):
return (input_shape[0], input_shape[1], input_shape[2], self.n_filters)
But weird problem is that my train code was stuck in fit_generator and cannot execute without error message. Then I thought that was probably because of Keras layer, so I have changed Keras layer Conv2D to tf.layers.conv2d. When I run the train code, I got a error message: AttributeError: 'Activation' object has no attribute 'outbound_nodes'. I don't know how to use tf.contrib.layers.recompute_grad in Keras. Does someone can help me?
I have created a custom layer (called GraphGather) in Keras, yet the output tensor prints as :
Tensor("graph_gather/Tanh:0", shape=(?, ?), dtype=float32)
For some reason the shape is being returned as (?,?), which is causing the next dense layer to raise the following error:
ValueError: The last dimension of the inputs to Dense should be defined. Found None.
The GraphGather layer code is as follows:
class GraphGather(tf.keras.layers.Layer):
def __init__(self, batch_size, num_mols_in_batch, activation_fn=None, **kwargs):
self.batch_size = batch_size
self.num_mols_in_batch = num_mols_in_batch
self.activation_fn = activation_fn
super(GraphGather, self).__init__(**kwargs)
def build(self, input_shape):
super(GraphGather, self).build(input_shape)
def call(self, x, **kwargs):
# some operations (most of def call omitted)
out_tensor = result_of_operations() # this line is pseudo code
if self.activation_fn is not None:
out_tensor = self.activation_fn(out_tensor)
out_tensor = out_tensor
return out_tensor
def compute_output_shape(self, input_shape):
return (self.num_mols_in_batch, 2 * input_shape[0][-1])}
I have also tried hardcoding compute_output_shape to be:
python
def compute_output_shape(self, input_shape):
return (64, 150)
```
Yet the output tensor when printed is still
Tensor("graph_gather/Tanh:0", shape=(?, ?), dtype=float32)
which causes the ValueError written above.
System information
Have written custom code
**OS Platform and Distribution*: Linux Ubuntu 16.04
TensorFlow version (use command below): 1.5.0
Python version: 3.5.5
I had the same problem. My workaround was to add the following lines to the call method:
input_shape = tf.shape(x)
and then:
return tf.reshape(out_tensor, self.compute_output_shape(input_shape))
I haven't run into any problems with it yet.
If Johnny's answer doesn't work, I found another way to get around this is to follow advice here https://github.com/tensorflow/tensorflow/issues/38296#issuecomment-623698709
which is to call the set_shape method on the output of your layer.
E.g.
l=GraphGather(...)
y=l(x)
y.set_shape( l.compute_output_shape(x.shape) )
This only works if you are using the functional API.