I am a building model with TensorFlow probability layers. When I do, model.output.shape, I get an error:
AttributeError: 'UserRegisteredSpec' object has no attribute '_shape'
If I do, output_shape = tf.shape(model.output) it gives a Keras Tensor:
<KerasTensor: shape=(5,) dtype=int32 inferred_value=[None, 3, 128, 128, 128] (created by layer 'tf.compat.v1.shape_15')
How can I get the actual values [None, 3, 128, 128, 128]?
I tried output_shape.get_shape(), but that gives the Tensor shape [5].
Code to reproduce error:
import tensorflow as tf
import tensorflow_probability as tfp
from tensorflow_probability import distributions as tfd
tfd = tfp.distributions
model = tf.keras.Sequential()
model.add(tf.keras.layers.Input(10))
model.add(tf.keras.layers.Dense(2, activation="linear"))
model.add(
tfp.layers.DistributionLambda(
lambda t: tfd.Normal(
loc=t[..., :1], scale=1e-3 + tf.math.softplus(0.1 * t[..., 1:])
)
)
)
model.output.shape
tf.shape will return a KerasTensor which is not easy to get the output shape directly.
However you can do this:
tf.shape(model.output)
>> `<KerasTensor: shape=(2,) dtype=int32 inferred_value=[None, 1] (created by layer 'tf.compat.v1.shape_168')>`
You want to get inferred_value, so:
tf.shape(model.output)._inferred_value
>> [None, 1]
Basically you can access any layer's output shape with:
tf.shape(model.layers[idx].output)._inferred_value
where idx is the index of the desired layer.
To get the output shape of all the layers you could do for instance:
out_shape_list=[]
for layer in model.layers:
out_shape = layer.output_shape
out_shape_list.append(out_shape)
You will get a list of output shapes, one for each layer
Related
I'm trying to build a Sequential model with tensorflow.
import tensorflow as tf
import keras
from tensorflow.keras import layers
from keras import optimizers
import numpy as np
model = keras.Sequential (name="model")
model.add(keras.Input(shape=(786,)))
model.add(layers.Dense(2048, activation="relu", name="layer1"))
model.add(layers.Dense(786, activation="relu", name="layer2"))
model.add(layers.Dense(786, activation="relu", name="layer3"))
output = model.add(layers.Dense(786, activation="relu", name="output"))
model.summary()
model.compile(
optimizer=tf.optimizers.Adam(), # Optimizer
loss=keras.losses.CategoricalCrossentropy(),
metrics=[keras.metrics.SparseCategoricalAccuracy()],
)
history = model.fit(
x_train,
y_train,
batch_size=1,
epochs=5,
)
The input shape is a vector with length of 768 (so the input shape is (768,) right?), representing a chess board:
def get_dataset():
container = np.load('/content/drive/MyDrive/test_data_vector.npz')
b, v = container['arr_0'], container['arr_1']
v = np.asarray(v / abs(v).max() / 2 + 0.5, dtype=np.float32) # normalization (0 - 1)
return b, v
xtrain, ytrain = get_dataset()
print(xtrain.shape)
print(ytrain.shape)
>> (37, 786) #there are 37 samples
>> (37, 786)
But I always get the error:
ValueError: Input 0 of layer model is incompatible with the layer: expected axis -1 of input shape to have value 786 but received input with shape (1, 1, 768)
I tried with np.expand_dims(), which ended in the same Error.
The error is just a typo, as the user mentioned the issue is resolved by changing the output shape from 786 to 768 and the issue is resolved.
One suggestion based on the model structure.
The number of units are not related to your input shape, you don't have to match that number.
The number of units like 2048 and 786 in dense layer is too large and this may not help the model to learn better.
Try with smaller numbers like 32,64 etc, you can refer some of the examples in the tensorflow document.
So I have this neural network and I am feeding examples "X" and labels "Y" whose shapes are:
X.shape = (10,10,2)
Y.shape = (10,10,2)
The code for the model looks like:
import tensorflow as tf
from convert import process
import numpy as np
X, Y, rate = process('songs/song1.wav')
X = np.array(X[:10])
Y = np.array(Y[:10])
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(128))
model.add(tf.keras.layers.Dense(128))
model.add(tf.keras.layers.Dense(20))
model.compile(optimizer='adam', loss='categorical_crossentropy')
model.fit(X, Y, epochs=2)
Now for some reason once I run this i get the error:
ValueError: Shapes (None, 10, 2) and (None, 20) are incompatible
I am confused because I fed it data where each example of both "X" and "Y" have shapes (10, 2). So why is it saying that I passed it (None, 10, 2) and (None, 20)
Your last layer uses linear activation whereas you choose categorical_crossentropy loss. Set either
model.add(tf.keras.layers.Dense(20, activations='softmax'))
....loss='categorical_crossentropy')
or,
model.add(tf.keras.layers.Dense(20))
....loss='mse')
Also check your data shape, especially the label (y).
I am trying the following code to save and load the TF Keras Model which has an LSTM layer with initial_states as inputs. But when I tried to load the model I am getting the error
ValueError: Layer lstm expects 1 inputs, but it received 3 input tensors. Inputs received: [<tf.Tensor 'reshape_1/Identity:0' shape=(None, 5, 5) dtype=float32>, <tf.Tensor 'dense_2/Identity:0' shape=(None, 5) dtype=float32>, <tf.Tensor 'dense_2/Identity:0' shape=(None, 5) dtype=float32>]
Is there any way to load the model with LSTM initial states?
import numpy as np
import tensorflow as tf
inputs = tf.keras.Input(name='input_1', shape=[25])
initial_state = tf.keras.Input(name='initial_state', shape=[5])
dense_x = tf.keras.layers.Dense(units=5)(initial_state)
reshape = tf.keras.layers.Reshape(target_shape=[5, 5])(inputs)
stacked_rnn = tf.keras.layers.LSTM(units=5, return_sequences=True)(inputs=reshape, initial_state=[dense_x, dense_x])
flatten = tf.keras.layers.Flatten()(stacked_rnn)
dense = tf.keras.layers.Dense(name='dense_1', units=1, activation='sigmoid')(flatten)
model = tf.keras.Model(inputs=[inputs, initial_state], outputs=dense, name="test_model")
print(model(inputs=[np.zeros(shape=[5, 25]), np.zeros(shape=[5, 5])]))
tf.keras.models.save_model(model, "sequential/", )
sequential_model = tf.keras.models.load_model("sequential/")
print(sequential_model(inputs=[np.zeros(shape=[5, 25]), np.zeros(shape=[5, 5])]))
The above code snippet is working on Tensorflow 2.4.
import numpy as np
import tensorflow as tf
inputs = tf.keras.Input(name='input_1', shape=[25])
initial_state = tf.keras.Input(name='initial_state', shape=[5])
dense_x = tf.keras.layers.Dense(units=5)(initial_state)
reshape = tf.keras.layers.Reshape(target_shape=[5, 5])(inputs)
stacked_rnn = tf.keras.layers.LSTM(units=5, return_sequences=True)(inputs=reshape, initial_state=[dense_x, dense_x])
flatten = tf.keras.layers.Flatten()(stacked_rnn)
dense = tf.keras.layers.Dense(name='dense_1', units=1, activation='sigmoid')(flatten)
model = tf.keras.Model(inputs=[inputs, initial_state], outputs=dense, name="test_model")
#initial states to the model
print(model(inputs=[np.zeros(shape=[5, 25]), np.zeros(shape=[5, 5])]))
tf.keras.models.save_model(model, "sequential/", )
sequential_model = tf.keras.models.load_model("sequential/")
#load model with initial states
print(sequential_model(inputs=[np.zeros(shape=[5, 25]), np.zeros(shape=[5, 5])]))
Output
tf.Tensor(
[[0.5]
[0.5]
[0.5]
[0.5]
[0.5]], shape=(5, 1), dtype=float32)
tf.Tensor(
[[0.5]
[0.5]
[0.5]
[0.5]
[0.5]], shape=(5, 1), dtype=float32)
when feeding a tf.data.Dataset to train EfficientnetB0 model I get the following error:
ValueError: in converted code:
C:\Users\fconrad\AppData\Local\Continuum\anaconda3\envs\venv_spielereien\lib\site-packages\tensorflow_core\python\keras\engine\training_v2.py:677 map_fn
batch_size=None)
C:\Users\fconrad\AppData\Local\Continuum\anaconda3\envs\venv_spielereien\lib\site-packages\tensorflow_core\python\keras\engine\training.py:2410 _standardize_tensors
exception_prefix='input')
C:\Users\fconrad\AppData\Local\Continuum\anaconda3\envs\venv_spielereien\lib\site-packages\tensorflow_core\python\keras\engine\training_utils.py:573 standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking input: expected efficientnet-b0_input to have 4 dimensions, but got array with shape (224, 224, 3)
I realy wonder why this happens, since when I create a batch from my Dataset:
train_generator = (tf.data.Dataset
.from_tensor_slices((train_imgs, train_labels))
.map(read_img)
.map(flip_img)
.map(brightness)
.map(blur)
.map(noise)
.map(rotate_90)
.repeat()
.shuffle(512)
.batch(BATCH_SIZE)
.prefetch(True))
validation_generator = (tf.data.Dataset
.from_tensor_slices((validation_imgs, validation_labels))
.map(read_img)
)
print(train_generator.__iter__().__next__()[0].shape)
I get the expected result (64, 224, 224, 3).
But after creating the model the error above raises when I start training:
effn = tfkeras.EfficientNetB0(include_top=False, input_shape=img_shape, classes=4)
effn_model = tf.keras.Sequential()
effn_model.add(effn)
effn_model.add(tf.keras.layers.GlobalAveragePooling2D())
effn_model.add(tf.keras.layers.Dense(4, 'softmax'))
effn_model.compile(optimizer= 'adam', loss='categorical_crossentropy', metrics= ['categorical_accuracy'])
effn_model.fit(train_generator,
epochs=20,
steps_per_epoch=train_imgs.shape[0] // BATCH_SIZE,
validation_data= validation_generator)
Does anyone know why the slices from dataset have shape (64,224,224,3) but the model doesnt recognize the batch dimension? when I try to train a keras.application model, everything works fine.
I use tensorflow 2.1 and the pip install of efficientnet. Thanks
as explained here keras.io/api/applications/efficientnet/
input_shape: Optional shape tuple, only to be specified if include_top is False. It should have exactly 3 inputs channels.
as so try this->
from tensorflow.keras.applications.efficientnet import EfficientNetB0, EfficientNetB5
mm = EfficientNetB0(include_top=True, weights=None, input_tensor=None, input_shape=(128, 128, 3), pooling=None, classes=2, classifier_activation="sigmoid")
mm.summary()
note the input_shape=(128, 128, 3) It has 3 channels.
I'm trying to customize the model taken from tf hub but
can't access the layers with following error 'KerasLayer' object has no attribute 'layers'
Here is my code as an example:
import tensorflow_hub as hub
from tensorflow.keras import layers
feature_extractor_url = "https://tfhub.dev/tensorflow/efficientnet/lite0/feature-vector/1"
base_model = hub.KerasLayer(feature_extractor_url,
input_shape=(224,224,3))
base_model.trainable = True
import tensorflow
from tensorflow.keras.models import Model
x = base_model.layers[-10].output
x = tensorflow.keras.layers.Conv2D(4, (3, 3), padding="same", activation="relu")(x)
x = tensorflow.keras.layers.GlobalMaxPooling2D()(x)
x = tensorflow.keras.layers.Flatten()(x)
outputs = tensorflow.keras.layers.Activation('sigmoid', name="example_output")(x)
model = Model(base_model.input, outputs=outputs)
model.summary()
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-43-0501ec56d6c4> in <module>()
14 from tensorflow.keras.models import Model
15
---> 16 x = base_model.layers[-10].output
17 x = tensorflow.keras.layers.Conv2D(4, (3, 3), padding="same", activation="relu")(x)
18 x = tensorflow.keras.layers.GlobalMaxPooling2D()(x)
AttributeError: 'KerasLayer' object has no attribute 'layers'
What I've tried:
I built the model using sequential api :
model = tf.keras.Sequential([
base_model,
layers.Dense(image_data.num_classes)
])
model.summary()
But still a I can't access the layers inside base_model.
How can I access the layers from KerasLayer?
Thank you!
You can access the layers via weights of the Hub model.
The topic is not straightforwardly mentioned in the TF-docs unfortunately. This is the depth I could dig up to so far and hopefully it sheds some light on accessing layers on Hub.
TF 2.5.0 & TF-Hub 0.12.0 have been used for the below tests.
Layers in KerasLayer object
>>> import tensorflow_hub as hub
>>> model = hub.KerasLayer("https://tfhub.dev/deepmind/ganeval-cifar10-convnet/1")
>>> model
<tensorflow_hub.keras_layer.KerasLayer object at 0x7f0c79372190>
>>> len(model.weights)
57
>>> model.weights[56]
<tf.Variable 'cifar10_convnet/linear/b:0' shape=(10,) dtype=float32, numpy=
array([-0.2734375 , -1.46875 , 0.484375 , 1.2265625 , 0.53515625,
0.96875 , 0.3671875 , 0.02282715, -0.7265625 , -1.078125 ],
dtype=float32)>
>>> model.weights[56].name
'cifar10_convnet/linear/b:0'
Notice the above weights variable. KerasLayer has a get_weights() function as well. The difference in output is as below. Basically, the former is of type TF-Variable, and the latter is a numpy array.
>>> len(model.get_weights())
57
>>> model.get_weights()[56]
array([-0.2734375 , -1.46875 , 0.484375 , 1.2265625 , 0.53515625,
0.96875 , 0.3671875 , 0.02282715, -0.7265625 , -1.078125 ],
dtype=float32)
To get, for example, the names of all layers, simply run:
layers = model.weights
[ layers[i].name for i in range( len(layers) ) ]
A hint of my output:
'cifar10_convnet/conv_net_2d/conv_2d_0/w:0',
'cifar10_convnet/conv_net_2d/conv_2d_0/b:0',
'cifar10_convnet/conv_net_2d/batch_norm_0/moving_mean:0',
'cifar10_convnet/conv_net_2d/batch_norm_0/moving_variance:0'
Note that weight, bias, mean, variance etc. are all listed separately as layers in the output.
Layers in AutoTrackable object
This is for low-level TF2 API users.
>>> import tensorflow_hub as hub
>>> model = hub.load("https://tfhub.dev/deepmind/ganeval-cifar10-convnet/1")
>>> model
<tensorflow.python.training.tracking.tracking.AutoTrackable object at 0x7f95943ec410>
>>> len(model.variables)
57
>>> model.variables[56]
<tf.Variable 'cifar10_convnet/linear/b:0' shape=(10,) dtype=float32, numpy=
array([-0.2734375 , -1.46875 , 0.484375 , 1.2265625 , 0.53515625,
0.96875 , 0.3671875 , 0.02282715, -0.7265625 , -1.078125 ],
dtype=float32)>
Use "variables" instead of "weights" with this API.
The inner structure of the SavedModel loaded into a hub.KerasLayer is inaccessible. For this level of detail, you'll have to turn to EfficientNet source code instead.
I met the same problem as you yesterday,but luckily for me, i have found two ways to solve the problem.
1.base_model = tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE,
include_top=False,
weights='imagenet'
)
base_model.trainable = False
global_average_layer = tf.keras.layers.GlobalAveragePooling2D()
prediction_layer = tf.keras.layers.Dense(59)
model = tf.keras.Sequential([
base_model,
global_average_layer,
prediction_layer
])
model.summary()
....
fine_tune_at =100
for layer in base_model.layers[:fine_tune_at]:
layer.trainable = False
use the ' tf.keras.applications' with '.layers' together
2.Reference tensorflow documentation:https://tensorflow.google.cn/hub/tf2_saved_model?hl=en
base_model = hub.KerasLayer(feature_extractor_url,
trainable=True,
input_shape=(224,224,3))
model.summary()
Model: "sequential_2"
Layer (type) Output Shape Param #
keras_layer_1 (KerasLayer) (None, 1024) 3228864
dense_2 (Dense) (None, 59) 60475
Total params: 3,289,339
Trainable params: 3,267,451
Non-trainable params: 21,888
I hope my answer will help you