I need to embed sentence universal encoder into my keras model using Google colab.
import tensorflow as tf
import tensorflow_hub as hub
from tensorflow import keras
def UniversalEmbedding(x):
results = embed(tf.squeeze(tf.cast(x, tf.string)))["outputs"]
return keras.backend.concatenate([results])
module_url = "https://tfhub.dev/google/universal-sentence-encoder/4"
embed = hub.load(module_url)
input_size = 1
embed_size = 512
input_1 = keras.layers.Input(shape=(input_size,), dtype=tf.string)
embed_layer = keras.layers.Lambda(UniversalEmbedding, output_shape=(embed_size,))
x1 = embed_layer(input_1)
It throws a type error as follows.
TypeError: Only integers, slices (`:`), ellipsis (`...`), tf.newaxis (`None`) and scalar tf.int32/tf.int64 tensors are valid indices, got 'outputs'
TF version: 2.3.0
Any hint to fix it is appreciated.
Related
I'm trying to convert a LSTM model from TensorFlow into ONNX. The code for generating data for TensorFlow model training is as below:
def make_dataset(self, data):
data = np.array(data, dtype=np.float32)
ds = tf.keras.utils.timeseries_dataset_from_array(
data=data,
targets=None,
sequence_length=self.total_window_size,
sequence_stride=1,
shuffle=True,
batch_size=32, )
ds = ds.map(self.split_window)
The model training code is actually from the official tutorial. Then after conversion to ONNX, I try to perform prediction as follows:
import onnx
import onnxruntime as rt
from tf_lstm import WindowGenerator
import tensorflow as tf
wide_window = WindowGenerator(
input_width=24, label_width=24, shift=1,
label_columns=['T (degC)'])
model = onnx.load_model('models/onnx/tf-lstm-weather.onnx')
print(model)
sess = rt.InferenceSession('models/onnx/tf-lstm-weather.onnx')
input_name = sess.get_inputs()[0].name
label_name = sess.get_outputs()[0].name
pred = sess.run([label_name], {input_name: wide_window.test})[0]
But it throws this error:
RuntimeError: Input must be a list of dictionaries or a single numpy array for input 'lstm_input'.
I tried to convert wide_window.test into numpy array and use it instead as follows:
test_data = []
test_label = []
for x, y in wide_window.test:
test_data.append(x.numpy())
test_label.append(y.numpy())
test_data2 = np.array(test_data, dtype=np.float)
pred = sess.run([label_name], {input_name: test_data2})[0]
Then it gives this error:
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (219,) + inhomogeneous part.
Any idea?
That's a numpy error. Each row you add to the input array has to have the same number of elements.
setting an array element with a sequence requested array has an inhomogeneous shape after 1 dimensions The detected shape was (2,)+inhomogeneous part
I'm trying to use huggingface and tensorflow to train a BERT model on some data. Here's my code:
First, I initialized the tokenizer.
from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased', sep_token = "||")
Then applied my tokenizer to my data.
def preprocess_function(x):
return tokenizer(x, truncation = True, return_tensors = 'tf')['input_ids']
from tqdm import tqdm
tqdm.pandas()
df["Text"] = df["Text"].progress_apply(preprocess_function)
And some more preprocessing..
df["intvwStatus"] = [0 if x == "Completed" else 1 for x in df["intvwStatus"]]
import numpy as np
train, validate, test = \
np.split(df.sample(frac=1, random_state=42),
[int(.6*len(df)), int(.8*len(df))])
Created an optimizer
from transformers import create_optimizer
import tensorflow as tf
batch_size = 16
num_epochs = 5
batches_per_epoch = len(train) // batch_size
total_train_steps = int(batches_per_epoch * num_epochs)
optimizer, schedule = create_optimizer(init_lr=2e-5, num_warmup_steps=0, num_train_steps=total_train_steps)
And then finally instantiated and compiled my model
from transformers import TFBertForSequenceClassification
model = TFBertForSequenceClassification.from_pretrained("bert-base-uncased")
import tensorflow as tf
model.compile(optimizer=optimizer)
Then fit my model
x_train = train["Text"]
y_train = train["intvwStatus"]
x_val = validate["Text"]
y_val = validate["intvwStatus"]
model.fit(x=x_train,y=y_train, validation_data=(x_val, y_val), epochs=3)
Which gives error:
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object type tensorflow.python.framework.ops.EagerTensor).
I'm confused. Why is it confusing tensorflow.python.framework.ops.EagerTensor to a NumPy array?
I have the following code, I am retrieving error when I try to add my own classifier.
import keras
from keras import layers,Model
from keras.layers import Input,GlobalAveragePooling2D,Flatten,Dense
MobileNetV2_model= tf.keras.applications.MobileNetV2(input_shape=None, alpha=1.0, include_top=False,
weights='imagenet')
#MobileNetV2_model.summary()
x= MobileNetV2_model.output
x = layers.GlobalAveragePooling2D()(x)
final_output=layers.Dense(2, activation='sigmoid')(x)
model = keras.Model(inputs=MobileNetV2.input, outputs = final_output)
model.compile(optimizer="adam", loss='BinaryCrossentropy', metrics=['accuracy'],loss_weights=0.1)
Error
TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that
you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying
to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.
You should never mix keras and tf.keras. You can refer working code as shown below
import tensorflow as tf
from tensorflow.keras import layers, Model
MobileNetV2_model= tf.keras.applications.MobileNetV2(input_shape=(224,224,3), alpha=1.0, include_top=False, weights='imagenet')
#MobileNetV2_model.summary()
x= MobileNetV2_model.output
x = layers.GlobalAveragePooling2D()(x)
final_output=layers.Dense(2, activation='sigmoid')(x)
model = Model(inputs=MobileNetV2_model.input, outputs = final_output)
model.compile(optimizer="adam", loss='BinaryCrossentropy', metrics=['accuracy'],loss_weights=0.1)
the data have 4 timestamps,but the embedding's input_length=3,so what's the meaning of input_length?
from tensorflow import keras
import numpy as np
data = np.array([[0,0,0,0]])
emb = keras.layers.Embedding(input_dim=2, output_dim=3, input_length=3)
emb(data)
As per the official documentation here,
input_length: Length of input sequences, when it is constant. This
argument is required if you are going to connect Flatten then Dense
layers upstream (without it, the shape of the dense outputs cannot be
computed).
from tensorflow import keras
import numpy as np
model = keras.models.Sequential()
model.add(keras.layers.Embedding(input_dim=2, output_dim=3, input_length=4))
# the model will take as input an integer matrix of size (batch, input_length).
input_array = np.array([[0,0,0,0]])
model.compile('rmsprop', 'mse')
output_array = model.predict(input_array)
print(output_array)
Above works fine, but if you change input_length to 3, then you will get below error:
ValueError: Error when checking input: expected embedding_input to
have shape (3,) but got array with shape (4,)
I'm facing a very curious bug in Keras when trying to use Inception inside a TimeDistributed wrapper.
This code is simple and should work with many models or layers, but weirdly, inception_v3 fails at prediction time:
import numpy as np
from keras.applications import inception_v3
from keras.layers import *
from keras.models import Model
imgShape = (299,299,3)
seqShape = (2,299,299,3)
incept = inception_v3.InceptionV3(weights=None, include_top=False)
inputs = Input(seqShape)
outputs = TimeDistributed(incept)(inputs)
model = Model(inputs,outputs)
Everything works perfectly until I try to predict something:
pred = model.predict(np.ones((1,2,299,299,3)))
The error is:
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'batch_normalization_1/keras_learning_phase' with dtype bool
[[Node: batch_normalization_1/keras_learning_phase = Placeholderdtype=DT_BOOL, shape=, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Any solutions to this?
Using Keras 2.1.0 and Tensorflow 1.4.0.