Multiple parameters for 2 models - tensorflow

I have 2 models which are merged into a single model. This single model is created as follows:
fm = layers.Concatenate()([m1.output, m2.output])
# create dense layer for fusion
t = layers.Dense(143, activation='relu')(fm )
d = layers.Dropout(0.5)(t)
# create softmax layer for predictions
output = layers.Dense(len(classes), activation='softmax')(d)
# create model from all those layers :)
model = keras.models.Model(inputs=[ m1.input, m2.input], outputs=output)
This single model accepts the input from the two models : these two models are
First model - 2 input parameters tx[0] and tx[1]
Second model - 1 input parameter vx
and a common Y vy
Here's how I try to use them:
history = model.fit(
[
[ np.array(tx[0]), np.array(tx[1]) ],
np.array(vx)
],
np.array(vy), verbose = 1,
validation_data = (
[
[ np.array(txv[0]), np.array(txv[1]) ],
np.array(tvv)
],
np.array(vy)),
epochs = 1200,
batch_size = 128,
callbacks = [es, mcp_save])
So my first model has 2 parameters and my second one has one parameter. They both have a common y.
But the problem is that for some reason the first parameters end up being inputted for the second model. How to resolve that?

If I understand your model correctly, it has three inputs layers and you only mentioned two in your final model.
Use the variable name of each of the two input_layers of the first model instead of m1.input.
#the two input layers of the first Model m1:
inputA = tf.keras.Input(shape=(32,))
inputB = tf.keras.Input(shape=(128,))
#the input layer of the second Model m2:
inputC = tf.keras.Input(shape=(128,))
#after merging the two models m1 and m2 into one Model model :
model = keras.models.Model(inputs=[inputA, inputB , inputC], outputs=output)

Related

Sampling for large class and augmentation for small classes in each batch

Let's say we have 2 classes one is small and the second is large.
I would like to use for data augmentation similar to ImageDataGenerator
for the small class, and sampling from each batch, in such a way, that, that each batch would be balanced. (Fro minor class- augmentation for major class- sampling).
Also, I would like to continue using image_dataset_from_directory (since the dataset doesn't fit into RAM).
What about
sample_from_datasets
function?
import tensorflow as tf
from tensorflow.python.data.experimental import sample_from_datasets
def augment(val):
# Example of augmentation function
return val - tf.random.uniform(shape=tf.shape(val), maxval=0.1)
big_dataset_size = 1000
small_dataset_size = 10
# Init some datasets
dataset_class_large_positive = tf.data.Dataset.from_tensor_slices(tf.range(100, 100 + big_dataset_size, dtype=tf.float32))
dataset_class_small_negative = tf.data.Dataset.from_tensor_slices(-tf.range(1, 1 + small_dataset_size, dtype=tf.float32))
# Upsample and augment small dataset
dataset_class_small_negative = dataset_class_small_negative \
.repeat(big_dataset_size // small_dataset_size) \
.map(augment)
dataset = sample_from_datasets(
datasets=[dataset_class_large_positive, dataset_class_small_negative],
weights=[0.5, 0.5]
)
dataset = dataset.shuffle(100)
dataset = dataset.batch(6)
iterator = dataset.as_numpy_iterator()
for i in range(5):
print(next(iterator))
# [109. -10.044552 136. 140. -1.0505208 -5.0829906]
# [122. 108. 141. -4.0211563 126. 116. ]
# [ -4.085523 111. -7.0003924 -7.027302 -8.0362625 -4.0226436]
# [ -9.039093 118. -1.0695585 110. 128. -5.0553837]
# [100. -2.004463 -9.032592 -8.041705 127. 149. ]
Set up the desired balance between the classes in the weights parameter of sample_from_datasets.
As it was noticed by
Yaoshiang,
the last batches are imbalanced and the datasets length are different. This can be avoided by
# Repeat infinitely both datasets and augment the small one
dataset_class_large_positive = dataset_class_large_positive.repeat()
dataset_class_small_negative = dataset_class_small_negative.repeat().map(augment)
instead of
# Upsample and augment small dataset
dataset_class_small_negative = dataset_class_small_negative \
.repeat(big_dataset_size // small_dataset_size) \
.map(augment)
This case, however, the dataset is infinite and the number of batches in epoch has to be further controlled.
You can use tf.data.Dataset.from_generator that allows more control on your data generation without loading all your data into RAM.
def generator():
i=0
while True :
if i%2 == 0:
elem = large_class_sample()
else :
elem =small_class_augmented()
yield elem
i=i+1
ds= tf.data.Dataset.from_generator(
generator,
output_signature=(
tf.TensorSpec(shape=yourElem_shape , dtype=yourElem_ype))
This generator will alterate samples between the two classes,and you can add more dataset operations(batch , shuffle..)
I didn't totally follow the problem. Would psuedo-code this work? Perhaps there are some operators on tf.data.Dataset that are sufficient to solve your problem.
ds = image_dataset_from_directory(...)
ds1=ds.filter(lambda image, label: label == MAJORITY)
ds2=ds.filter(lambda image, label: label != MAJORITY)
ds2 = ds2.map(lambda image, label: data_augment(image), label)
ds1.batch(int(10. / MAJORITY_RATIO))
ds2.batch(int(10. / MINORITY_RATIO))
ds3 = ds1.zip(ds2)
ds3 = ds3.map(lambda left, right: tf.concat(left, right, axis=0)
You can use the tf.data.Dataset.from_tensor_slices to load the images of two categories seperately and do data augmentation for the minority class. Now that you have two datasets combine them with tf.data.Dataset.sample_from_datasets.
# assume class1 is the minority class
files_class1 = glob('class1\\*.jpg')
files_class2 = glob('class2\\*.jpg')
def augment(filepath):
class_name = tf.strings.split(filepath, os.sep)[0]
image = tf.io.read_file(filepath)
image = tf.expand_dims(image, 0)
if tf.equal(class_name, 'class1'):
# do all the data augmentation
image_flip = tf.image.flip_left_right(image)
return [[image, class_name],[image_flip, class_name]]
# apply data augmentation for class1
train_class1 = tf.data.Dataset.from_tensor_slices(files_class1).\
map(augment,num_parallel_calls=tf.data.AUTOTUNE)
train_class2 = tf.data.Dataset.from_tensor_slices(files_class2)
dataset = tf.python.data.experimental.sample_from_datasets(
datasets=[train_class1,train_class2],
weights=[0.5, 0.5])
dataset = dataset.batch(BATCH_SIZE)

Performing inference with a BERT (TF 1.x) saved model

I'm stuck on one line of code and have been stalled on a project all weekend as a result.
I am working on a project that uses BERT for sentence classification. I have successfully trained the model, and I can test the results using the example code from run_classifier.py.
I can export the model using this example code (which has been reposted repeatedly, so I believe that it's right for this model):
def export(self):
def serving_input_fn():
label_ids = tf.placeholder(tf.int32, [None], name='label_ids')
input_ids = tf.placeholder(tf.int32, [None, self.max_seq_length], name='input_ids')
input_mask = tf.placeholder(tf.int32, [None, self.max_seq_length], name='input_mask')
segment_ids = tf.placeholder(tf.int32, [None, self.max_seq_length], name='segment_ids')
input_fn = tf.estimator.export.build_raw_serving_input_receiver_fn({
'label_ids': label_ids, 'input_ids': input_ids,
'input_mask': input_mask, 'segment_ids': segment_ids})()
return input_fn
self.estimator._export_to_tpu = False
self.estimator.export_savedmodel(self.output_dir, serving_input_fn)
I can also load the exported estimator (where the export function saves the exported model into a subdirectory labeled with a timestamp):
predict_fn = predictor.from_saved_model(self.output_dir + timestamp_number)
However, for the life of me, I cannot figure out what to provide to predict_fn as input for inference. Here is my best code at the moment:
def predict(self):
input = 'Test input'
guid = 'predict-0'
text_a = tokenization.convert_to_unicode(input)
label = self.label_list[0]
examples = [InputExample(guid=guid, text_a=text_a, text_b=None, label=label)]
features = convert_examples_to_features(examples, self.label_list,
self.max_seq_length, self.tokenizer)
predict_input_fn = input_fn_builder(features, self.max_seq_length, False)
predict_fn = predictor.from_saved_model(self.output_dir + timestamp_number)
result = predict_fn(predict_input_fn) # this generates an error
print(result)
It doesn't seem to matter what I provide to predict_fn: the examples array, the features array, the predict_input_fn function. Clearly, predict_fn wants a dictionary of some type - but every single thing that I've tried generates an exception due to a tensor mismatch or other errors that generally mean: bad input.
I presumed that the from_saved_model function wants the same sort of input as the model test function - apparently, that's not the case.
It seems that lots of people have asked this very question - "how do I use an exported BERT TensorFlow model for inference?" - and have gotten no answers:
Thread #1
Thread #2
Thread #3
Thread #4
Any help? Thanks in advance.
Thank you for this post. Your serving_input_fn was the piece I was missing! Your predict function needs to be changed to feed the features dict directly, rather than use the predict_input_fn:
def predict(sentences):
labels = [0, 1]
input_examples = [
run_classifier.InputExample(
guid="",
text_a = x,
text_b = None,
label = 0
) for x in sentences] # here, "" is just a dummy label
input_features = run_classifier.convert_examples_to_features(
input_examples, labels, MAX_SEQ_LEN, tokenizer
)
# this is where pred_input_fn is replaced
all_input_ids = []
all_input_mask = []
all_segment_ids = []
all_label_ids = []
for feature in input_features:
all_input_ids.append(feature.input_ids)
all_input_mask.append(feature.input_mask)
all_segment_ids.append(feature.segment_ids)
all_label_ids.append(feature.label_id)
pred_dict = {
'input_ids': all_input_ids,
'input_mask': all_input_mask,
'segment_ids': all_segment_ids,
'label_ids': all_label_ids
}
predict_fn = predictor.from_saved_model('../testing/1589418540')
result = predict_fn(pred_dict)
print(result)
pred_sentences = [
"That movie was absolutely awful",
"The acting was a bit lacking",
"The film was creative and surprising",
"Absolutely fantastic!",
]
predict(pred_sentences)
{'probabilities': array([[-0.3579178 , -1.2010787 ],
[-0.36648935, -1.1814401 ],
[-0.30407643, -1.3386648 ],
[-0.45970002, -0.9982413 ],
[-0.36113673, -1.1936386 ],
[-0.36672896, -1.1808994 ]], dtype=float32), 'labels': array([0, 0, 0, 0, 0, 0])}
However, the probabilities returned for sentences in pred_sentences do not match the probabilities I get use estimator.predict(predict_input_fn) where estimator is the fine-tuned model being used within the same (python) session. For example, [-0.27276006, -1.4324446 ] using estimator vs [-0.26713806, -1.4505868 ] using predictor.

How to apply class weights in linear classifier for binary classification?

This is the linear classifier that I am using to perform binary classification, here is code snippet:
my_optimizer = tf.train.AdagradOptimizer(learning_rate = learning_rate)
my_optimizer = tf.contrib.estimator.clip_gradients_by_norm(my_optimizer,5.0)
# Create a linear classifier object
linear_classifier = tf.estimator.LinearClassifier(
feature_columns = feature_columns,
optimizer = my_optimizer
)
linear_classifier.train(input_fn = training_input_fn, steps = steps)
The dataset is imbalanced, there are only two classes yes/no. The number of NO class examples are 36548 while number of YES class examples are 4640.
How can I apply balancing to this data? I have been searching around and I could find stuff related to class weights etc but I couldn't find how can I create class weights and how to apply to the train method of tensor flow.
Here is how I am calculating losses:
training_probabilities = linear_classifier.predict(input_fn = training_predict_input_fn)
training_probabilities = np.array([item['probabilities'] for item in training_probabilities])
validation_probabilities = linear_classifier.predict(input_fn=validation_predict_input_fn)
validation_probabilities = np.array([item['probabilities'] for item in validation_probabilities])
training_log_loss = metrics.log_loss(training_targets, training_probabilities)
validation_log_loss = metrics.log_loss(validation_targets, validation_probabilities)
I assume that you are using the log_loss function from sklearn for computing your loss. If that is the case you can add class weights by using the argument sample_weight and pass on an array containing the weight to be given for each data point. sample_weight is an rolled out version of class_weights. You can compute sample_weight array by passing on the sample weights as given here.
Add the following lines to your code:
sample_wts = compute_sample_weight("balanced", training_targets)
training_log_loss = metrics.log_loss(training_targets, training_probabilities, sample_weight= sample_wts)

Output node for tensorflow graph created with tf.layers

I have built a tensorflow neural net and now want to run the graph_util.convert_variables_to_constants function on it. However this requires an output_node_names parameter. The last layer in the net has the name logit and is built as follows:
logits = tf.layers.dense(inputs=dropout, units=5, name='logit')
however there are many nodes in that scope:
gd = sess.graph_def
for n in gd.node:
if 'logit' in n.name:print(n.name)
prints:
logit/kernel/Initializer/random_uniform/shape
logit/kernel/Initializer/random_uniform/min
logit/kernel/Initializer/random_uniform/max
logit/kernel/Initializer/random_uniform/RandomUniform
logit/kernel/Initializer/random_uniform/sub
logit/kernel/Initializer/random_uniform/mul
logit/kernel/Initializer/random_uniform
logit/kernel
logit/kernel/Assign
logit/kernel/read
logit/bias/Initializer/zeros
logit/bias
logit/bias/Assign
logit/bias/read
logit/Tensordot/Shape
logit/Tensordot/Rank
logit/Tensordot/axes
...
logit/Tensordot/Reshape_1
logit/Tensordot/MatMul
logit/Tensordot/Const_2
logit/Tensordot/concat_2/axis
logit/Tensordot/concat_2
logit/Tensordot
logit/BiasAdd
...
How do I work out which of these nodes is the output node?
If the graph is complex, a common way is to add an identity node at the end:
output = tf.identity(logits, 'output')
# you can use the name "output"
For example, the following code should work:
logits = tf.layers.dense(inputs=dropout, units=5, name='logit')
output = tf.identity(logits, 'output')
output_graph_def = tf.graph_util.convert_variables_to_constants(
ss, tf.get_default_graph().as_graph_def(), ['output'])

Which variables to pass to a tensor flow predictor for tf.feature_coloumns using wide and deep learning model?

I've been trying out the wide and deep learning example from the tensor flow site: https://www.tensorflow.org/tutorials/wide_and_deep
I can train and evaluate the model and even predict within that same process but I can't seem to figure out what input I need to pass into the predictor function when I try to do a prediction from a model that was saved and then reloaded via the predictor.from_saved_model function.
My feature columns and model look like this which runs fine:
term = tf.feature_column.categorical_column_with_vocabulary_list("term", unique_terms['term'].tolist())
name = tf.feature_column.categorical_column_with_vocabulary_list("name", unique_name['name'].tolist())
base_columns = [term, cust_name]
crossed_columns = [
tf.feature_column.crossed_column(["term", "cust_name"], hash_bucket_size=100000),
]
deep_columns = [
tf.feature_column.indicator_column(term),
tf.feature_column.indicator_column(cust_name),
]
model_dir = export_dir
search_model = tf.estimator.DNNLinearCombinedClassifier(
model_dir=model_dir,
linear_feature_columns=crossed_columns,
dnn_feature_columns=deep_columns,
dnn_hidden_units=[100, 50])
I saved the model like this:
feature_columns = crossed_columns + deep_columns
feature_spec = tf.feature_column.make_parse_example_spec(feature_columns)
export_input_fn = tf.estimator.export.build_parsing_serving_input_receiver_fn(feature_spec)
servable_model_dir = export_dir
servable_model_path = search_model.export_savedmodel(servable_model_dir, export_input_fn)
And then I load it back from file like this:
predict_fn = predictor.from_saved_model(export_dir)
predictions = predict_fn({'X':[10]})
The predict_fn is expecting a dictionary with key "inputs" not "x" except I am not sure what the value of "inputs" should be. Can anyone help me out on this please?