I'm new to Keras and I intend to store the output of my network for every epoch. To that end, I want to use Tensorbaord to observe the output layer in its environment.
class OutputObserver(Callback):
""""
callback to observe the output of the network
"""
def on_train_begin(self, logs={}):
self.epoch = []
self.out_log = []
def on_epoch_end(self, epoch, logs={}):
self.epoch.append(epoch)
self.out_log.append(self.model.get_layer('Dense03/Output').output)
This will store the outputs tensors into a list. The problem is that I can do neither 1. convert this to a Numpy array so that can be read a CSV, ... file, 2. write a summary using Tensorflow (as Keras does not have this ability) and then analyze the output in Tensorboard.
I would be pleased to hear your opinions on storing and visualizing the output layer in every epoch of training.
Sincerely,
Saeed.
To save the output layer for every epoch you need to pass the training/validation data to the callback object. The callback I used is as follow,
class OutputObserver(Callback):
""""
callback to observe the output of the network
"""
def __init__(self, xy):
self.out_log = []
self.xy = xy
def on_epoch_end(self, epoch, logs={}):
self.out_log.append(self.model.predict(self.xy.x_train, batch_size=p.bath_size))
which xy.x_train is the training data.
now, the out_log array is a numpy.ndarray of shape (epoch_number, data_number, prediction_length):
type(prediction_logs[0])
Out[62]: numpy.ndarray
Related
I would like to train a keras model (say a simple FFNN) using the model.fit() method and not doing it 'by hand' (i.e. by using the gradient.tape method explained for example here). However, the loss function I need to use is quite elaborated and cannot be computed on randomly generated batches of data. As a result, I need to train the model using batches of data computed 'by hand' (i.e. the data that goes into each batch needs to have certain properties and cannot be randomly assigned).
Can I pass somehow pre-computed batches to the fit() method?
One solution consists in sub-classing the Tensorflow Sequence. You can create your own batch for a given index using the __getitem__ method.
class MySequence(tf.keras.utils.Sequence):
def __init__(self, x_batch, y_batch) -> None:
super().__init__()
self.x_batch = x_batch # ordered list of batches
self.y_batch = y_batch # idem
self.leny = len(y_batch)
def __len__(self):
return self.leny
def __getitem__(self, idx):
x = self.x_batch[idx]
y = self.y_batch[idx]
return x, y
You can pass of an instance of this Sequence sub-class to the Model fit method.
Also set shuffle=False in the Model fit arguments.
Can I plot accuracy, loss... if I didn't assign model.fit to something.
I just wrote model.fit and trained the model.
Thanks
There are 2 ways to do this without a history in keras:
take the text output of the keras training and manually take the loss values of every epoch and do the plot by hand by filling 2 numpy arrays with the values ( one for the loss and an other for the validation loss).
This seems a long task but with a text editor like visual studio code it's a matter of seconds.
write a callback that after every epoch writes the result of the epoch on an external text file and then take the values in a similar way as point 1.
something like this:
class print_log_Callback(Callback):
def __init__(self, logpath, steps):
self.logpath = logpath
self.losslst = np.zeros(steps)
def on_epoch_end(self, epoch, logs=None):
with open(logpath, 'a') as writefile:
with redirect_stdout(writefile):
print("The average loss for epoch {} is {:7.2f} and val_loss is {:7.2f}.".format(epoch, logs["loss"], logs['val_loss']))
writefile.write("\n")
print("The mean train loss is: ", np.mean(self.losslst))
writefile.write("\n")
writefile.write("\n")
I'm want to train my model with Keras. I'm using a huge dataset Where one training epoch has more than 30000 steps. My problem is that I don't want to wait for an epoch before checking the model improvement on the validation dataset. Is there any way to make Keras evaluate the validation data every 1000 steps of the training data? I think one option will be to use a callback but is there any built-in solution with Keras?
if train:
log('Start training')
history = model.fit(train_dataset,
steps_per_epoch=train_steps,
epochs=50,
validation_data=val_dataset,
validation_steps=val_steps,
callbacks=[
keras.callbacks.EarlyStopping(
monitor='loss',
patience=10,
restore_best_weights=True,
),
keras.callbacks.ModelCheckpoint(
filepath=f'model.h5',
monitor='val_loss',
save_best_only=True,
save_weights_only=True,
),
keras.callbacks.ReduceLROnPlateau(
monitor = "val_loss",
factor = 0.5,
patience = 3,
min_lr=0.001,
),
],
)
With the in-built callbacks, you cannot do that. What you need is to implement a custom callback.
class MyCustomCallback(tf.keras.callbacks.Callback):
def on_train_batch_begin(self, batch, logs=None):
print('Training: batch {} begins at {}'.format(batch, datetime.datetime.now().time()))
def on_train_batch_end(self, batch, logs=None):
print('Training: batch {} ends at {}'.format(batch, datetime.datetime.now().time()))
def on_test_batch_begin(self, batch, logs=None):
print('Evaluating: batch {} begins at {}'.format(batch, datetime.datetime.now().time()))
def on_test_batch_end(self, batch, logs=None):
print('Evaluating: batch {} ends at {}'.format(batch, datetime.datetime.now().time()))
This is taken from the TensorFlow documentation.
You can override the on_train_batch_end() function and, since the batch parameter is an integer, you can verify batch % 100 == 0, then self.model.predict(val_data) etc. to your needs.
Please check my answer here: How to get other metrics in Tensorflow 2.0 (not only accuracy)? to have a good overview on how to override a custom callback function. Please note that in your case it is the on_train_batch_end() not on_epoch_end() that is important.
Below is my custom callback function that I wrote but it doesn't work:
class bestval(tf.keras.callbacks.Callback):
def on_train_begin(self, logs={}):
self.history={'loss': [],'acc': [],'val_loss': [],'val_acc': []}
def on_epoch_end(self, epoch, logs={}):
#appending val_acc in history
if logs.get('val_acc', -1) != -1:
self.history['val_acc'].append(logs.get('val_acc'))
# Trying to compare current epoch val_acc with all the values in self.history['val_acc']
if logs.get('val_acc')> [i for i in self.history['val_acc']]:
filepath="model_save/weights-{epoch:02d}-{val_acc:.4f}.hdf5"
# Saving the model using TF built-in callback
checkpoint = tensorflow.keras.callbacks.ModelCheckpoint(filepath=filepath,
monitor='val_acc', verbose=1, mode='auto')
bestobj= bestval()
Fitting the model:
model.fit(xtr,ytr, epochs=4, validation_data=(xte,yte), batch_size=128, callbacks=[bestobj])
When I run the above I get the below error:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
I know I am doing something stupid but I don't know how to fix. Any help would be appreciated.
I guess the error is in following line, you are trying to compare a value with a list.
if logs.get('val_acc')> [i for i in self.history['val_acc']]:
try,
for i in self.history['val_acc']:
if logs.get('val_acc')>i:
#your code
Instead of
if logs.get('val_acc') > [i for i in self.history['val_acc']]
Use
if any(logs.get('val_acc')> val for val in self.history['val_acc'])
I want to add to my model a layer that, during evaluation, takes the input, applies some transformations (a quantization in this case, but can be whatever) and return it as the output. This layer must, however, be completely transparent during training, meaning that it must return the same input tensor.
I have written the following function
from keras.layers import Lambda
import keras.backend as K
def myquantize(x):
return K.in_test_phase( K.clip(K.round(x*(2**5))/(2**5),-3.9,3.9) , x)
which I then use via a Lambda layer
y = keras.layers.Conv1D(**args1)
y = keras.layers.AveragePooling1D(pool_size=2)(y)
y = keras.layers.Lambda(myquantize)(y)
y = keras.layers.Conv1D(**args2)
#...
Now, in principle the K.in_test_phase should return x during training, and that expression during test.
However, training the network with such layer prevent the network from learning (i.e. the train loss stops decreasing after 3 epochs), while if I remove it the network keeps training normally. I assume this layer is not actually transparent during training as expected.
in_test_phase has a parameter of training which you can explicitly set to indicate whether you are training or not. If you don't set it explicitly, then the value of learning_phase is used. This value keeps changing when you reset the graph or when you call different types of fit/predict/evaluate functions of model.
Since your full code isn't present, you can make use of training parameter. Set it to True during training. Then save the weights of the model using save_weights function of model. When you wish to test your model, set the training parameter to False. Then load the weights using load_weights function and then you can proceed accordingly.
For those who are in a similar situation, I created a custom layer like the following, which I only use during training:
class MyLayer(keras.layers.Layer):
def __init__(self, **kwargs):
super(MyLayer, self).__init__(**kwargs)
def compute_output_shape(self, input_shape):
return input_shape
def call(self, inputs, **kwargs):
x=inputs
return K.identity(x)
note that this layer always returns the input tensor, but it serves as 'placeholder' for the next step. On the evaluation part of the code, I wrote the following code:
class MyLayer(keras.layers.Layer):
def __init__(self, **kwargs):
super(MyLayer, self).__init__(**kwargs)
def compute_output_shape(self, input_shape):
return input_shape
def call(self, inputs, **kwargs):
x=inputs
return #Your actual processing here
Here, the only difference is that you actually perform the desired processing steps on your tensor. When I load my stored model, I pass this class as custom object
model = keras.models.load_model(model_file,custom_objects={'MyLayer':MyLayer})
be careful to pass as MyLayer the one where the actual processing is performed.
This is my solution, other suggestions are welcome