I'm trying to Build and LSTM model for intent classification using Tensorflow, Keras. But whenever I'm training the model with 30 or 40 epochs, my 1st 20 validation accuracy is zero and loss is more than accuracy. and if I try to change the code a little bit, validation accuracy is getting lower than Loss.
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
embedding (Embedding) (None, 100, 16) 16000
_________________________________________________________________
bidirectional (Bidirectional (None, 64) 12544
_________________________________________________________________
dense (Dense) (None, 24) 1560
_________________________________________________________________
dense_1 (Dense) (None, 3) 75
=================================================================
Total params: 30,179
Trainable params: 30,179
Non-trainable params: 0
_________________________________________________________________
Train on 200 samples, validate on 79 samples
loss='sparse_categorical_crossentropy',optimizer='adam',metrics=['accuracy']
Epoch 1/40
2020-08-16 14:08:17.986786: W tensorflow/core/grappler/optimizers/implementation_selector.cc:310] Skipping optimization due to error while loading function libraries: Invalid argument: Functions '__inference___backward_standard_lstm_4932_5417' and '__inference___backward_standard_lstm_4932_5417_specialized_for_StatefulPartitionedCall_at___inference_distributed_function_6117' both implement 'lstm_6b97c168-2c7b-4dc3-93dd-5e68cddc574f' but their signatures do not match.
2020-08-16 14:08:20.550894: W tensorflow/core/grappler/optimizers/implementation_selector.cc:310] Skipping optimization due to error while loading function libraries: Invalid argument: Functions '__inference_standard_lstm_6338' and '__inference_standard_lstm_6338_specialized_for_sequential_bidirectional_forward_lstm_StatefulPartitionedCall_at___inference_distributed_function_7185' both implement 'lstm_62f19cc7-a1f0-447b-a17f-84a70fc095cd' but their signatures do not match.
200/200 - 10s - loss: 1.0798 - accuracy: 0.7850 - val_loss: 1.1327 - val_accuracy: 0.0000e+00
Epoch 2/40
200/200 - 1s - loss: 1.0286 - accuracy: 0.7850 - val_loss: 1.1956 - val_accuracy: 0.0000e+00
Epoch 3/40
200/200 - 1s - loss: 0.9294 - accuracy: 0.7850 - val_loss: 1.4287 - val_accuracy: 0.0000e+00
Epoch 4/40
200/200 - 1s - loss: 0.7026 - accuracy: 0.7850 - val_loss: 2.2190 - val_accuracy: 0.0000e+00
Epoch 5/40
200/200 - 1s - loss: 0.6183 - accuracy: 0.7850 - val_loss: 1.8499 - val_accuracy: 0.0000e+00
Epoch 6/40
200/200 - 1s - loss: 0.5980 - accuracy: 0.7850 - val_loss: 1.5809 - val_accuracy: 0.0000e+00
Epoch 7/40
200/200 - 1s - loss: 0.5927 - accuracy: 0.7850 - val_loss: 1.5118 - val_accuracy: 0.0000e+00
Epoch 8/40
200/200 - 1s - loss: 0.5861 - accuracy: 0.7850 - val_loss: 1.5711 - val_accuracy: 0.0000e+00
Epoch 9/40
200/200 - 1s - loss: 0.5728 - accuracy: 0.7850 - val_loss: 1.5106 - val_accuracy: 0.0000e+00
Epoch 10/40
200/200 - 1s - loss: 0.5509 - accuracy: 0.7850 - val_loss: 1.6389 - val_accuracy: 0.0000e+00
Epoch 11/40
200/200 - 1s - loss: 0.5239 - accuracy: 0.7850 - val_loss: 1.5991 - val_accuracy: 0.0000e+00
Epoch 12/40
200/200 - 1s - loss: 0.4860 - accuracy: 0.7850 - val_loss: 1.4903 - val_accuracy: 0.0000e+00
Epoch 13/40
200/200 - 1s - loss: 0.4388 - accuracy: 0.7850 - val_loss: 1.3937 - val_accuracy: 0.0000e+00
Epoch 14/40
200/200 - 1s - loss: 0.3859 - accuracy: 0.7850 - val_loss: 1.2329 - val_accuracy: 0.0000e+00
Epoch 15/40
200/200 - 1s - loss: 0.3460 - accuracy: 0.7850 - val_loss: 1.1700 - val_accuracy: 0.0000e+00
Epoch 16/40
200/200 - 1s - loss: 0.3323 - accuracy: 0.7850 - val_loss: 1.0077 - val_accuracy: 0.0127
Epoch 17/40
200/200 - 1s - loss: 0.3007 - accuracy: 0.8150 - val_loss: 1.2465 - val_accuracy: 0.2278
Epoch 18/40
200/200 - 0s - loss: 0.2752 - accuracy: 0.9200 - val_loss: 0.8890 - val_accuracy: 0.6329
Epoch 19/40
200/200 - 1s - loss: 0.2613 - accuracy: 0.9700 - val_loss: 0.9181 - val_accuracy: 0.6582
Epoch 20/40
200/200 - 1s - loss: 0.2447 - accuracy: 0.9600 - val_loss: 0.8786 - val_accuracy: 0.7468
Epoch 21/40
200/200 - 1s - loss: 0.2171 - accuracy: 0.9700 - val_loss: 0.7162 - val_accuracy: 0.8481
Epoch 22/40
200/200 - 1s - loss: 0.1949 - accuracy: 0.9700 - val_loss: 0.8051 - val_accuracy: 0.7848
Epoch 23/40
200/200 - 1s - loss: 0.1654 - accuracy: 0.9700 - val_loss: 0.4710 - val_accuracy: 0.8861
Epoch 24/40
200/200 - 1s - loss: 0.1481 - accuracy: 0.9700 - val_loss: 0.4209 - val_accuracy: 0.8861
Epoch 25/40
200/200 - 1s - loss: 0.1192 - accuracy: 0.9700 - val_loss: 0.3792 - val_accuracy: 0.8861
Epoch 26/40
200/200 - 1s - loss: 0.1022 - accuracy: 0.9700 - val_loss: 0.7279 - val_accuracy: 0.8101
Epoch 27/40
200/200 - 1s - loss: 0.0995 - accuracy: 0.9700 - val_loss: 1.3112 - val_accuracy: 0.6582
Epoch 28/40
200/200 - 1s - loss: 0.1161 - accuracy: 0.9650 - val_loss: 0.1435 - val_accuracy: 0.9747
Epoch 29/40
200/200 - 1s - loss: 0.0889 - accuracy: 0.9700 - val_loss: 0.3896 - val_accuracy: 0.8608
Epoch 30/40
200/200 - 1s - loss: 0.0830 - accuracy: 0.9700 - val_loss: 0.3840 - val_accuracy: 0.8608
Epoch 31/40
200/200 - 1s - loss: 0.0688 - accuracy: 0.9700 - val_loss: 0.3100 - val_accuracy: 0.9241
Epoch 32/40
200/200 - 1s - loss: 0.0611 - accuracy: 0.9700 - val_loss: 0.3524 - val_accuracy: 0.8987
Epoch 33/40
200/200 - 1s - loss: 0.0518 - accuracy: 0.9750 - val_loss: 0.4621 - val_accuracy: 0.8481
Epoch 34/40
200/200 - 1s - loss: 0.0457 - accuracy: 0.9900 - val_loss: 0.4344 - val_accuracy: 0.8481
Epoch 35/40
200/200 - 1s - loss: 0.0423 - accuracy: 0.9900 - val_loss: 0.4417 - val_accuracy: 0.8608
Epoch 36/40
200/200 - 1s - loss: 0.0372 - accuracy: 0.9900 - val_loss: 0.4701 - val_accuracy: 0.8481
Epoch 37/40
200/200 - 1s - loss: 0.0319 - accuracy: 0.9950 - val_loss: 0.3913 - val_accuracy: 0.8608
Epoch 38/40
200/200 - 1s - loss: 0.0309 - accuracy: 0.9950 - val_loss: 0.5739 - val_accuracy: 0.7975
Epoch 39/40
200/200 - 1s - loss: 0.0290 - accuracy: 0.9950 - val_loss: 0.5416 - val_accuracy: 0.8228
Epoch 40/40
200/200 - 1s - loss: 0.0292 - accuracy: 1.0000 - val_loss: 0.3162 - val_accuracy: 0.8861
There can be multiple reasons for the validation accuracy to be zero, you can check on these below things to make changes accordingly.
The samples you had taken are very less, train on 200 samples and validate on 79 samples, you can try increasing samples through some upsampling methods.
There is a possibility that there can be unseen data on validation and weights are not useful with unseen data.
For example, if you have digits from 1-9 to classify, if you keep 1-7 for training and 8-9 for validation this data becomes an unseen scenario and val_acc will be 0.
To fix this, shuffle the data before sending it to model.fit.
This can be a silly mistake, check if your validation data is mapped properly to its label.
You can also give validation data explicitly like below.
model.fit(X_train, y_train, batch_size = 32, nb_epoch=30,
shuffle=True,verbose=1,callbacks=[remote, early_stopping],
validation_data (X_validation, y_validation))
Related
i've been working on a multiclass classification neural network model on Rstudio with KERAS and Tensorflow where i am trying to achieve at least 70 to 80% accuracy because i had tried tuning the hyperparameters to increase the accuracy but no matter how much that i had tuned on number of epochs, number of layers, or even changing on batch size, both training and validation accuracy doesn't seem to be increasing and training and validation loss stuck at a certain loss value.
this is my model that im working on:
model %>%
layer_dense(units = 28, activation = 'relu',input_shape = c(24), kernel_regularizer = regularizer_l2(0.0001))%>%
layer_dropout(0.01)%>%
layer_dense(units = 28, activation = 'relu', kernel_regularizer = regularizer_l2(0.0001))%>%
layer_dropout(0.01)%>%
layer_dense(units = 28, activation = 'relu', kernel_regularizer = regularizer_l2(0.0001))%>%
layer_dropout(0.01)%>%
layer_dense(units = 28, activation = 'relu', kernel_regularizer = regularizer_l2(0.0001))%>%
layer_dropout(0.01)%>%
layer_dense(units = 28, activation = 'relu', kernel_regularizer = regularizer_l2(0.0001))%>%
layer_dropout(0.01)%>%
layer_dense(units = 28, activation = 'relu', kernel_regularizer = regularizer_l2(0.0001))%>%
layer_dropout(0.01)%>%
layer_dense(units = 8, activation = 'softmax', kernel_regularizer = regularizer_l2(0.0001))
model %>%
compile(loss = 'categorical_crossentropy', optimizer = 'Adam', metrics = 'accuracy')
history <- model%>%
fit(training, trainlabels, epoch = 12000, batch_size = 64, validation_split = 0.2, shuffle = TRUE )
and this is the epoch progression:
Epoch 11900/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3010 - accuracy: 0.5000 - val_loss: 1.3180 - val_accuracy: 0.4823
Epoch 11901/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3008 - accuracy: 0.5007 - val_loss: 1.3326 - val_accuracy: 0.4694
Epoch 11902/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2999 - accuracy: 0.4961 - val_loss: 1.3188 - val_accuracy: 0.4929
Epoch 11903/12000
326/326 [==============================] - 1s 3ms/step - loss: 1.2983 - accuracy: 0.4945 - val_loss: 1.3154 - val_accuracy: 0.4860
Epoch 11904/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3040 - accuracy: 0.4973 - val_loss: 1.3030 - val_accuracy: 0.4887
Epoch 11905/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2904 - accuracy: 0.5021 - val_loss: 1.3125 - val_accuracy: 0.4896
Epoch 11906/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2956 - accuracy: 0.4947 - val_loss: 1.3187 - val_accuracy: 0.4814
Epoch 11907/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3049 - accuracy: 0.4953 - val_loss: 1.3355 - val_accuracy: 0.4727
Epoch 11908/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2978 - accuracy: 0.5025 - val_loss: 1.3250 - val_accuracy: 0.4864
Epoch 11909/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2958 - accuracy: 0.4979 - val_loss: 1.3273 - val_accuracy: 0.4850
Epoch 11910/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2991 - accuracy: 0.4994 - val_loss: 1.3150 - val_accuracy: 0.4864
Epoch 11911/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3040 - accuracy: 0.5006 - val_loss: 1.3458 - val_accuracy: 0.4775
Epoch 11912/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3066 - accuracy: 0.4969 - val_loss: 1.3051 - val_accuracy: 0.4842
Epoch 11913/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3029 - accuracy: 0.4988 - val_loss: 1.3239 - val_accuracy: 0.4885
Epoch 11914/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2890 - accuracy: 0.5040 - val_loss: 1.3321 - val_accuracy: 0.4719
Epoch 11915/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2987 - accuracy: 0.4969 - val_loss: 1.3495 - val_accuracy: 0.4708
Epoch 11916/12000
326/326 [==============================] - 1s 3ms/step - loss: 1.2952 - accuracy: 0.4981 - val_loss: 1.3177 - val_accuracy: 0.4779
Epoch 11917/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2980 - accuracy: 0.4977 - val_loss: 1.3104 - val_accuracy: 0.4904
Epoch 11918/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2985 - accuracy: 0.4990 - val_loss: 1.3220 - val_accuracy: 0.4850
Epoch 11919/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2940 - accuracy: 0.4988 - val_loss: 1.2994 - val_accuracy: 0.4898
Epoch 11920/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3000 - accuracy: 0.5001 - val_loss: 1.3183 - val_accuracy: 0.4865
Epoch 11921/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2957 - accuracy: 0.4991 - val_loss: 1.3215 - val_accuracy: 0.4841
Epoch 11922/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2930 - accuracy: 0.5024 - val_loss: 1.3205 - val_accuracy: 0.4823
Epoch 11923/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3026 - accuracy: 0.4972 - val_loss: 1.3034 - val_accuracy: 0.4864
Epoch 11924/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2943 - accuracy: 0.4995 - val_loss: 1.3423 - val_accuracy: 0.4812
Epoch 11925/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3033 - accuracy: 0.4998 - val_loss: 1.3082 - val_accuracy: 0.4875
Epoch 11926/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3032 - accuracy: 0.4977 - val_loss: 1.3241 - val_accuracy: 0.4806
Epoch 11927/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2988 - accuracy: 0.4975 - val_loss: 1.3132 - val_accuracy: 0.4827
Epoch 11928/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3064 - accuracy: 0.4952 - val_loss: 1.3422 - val_accuracy: 0.4794
Epoch 11929/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2988 - accuracy: 0.4953 - val_loss: 1.3279 - val_accuracy: 0.4892
Epoch 11930/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3015 - accuracy: 0.4994 - val_loss: 1.3181 - val_accuracy: 0.4779
Epoch 11931/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2967 - accuracy: 0.4990 - val_loss: 1.3400 - val_accuracy: 0.4858
Epoch 11932/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3042 - accuracy: 0.4968 - val_loss: 1.3152 - val_accuracy: 0.4867
Epoch 11933/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3052 - accuracy: 0.4958 - val_loss: 1.3246 - val_accuracy: 0.4864
Epoch 11934/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2974 - accuracy: 0.4971 - val_loss: 1.3155 - val_accuracy: 0.4841
Epoch 11935/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2999 - accuracy: 0.4975 - val_loss: 1.3182 - val_accuracy: 0.4940
Epoch 11936/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3013 - accuracy: 0.5006 - val_loss: 1.3594 - val_accuracy: 0.4660
Epoch 11937/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2975 - accuracy: 0.4955 - val_loss: 1.3295 - val_accuracy: 0.4779
Epoch 11938/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2925 - accuracy: 0.4983 - val_loss: 1.3597 - val_accuracy: 0.4731
Epoch 11939/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3017 - accuracy: 0.4991 - val_loss: 1.3321 - val_accuracy: 0.4854
Epoch 11940/12000
326/326 [==============================] - 1s 3ms/step - loss: 1.2992 - accuracy: 0.4985 - val_loss: 1.3251 - val_accuracy: 0.4831
Epoch 11941/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3023 - accuracy: 0.4955 - val_loss: 1.3430 - val_accuracy: 0.4781
Epoch 11942/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2984 - accuracy: 0.5010 - val_loss: 1.3113 - val_accuracy: 0.4933
Epoch 11943/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3000 - accuracy: 0.4972 - val_loss: 1.3267 - val_accuracy: 0.4783
Epoch 11944/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2963 - accuracy: 0.4989 - val_loss: 1.3713 - val_accuracy: 0.4671
Epoch 11945/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3088 - accuracy: 0.4945 - val_loss: 1.3122 - val_accuracy: 0.4862
Epoch 11946/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3062 - accuracy: 0.4965 - val_loss: 1.3142 - val_accuracy: 0.4802
Epoch 11947/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3005 - accuracy: 0.4993 - val_loss: 1.3279 - val_accuracy: 0.4852
Epoch 11948/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3069 - accuracy: 0.4953 - val_loss: 1.3111 - val_accuracy: 0.4873
Epoch 11949/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3035 - accuracy: 0.4966 - val_loss: 1.3289 - val_accuracy: 0.4867
Epoch 11950/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3024 - accuracy: 0.4963 - val_loss: 1.3416 - val_accuracy: 0.4816
Epoch 11951/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2966 - accuracy: 0.5009 - val_loss: 1.3485 - val_accuracy: 0.4727
Epoch 11952/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2989 - accuracy: 0.4978 - val_loss: 1.3178 - val_accuracy: 0.4829
Epoch 11953/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2961 - accuracy: 0.4982 - val_loss: 1.3267 - val_accuracy: 0.4785
Epoch 11954/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3022 - accuracy: 0.4989 - val_loss: 1.3302 - val_accuracy: 0.4823
Epoch 11955/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2982 - accuracy: 0.4957 - val_loss: 1.3090 - val_accuracy: 0.4839
Epoch 11956/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2947 - accuracy: 0.5020 - val_loss: 1.3337 - val_accuracy: 0.4791
Epoch 11957/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2881 - accuracy: 0.4985 - val_loss: 1.3179 - val_accuracy: 0.4817
Epoch 11958/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2929 - accuracy: 0.5023 - val_loss: 1.3211 - val_accuracy: 0.4792
Epoch 11959/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2977 - accuracy: 0.4980 - val_loss: 1.3222 - val_accuracy: 0.4800
Epoch 11960/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2992 - accuracy: 0.4972 - val_loss: 1.3390 - val_accuracy: 0.4769
Epoch 11961/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3014 - accuracy: 0.5003 - val_loss: 1.3238 - val_accuracy: 0.4856
Epoch 11962/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3005 - accuracy: 0.4960 - val_loss: 1.3300 - val_accuracy: 0.4789
Epoch 11963/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3025 - accuracy: 0.4987 - val_loss: 1.3037 - val_accuracy: 0.4802
Epoch 11964/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2983 - accuracy: 0.4969 - val_loss: 1.3381 - val_accuracy: 0.4792
Epoch 11965/12000
326/326 [==============================] - 1s 3ms/step - loss: 1.2969 - accuracy: 0.4982 - val_loss: 1.3007 - val_accuracy: 0.4871
Epoch 11966/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3018 - accuracy: 0.4967 - val_loss: 1.3417 - val_accuracy: 0.4837
Epoch 11967/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2987 - accuracy: 0.4970 - val_loss: 1.3202 - val_accuracy: 0.4860
Epoch 11968/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2931 - accuracy: 0.5005 - val_loss: 1.3095 - val_accuracy: 0.4871
Epoch 11969/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2977 - accuracy: 0.4967 - val_loss: 1.3038 - val_accuracy: 0.4860
Epoch 11970/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2956 - accuracy: 0.4974 - val_loss: 1.3440 - val_accuracy: 0.4833
Epoch 11971/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3007 - accuracy: 0.5018 - val_loss: 1.3378 - val_accuracy: 0.4777
Epoch 11972/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2973 - accuracy: 0.4972 - val_loss: 1.3173 - val_accuracy: 0.4848
Epoch 11973/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2901 - accuracy: 0.4974 - val_loss: 1.3145 - val_accuracy: 0.4814
Epoch 11974/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2952 - accuracy: 0.4995 - val_loss: 1.3260 - val_accuracy: 0.4850
Epoch 11975/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2976 - accuracy: 0.4982 - val_loss: 1.3476 - val_accuracy: 0.4677
Epoch 11976/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2979 - accuracy: 0.5014 - val_loss: 1.3270 - val_accuracy: 0.4754
Epoch 11977/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2991 - accuracy: 0.5001 - val_loss: 1.3295 - val_accuracy: 0.4779
Epoch 11978/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2974 - accuracy: 0.4973 - val_loss: 1.3053 - val_accuracy: 0.4931
Epoch 11979/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2963 - accuracy: 0.4958 - val_loss: 1.3156 - val_accuracy: 0.4919
Epoch 11980/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2962 - accuracy: 0.5031 - val_loss: 1.3201 - val_accuracy: 0.4791
Epoch 11981/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2908 - accuracy: 0.4987 - val_loss: 1.3110 - val_accuracy: 0.4881
Epoch 11982/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3048 - accuracy: 0.4988 - val_loss: 1.3283 - val_accuracy: 0.4798
Epoch 11983/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3063 - accuracy: 0.4973 - val_loss: 1.3170 - val_accuracy: 0.4821
Epoch 11984/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3019 - accuracy: 0.4958 - val_loss: 1.3084 - val_accuracy: 0.4835
Epoch 11985/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2981 - accuracy: 0.4977 - val_loss: 1.3381 - val_accuracy: 0.4696
Epoch 11986/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3006 - accuracy: 0.4969 - val_loss: 1.3087 - val_accuracy: 0.4827
Epoch 11987/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3093 - accuracy: 0.4911 - val_loss: 1.3154 - val_accuracy: 0.4889
Epoch 11988/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2981 - accuracy: 0.5002 - val_loss: 1.3164 - val_accuracy: 0.4827
Epoch 11989/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3022 - accuracy: 0.4938 - val_loss: 1.3202 - val_accuracy: 0.4808
Epoch 11990/12000
326/326 [==============================] - 1s 3ms/step - loss: 1.2953 - accuracy: 0.4960 - val_loss: 1.3113 - val_accuracy: 0.4908
Epoch 11991/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2984 - accuracy: 0.4963 - val_loss: 1.3170 - val_accuracy: 0.4839
Epoch 11992/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2917 - accuracy: 0.5012 - val_loss: 1.3479 - val_accuracy: 0.4769
Epoch 11993/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3008 - accuracy: 0.4955 - val_loss: 1.3423 - val_accuracy: 0.4833
Epoch 11994/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3007 - accuracy: 0.4984 - val_loss: 1.3168 - val_accuracy: 0.4850
Epoch 11995/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3020 - accuracy: 0.4966 - val_loss: 1.3266 - val_accuracy: 0.4754
Epoch 11996/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2979 - accuracy: 0.4993 - val_loss: 1.3091 - val_accuracy: 0.4808
Epoch 11997/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2962 - accuracy: 0.5026 - val_loss: 1.3339 - val_accuracy: 0.4792
Epoch 11998/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3100 - accuracy: 0.4973 - val_loss: 1.3150 - val_accuracy: 0.4833
Epoch 11999/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.3036 - accuracy: 0.4978 - val_loss: 1.3295 - val_accuracy: 0.4819
Epoch 12000/12000
326/326 [==============================] - 1s 2ms/step - loss: 1.2901 - accuracy: 0.5007 - val_loss: 1.3061 - val_accuracy: 0.4802
but after all the trainings, the scores concludes
loss: 1.3303 - accuracy: 0.4683
the accuracy and the loss is as shown below:
accuracy of the model
loss of the model
Loss and Accuracy
am i underfitting the training model?? how can i actually increase the accuracy and letting the model to learn something?
Please advise how to confirm if ReduceLROnPlateau has been actually applied and what learning rates have been applied at each epoch.
The patience of ReduceLROnPlateau is set to 2 and monitor var_loss.
self._history = self.model.fit(
self.X.shuffle(1000).batch(self.batch_size).prefetch(1),
epochs=self.num_epochs,
batch_size=self.batch_size,
validation_data=self.V.shuffle(1000).batch(self.batch_size).prefetch(1),
callbacks=[
EaryStoppingCallback(patience=self.early_stop_patience),
ReduceLRCallback(patience=self.reduce_lr_patience), # <---- set to 2
TensorBoardCallback(self.log_directory)
]
)
class ReduceLRCallback(tf.keras.callbacks.ReduceLROnPlateau):
"""Reduce learning rate when a metric has stopped improving.
See https://keras.io/api/callbacks/reduce_lr_on_plateau/
"""
def __init__(self, patience=3):
assert patience > 0
super().__init__(
monitor="val_loss",
factor=0.3,
patience=patience,
)
The training val_loss have increased more than twice but have not seen any information if ReduceLROnPlateau has been applied.
_________________________________________________________________
Epoch 1/20
3990/3990 [==============================] - 860s 214ms/step - loss: 0.1705 - accuracy: 0.9386 - val_loss: 0.1626 - val_accuracy: 0.9456
Epoch 2/20
3990/3990 [==============================] - 847s 212ms/step - loss: 0.1618 - accuracy: 0.9412 - val_loss: 0.1433 - val_accuracy: 0.9456
Epoch 3/20
3990/3990 [==============================] - 846s 212ms/step - loss: 0.1593 - accuracy: 0.9425 - val_loss: 0.1478 - val_accuracy: 0.9438
Epoch 4/20
3990/3990 [==============================] - 846s 212ms/step - loss: 0.1567 - accuracy: 0.9427 - val_loss: 0.1428 - val_accuracy: 0.9468
Epoch 5/20
3990/3990 [==============================] - 846s 212ms/step - loss: 0.1558 - accuracy: 0.9425 - val_loss: 0.1502 - val_accuracy: 0.9425
Epoch 6/20
3990/3990 [==============================] - 843s 211ms/step - loss: 0.1554 - accuracy: 0.9433 - val_loss: 0.1453 - val_accuracy: 0.9456
Epoch 7/20
3990/3990 [==============================] - 843s 211ms/step - loss: 0.1482 - accuracy: 0.9454 - val_loss: 0.1362 - val_accuracy: 0.9477
Epoch 8/20
3990/3990 [==============================] - 843s 211ms/step - loss: 0.1475 - accuracy: 0.9449 - val_loss: 0.1373 - val_accuracy: 0.9471
Epoch 9/20
3990/3990 [==============================] - 845s 212ms/step - loss: 0.1468 - accuracy: 0.9460 - val_loss: 0.1362 - val_accuracy: 0.9485
Epoch 10/20
3990/3990 [==============================] - 843s 211ms/step - loss: 0.1448 - accuracy: 0.9462 - val_loss: 0.1344 - val_accuracy: 0.9489
Epoch 11/20
3990/3990 [==============================] - 846s 212ms/step - loss: 0.1447 - accuracy: 0.9458 - val_loss: 0.1346 - val_accuracy: 0.9483
Epoch 12/20
3990/3990 [==============================] - 843s 211ms/step - loss: 0.1444 - accuracy: 0.9460 - val_loss: 0.1342 - val_accuracy: 0.9483
Marco Cerliani's answer will probably solve it for you. That gives console output so you can verify the LR was actually reduced.
If you needed to check the model's learning rate in code, you can use
tf.keras.backend.get_value(model.optimizer.lr)
Following is the reference code:
Xtrain2 = df2.iloc[:,:-1].values
ytrain2 = df2['L'].values.reshape((200,1))
print(Xtrain2.shape, ytrain2.shape)
#--------------------------
lrelu = lambda x: tf.keras.activations.relu(x, alpha=0.1)
model2 = tf.keras.models.Sequential([
tf.keras.layers.Dense(1501, input_dim=1501, activation='relu'),
tf.keras.layers.Dense(100, activation='relu'),
tf.keras.layers.Dense(100, activation='relu'),
#tf.keras.layers.Dense(1),
#tf.keras.layers.Dense(1, activation=lrelu)
tf.keras.layers.Dense(1, activation='sigmoid')
])
model2.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
#--------------------------
model2.fit(Xtrain2, ytrain2, epochs=50)#, verbose=0)
Just a simple attempt at a classifier. The last layer is sigmoid since it's just a binary classifier. The loss is also appropriate for the problem. Dimension of input is 1500 and the number of samples is 200. I get the following output:
(200, 1501) (200, 1)
Train on 200 samples
Epoch 1/50
200/200 [==============================] - 0s 2ms/sample - loss: 0.4201 - accuracy: 0.0300
Epoch 2/50
200/200 [==============================] - 0s 359us/sample - loss: -1.1114 - accuracy: 0.0000e+00
Epoch 3/50
200/200 [==============================] - 0s 339us/sample - loss: -4.6102 - accuracy: 0.0000e+00
Epoch 4/50
200/200 [==============================] - 0s 344us/sample - loss: -13.7864 - accuracy: 0.0000e+00
Epoch 5/50
200/200 [==============================] - 0s 342us/sample - loss: -34.7789 - accuracy: 0.0000e+00
.
.
.
Epoch 40/50
200/200 [==============================] - 0s 348us/sample - loss: -905166.4000 - accuracy: 0.3750
Epoch 41/50
200/200 [==============================] - 0s 344us/sample - loss: -1010177.5300 - accuracy: 0.3400
Epoch 42/50
200/200 [==============================] - 0s 354us/sample - loss: -1129819.1825 - accuracy: 0.3450
Epoch 43/50
200/200 [==============================] - 0s 379us/sample - loss: -1263355.3200 - accuracy: 0.3900
Epoch 44/50
200/200 [==============================] - 0s 359us/sample - loss: -1408803.0400 - accuracy: 0.3750
Epoch 45/50
200/200 [==============================] - 0s 355us/sample - loss: -1566850.5900 - accuracy: 0.3300
Epoch 46/50
200/200 [==============================] - 0s 359us/sample - loss: -1728280.7550 - accuracy: 0.3550
Epoch 47/50
200/200 [==============================] - 0s 354us/sample - loss: -1909759.2400 - accuracy: 0.3400
Epoch 48/50
200/200 [==============================] - 0s 379us/sample - loss: -2108889.7200 - accuracy: 0.3750
Epoch 49/50
200/200 [==============================] - 0s 369us/sample - loss: -2305491.9800 - accuracy: 0.3700
Epoch 50/50
200/200 [==============================] - 0s 374us/sample - loss: -2524282.6300 - accuracy: 0.3050
I don't see where I'm going wrong in the above code. Any help would be appreciated!
I am using tf.distribute.MirroredStrategy() to train the textcnn model, but when I set vocab_size=0 or other wrong values, no error will be reported in this mode. When tf.distribute.MirroredStrategy() is not used, the wrong vocab_size will immediately report an error
use wrong value fro vocab_size:
model=TextCNN(padding_size,vocab_size-10,embed_size,filter_num,num_classes)
model.compile(loss='sparse_categorical_crossentropy',optimizer=tf.keras.optimizers.Adam(),metrics=['accuracy'])
model.fit(train_dataset, epochs=epoch,validation_data=valid_dataset, callbacks=callbacks)
Error:
2 root error(s) found.
(0) Invalid argument: indices[63,10] = 4726 is not in [0, 4726)
[[node text_cnn_1/embedding/embedding_lookup (defined at <ipython-input-7-6ef8a4397184>:37) ]]
[[Adam/Adam/update/AssignSubVariableOp/_45]]
(1) Invalid argument: indices[63,10] = 4726 is not in [0, 4726)
[[node text_cnn_1/embedding/embedding_lookup (defined at <ipython-input-7-6ef8a4397184>:37) ]]
0 successful operations.
0 derived errors ignored. [Op:__inference_train_function_234431]
but with strategy.scope() no Error and works well:
strategy = tf.distribute.MirroredStrategy()
with strategy.scope():
print(vocab_size)
model=TextCNN(padding_size,vocab_size-1000,embed_size,filter_num,num_classes)
model.compile(loss='sparse_categorical_crossentropy',optimizer=tf.keras.optimizers.Adam(),metrics=['accuracy'])
model.fit(train_dataset, epochs=epoch,validation_data=valid_dataset, callbacks=callbacks)
log like this（looks very good）:
Learning rate for epoch 1 is 0.0010000000474974513
2813/2813 [==============================] - 16s 6ms/step - loss: 0.8097 - accuracy: 0.7418 - val_loss: 0.4567 - val_accuracy: 0.8586 - lr: 0.0010
Epoch 2/15
2813/2813 [==============================] - ETA: 0s - loss: 0.4583 - accuracy: 0.8560
Learning rate for epoch 2 is 0.0010000000474974513
2813/2813 [==============================] - 14s 5ms/step - loss: 0.4583 - accuracy: 0.8560 - val_loss: 0.4051 - val_accuracy: 0.8756 - lr: 0.0010
Epoch 3/15
2810/2813 [============================>.] - ETA: 0s - loss: 0.3909 - accuracy: 0.8768
Learning rate for epoch 3 is 0.0010000000474974513
2813/2813 [==============================] - 14s 5ms/step - loss: 0.3909 - accuracy: 0.8767 - val_loss: 0.3853 - val_accuracy: 0.8844 - lr: 0.0010
Epoch 4/15
2811/2813 [============================>.] - ETA: 0s - loss: 0.2999 - accuracy: 0.9047
Learning rate for epoch 4 is 9.999999747378752e-05
2813/2813 [==============================] - 14s 5ms/step - loss: 0.2998 - accuracy: 0.9047 - val_loss: 0.3700 - val_accuracy: 0.8865 - lr: 1.0000e-04
Epoch 5/15
2807/2813 [============================>.] - ETA: 0s - loss: 0.2803 - accuracy: 0.9114
Learning rate for epoch 5 is 9.999999747378752e-05
2813/2813 [==============================] - 15s 5ms/step - loss: 0.2803 - accuracy: 0.9114 - val_loss: 0.3644 - val_accuracy: 0.8888 - lr: 1.0000e-04
Epoch 6/15
2803/2813 [============================>.] - ETA: 0s - loss: 0.2639 - accuracy: 0.9162
Learning rate for epoch 6 is 9.999999747378752e-05
2813/2813 [==============================] - 14s 5ms/step - loss: 0.2636 - accuracy: 0.9163 - val_loss: 0.3615 - val_accuracy: 0.8896 - lr: 1.0000e-04
Epoch 7/15
2805/2813 [============================>.] - ETA: 0s - loss: 0.2528 - accuracy: 0.9188
Learning rate for epoch 7 is 9.999999747378752e-05
2813/2813 [==============================] - 14s 5ms/step - loss: 0.2526 - accuracy: 0.9189 - val_loss: 0.3607 - val_accuracy: 0.8909 - lr: 1.0000e-04
More simply，like this,run and no error:
strategy = tf.distribute.MirroredStrategy()
with strategy.scope():
model = Sequential()
model.add(Embedding(1000, 64, input_length=20))
test_array=np.random.randint(10000,size=(32,20))
model.predict(test_array)
why???
I've trained some networks with ResNetV2 50 ( https://tfhub.dev/google/imagenet/resnet_v2_50/feature_vector/4 ) and it work very well for my datasets.
Then I tried tf.keras.applications.ResNet50 and accuracy is very lower than the other.
Here two models:
The first (with hub)
base_model = hub.KerasLayer('https://tfhub.dev/google/imagenet/resnet_v2_50/feature_vector/4', input_shape=(IMAGE_H, IMAGE_W, 3))
base_model.trainable = False
model = tf.keras.Sequential([
base_model ,
Dense(num_classes, activation='softmax')
])
The second (with keras.applications)
base_model = tf.keras.applications.ResNet50V2(input_shape=(IMAGE_H, IMAGE_W, 3), include_top=False, weights='imagenet', pooling='avg')
base_model.trainable = False
model = tf.keras.Sequential([
base_model,
Dense(num_classes, activation='softmax')
])
The optimizer is the same (Adam), epochs, steps, dataset (train and validation), learning rate are the same as well. But the first start with a val_accuracy near 80% and end with an accuracy near 99%, the second start with 85% of val_accuracy from first to last epoch, as it's overfitting. I got the same behavior changing dataset and parameters for each model.
What am I doing wrong?
Both tf.keras.applications.ResNet50V2 and hub.KerasLayer for ResNet50V2 are trained on the same image_net dataset and share the same weights, there is no difference between both.
Coming to the accuracy difference, I have tried both the API's to load base model and run on 10 epochs.
I saw a minor difference in the accuracy of both training and validation.
Then again after running the TensorFlow hub base model for the second time, again I found some minor change in accuracy.This may be due to the order of calculation of these and also could be floating point precision error.
Below are the outputs:
tf.keras.applications.ResNet50V2 as base model.
base_model = tf.keras.applications.ResNet50V2(input_shape=(96, 96, 3), include_top=False, weights='imagenet', pooling='avg')
base_model.trainable = False
model = tf.keras.Sequential([
base_model,
Dense(10, activation='softmax')
])
model.compile(loss="categorical_crossentropy",optimizer="adam",metrics=["accuracy"])
H = model.fit_generator(aug.flow(trainX, trainY, batch_size=16),
validation_data=(testX, testY), steps_per_epoch=len(trainX) // 16,
epochs=10)
Output 1:
Epoch 1/10
103/103 [==============================] - 55s 535ms/step - loss: 1.5820 - accuracy: 0.4789 - val_loss: 0.9162 - val_accuracy: 0.6949
Epoch 2/10
103/103 [==============================] - 57s 554ms/step - loss: 0.9539 - accuracy: 0.6534 - val_loss: 0.8376 - val_accuracy: 0.6852
Epoch 3/10
103/103 [==============================] - 55s 532ms/step - loss: 0.8610 - accuracy: 0.6944 - val_loss: 0.7104 - val_accuracy: 0.7240
Epoch 4/10
103/103 [==============================] - 55s 533ms/step - loss: 0.7671 - accuracy: 0.7214 - val_loss: 0.5988 - val_accuracy: 0.7918
Epoch 5/10
103/103 [==============================] - 55s 536ms/step - loss: 0.6994 - accuracy: 0.7526 - val_loss: 0.6029 - val_accuracy: 0.7676
Epoch 6/10
103/103 [==============================] - 55s 537ms/step - loss: 0.6880 - accuracy: 0.7508 - val_loss: 0.6121 - val_accuracy: 0.7724
Epoch 7/10
103/103 [==============================] - 55s 533ms/step - loss: 0.6588 - accuracy: 0.7593 - val_loss: 0.5486 - val_accuracy: 0.8015
Epoch 8/10
103/103 [==============================] - 55s 534ms/step - loss: 0.6640 - accuracy: 0.7630 - val_loss: 0.5287 - val_accuracy: 0.8232
Epoch 9/10
103/103 [==============================] - 54s 528ms/step - loss: 0.6004 - accuracy: 0.7881 - val_loss: 0.4598 - val_accuracy: 0.8426
Epoch 10/10
103/103 [==============================] - 55s 530ms/step - loss: 0.5583 - accuracy: 0.8016 - val_loss: 0.4605 - val_accuracy: 0.8426
Tensorflow Hub as base model.
tf.keras.backend.clear_session()
np.random.seed(42)
tf.random.set_seed(42)
base_model =hub.KerasLayer('https://tfhub.dev/google/imagenet/resnet_v2_50/feature_vector/4', input_shape=(96, 96, 3))
base_model.trainable = False
model = tf.keras.Sequential([base_model ,Dense(10, activation='softmax')])
model.compile(loss="categorical_crossentropy",optimizer="adam",metrics=["accuracy"])
H = model.fit_generator(aug.flow(trainX, trainY, batch_size=16),
validation_data=(testX, testY), steps_per_epoch=len(trainX) // 16,
epochs=10)
Output 2:
Epoch 1/10
103/103 [==============================] - 54s 526ms/step - loss: 1.7543 - accuracy: 0.4464 - val_loss: 1.0185 - val_accuracy: 0.5981
Epoch 2/10
103/103 [==============================] - 53s 519ms/step - loss: 0.9827 - accuracy: 0.6283 - val_loss: 1.0067 - val_accuracy: 0.6416
Epoch 3/10
103/103 [==============================] - 56s 548ms/step - loss: 0.8719 - accuracy: 0.6944 - val_loss: 0.7195 - val_accuracy: 0.7240
Epoch 4/10
103/103 [==============================] - 54s 521ms/step - loss: 0.8177 - accuracy: 0.7208 - val_loss: 0.7490 - val_accuracy: 0.7385
Epoch 5/10
103/103 [==============================] - 54s 522ms/step - loss: 0.7641 - accuracy: 0.7379 - val_loss: 0.6325 - val_accuracy: 0.7797
Epoch 6/10
103/103 [==============================] - 54s 523ms/step - loss: 0.6551 - accuracy: 0.7655 - val_loss: 0.6431 - val_accuracy: 0.7579
Epoch 7/10
103/103 [==============================] - 55s 530ms/step - loss: 0.6538 - accuracy: 0.7734 - val_loss: 0.5824 - val_accuracy: 0.7797
Epoch 8/10
103/103 [==============================] - 55s 536ms/step - loss: 0.6387 - accuracy: 0.7691 - val_loss: 0.6254 - val_accuracy: 0.7772
Epoch 9/10
103/103 [==============================] - 56s 540ms/step - loss: 0.6394 - accuracy: 0.7685 - val_loss: 0.6539 - val_accuracy: 0.7554
Epoch 10/10
103/103 [==============================] - 55s 536ms/step - loss: 0.5816 - accuracy: 0.7955 - val_loss: 0.5703 - val_accuracy: 0.7990
Now if I run the Tensorflow hub model again.
Output 3:
Epoch 1/10
103/103 [==============================] - 55s 534ms/step - loss: 1.6412 - accuracy: 0.4764 - val_loss: 1.0697 - val_accuracy: 0.5738
Epoch 2/10
103/103 [==============================] - 54s 528ms/step - loss: 1.0312 - accuracy: 0.6412 - val_loss: 1.0196 - val_accuracy: 0.6077
Epoch 3/10
103/103 [==============================] - 59s 570ms/step - loss: 0.8710 - accuracy: 0.6975 - val_loss: 0.7088 - val_accuracy: 0.7240
Epoch 4/10
103/103 [==============================] - 54s 529ms/step - loss: 0.8108 - accuracy: 0.7128 - val_loss: 0.6539 - val_accuracy: 0.7458
Epoch 5/10
103/103 [==============================] - 54s 522ms/step - loss: 0.7311 - accuracy: 0.7440 - val_loss: 0.6029 - val_accuracy: 0.7676
Epoch 6/10
103/103 [==============================] - 54s 523ms/step - loss: 0.6683 - accuracy: 0.7612 - val_loss: 0.6621 - val_accuracy: 0.7506
Epoch 7/10
103/103 [==============================] - 54s 527ms/step - loss: 0.6518 - accuracy: 0.7753 - val_loss: 0.6166 - val_accuracy: 0.7700
Epoch 8/10
103/103 [==============================] - 54s 524ms/step - loss: 0.6147 - accuracy: 0.7795 - val_loss: 0.5611 - val_accuracy: 0.7797
Epoch 9/10
103/103 [==============================] - 54s 522ms/step - loss: 0.6671 - accuracy: 0.7667 - val_loss: 0.5126 - val_accuracy: 0.8087
Epoch 10/10
103/103 [==============================] - 54s 525ms/step - loss: 0.6090 - accuracy: 0.7832 - val_loss: 0.6355 - val_accuracy: 0.7627
Hope this answeres your question.