TF2 Keras - Feature Engineering in Keras saved model via Tensorflow Serving - tensorflow

Question

The Tensorflow 2 documentation for preprocessing / feature engineering over a Keras model seems to be quite confusing and isn't very friendly.
Currently I have a simple Keras N-layer model with TF feature columns feeding as dense layer. For training I have CSV files read using tf.dataset API and I have written a feature engineering function that creates new features using dataset.map function.
def feature_engg_features(features):
#Add new features
features['nodlgrbyvpatd'] = features['NODLGR'] / features['VPATD']
return(features)
I can save the model easily using tf.keras.models.save_model method. However I am having trouble figuring out how to attach the feature_engineering steps in the serving function.
Requirement: Now I want to take the same feature engineering function above and attach it to my serving function so that in JSON input via tensorflow_model_server the same feature engineering steps are applied. I know about the lambda Layer option in Keras but I want to do this via saved_model method but there are a lot of difficulties here.
For Example, below code gives error:
def feature_engg_features(features):
#Add new features
features['nodlgrbyvpatd'] = features['NODLGR'] / features['VPATD']
return(features)
#tf.function
def serving(data):
data = tf.map_fn(feature_engg_features, data, dtype=tf.float32)
# Predict
predictions = m_(data)
version = "1"
tf.keras.models.save_model(
m_,
"./exported_model/" + version,
overwrite=True,
include_optimizer=True,
save_format=None,
signatures=serving,
options=None
)
Error:
Only `tf.functions` with an input signature or concrete functions can be used as a signature.
The above error is because I have not provided InputSignature of my Keras model but I am not able to understand that I have 13 input fields, what is expected as input signature.
So I wanted to know if anyone knows the shortest way of solving this out. This is a very basic requirement and Tensorflow seems to have kept this quite complicated for Keras Tensorflow model serving.
GIST: https://colab.research.google.com/gist/rafiqhasan/6abe93ac454e942317005febef59a459/copy-of-dl-e2e-structured-mixed-data-tf-2-keras-estimator.ipynb
EDIT:
I fixed it, so TensorSpec has to be generated and passed for each feature and also model( ) has to be called in serving function.
#tf.function
def serving(WERKS, DIFGRIRD, SCENARIO, TOTIRQTY, VSTATU, EKGRP, TOTGRQTY, VPATD, EKORG, NODLGR, DIFGRIRV, NODLIR, KTOKK):
##Feature engineering
nodlgrbyvpatd = tf.cast(NODLGR / VPATD, tf.float32)
payload = {
'WERKS': WERKS,
'DIFGRIRD': DIFGRIRD,
'SCENARIO': SCENARIO,
'TOTIRQTY': TOTIRQTY,
'VSTATU': VSTATU,
'EKGRP': EKGRP,
'TOTGRQTY': TOTGRQTY,
'VPATD': VPATD,
'EKORG': EKORG,
'NODLGR': NODLGR,
'DIFGRIRV': DIFGRIRV,
'NODLIR': NODLIR,
'KTOKK': KTOKK,
'nodlgrbyvpatd': nodlgrbyvpatd,
}
## Predict
##IF THERE IS AN ERROR IN NUMBER OF PARAMS PASSED HERE OR DATA TYPE THEN IT GIVES ERROR, "COULDN'T COMPUTE OUTPUT TENSOR"
predictions = m_(payload)
return predictions
serving = serving.get_concrete_function(WERKS=tf.TensorSpec([None,], dtype= tf.string, name='WERKS'),
DIFGRIRD=tf.TensorSpec([None,], name='DIFGRIRD'),
SCENARIO=tf.TensorSpec([None,], dtype= tf.string, name='SCENARIO'),
TOTIRQTY=tf.TensorSpec([None,], name='TOTIRQTY'),
VSTATU=tf.TensorSpec([None,], dtype= tf.string, name='VSTATU'),
EKGRP=tf.TensorSpec([None,], dtype= tf.string, name='EKGRP'),
TOTGRQTY=tf.TensorSpec([None,], name='TOTGRQTY'),
VPATD=tf.TensorSpec([None,], name='VPATD'),
EKORG=tf.TensorSpec([None,], dtype= tf.string, name='EKORG'),
NODLGR=tf.TensorSpec([None,], name='NODLGR'),
DIFGRIRV=tf.TensorSpec([None,], name='DIFGRIRV'),
NODLIR=tf.TensorSpec([None,], name='NODLIR'),
KTOKK=tf.TensorSpec([None,], dtype= tf.string, name='KTOKK')
)
version = "1"
tf.saved_model.save(
m_,
"./exported_model/" + version,
signatures=serving
)

Answer 1

So the right way to do this is here, Feature engineering and Pre-processing can be done in the serving_default method through below option. I tested it further via Tensorflow serving.
#tf.function
def serving(WERKS, DIFGRIRD, SCENARIO, TOTIRQTY, VSTATU, EKGRP, TOTGRQTY, VPATD, EKORG, NODLGR, DIFGRIRV, NODLIR, KTOKK):
##Feature engineering
nodlgrbyvpatd = tf.cast(NODLGR / VPATD, tf.float32)
payload = {
'WERKS': WERKS,
'DIFGRIRD': DIFGRIRD,
'SCENARIO': SCENARIO,
'TOTIRQTY': TOTIRQTY,
'VSTATU': VSTATU,
'EKGRP': EKGRP,
'TOTGRQTY': TOTGRQTY,
'VPATD': VPATD,
'EKORG': EKORG,
'NODLGR': NODLGR,
'DIFGRIRV': DIFGRIRV,
'NODLIR': NODLIR,
'KTOKK': KTOKK,
'nodlgrbyvpatd': nodlgrbyvpatd,
}
## Predict
##IF THERE IS AN ERROR IN NUMBER OF PARAMS PASSED HERE OR DATA TYPE THEN IT GIVES ERROR, "COULDN'T COMPUTE OUTPUT TENSOR"
predictions = m_(payload)
return predictions
serving = serving.get_concrete_function(WERKS=tf.TensorSpec([None,], dtype= tf.string, name='WERKS'),
DIFGRIRD=tf.TensorSpec([None,], name='DIFGRIRD'),
SCENARIO=tf.TensorSpec([None,], dtype= tf.string, name='SCENARIO'),
TOTIRQTY=tf.TensorSpec([None,], name='TOTIRQTY'),
VSTATU=tf.TensorSpec([None,], dtype= tf.string, name='VSTATU'),
EKGRP=tf.TensorSpec([None,], dtype= tf.string, name='EKGRP'),
TOTGRQTY=tf.TensorSpec([None,], name='TOTGRQTY'),
VPATD=tf.TensorSpec([None,], name='VPATD'),
EKORG=tf.TensorSpec([None,], dtype= tf.string, name='EKORG'),
NODLGR=tf.TensorSpec([None,], name='NODLGR'),
DIFGRIRV=tf.TensorSpec([None,], name='DIFGRIRV'),
NODLIR=tf.TensorSpec([None,], name='NODLIR'),
KTOKK=tf.TensorSpec([None,], dtype= tf.string, name='KTOKK')
)
version = "1"
tf.saved_model.save(
m_,
"./exported_model/" + version,
signatures=serving
)