Hyperparameters tuning with tf.estimator.DNNClassifier - tensorflow2.0

Question

I've implemented the following model using the DNNClassifier class. The model is paramatrized as follows
classifier = tf.estimator.DNNClassifier(
hidden_units=[60, 30, 20],
feature_columns=feature_columns,
n_classes=len(labels),
label_vocabulary=labels,
batch_norm=True,
optimizer=lambda: tf.keras.optimizers.Adam(
learning_rate=tf.compat.v1.train.exponential_decay(
learning_rate=0.1,
global_step=tf.compat.v1.train.get_global_step(),
decay_steps=10000,
decay_rate=0.96)
)
)
Now I would like to do some hyperparameters tuning (e.g. learning rate, number of units, etc).
DNNClassifier, being a premade estimator class, inherits from the Estimator class.
But while Estimator has a params argument to pass hyperparameters, DNNClassifier has none.
So what should be the preferred way to do hyperparameter tuning using DNNClassifier?

Answer 1

First of all you need an input function for your estimator, assuming you are using pandas dataframes to keep your data, (data_df and label_df are dataframes) you can write something like:
def make_input_fn(data_df, label_df, num_epochs=10, shuffle=True, batch_size=32):
def input_function():
ds = tf.data.Dataset.from_tensor_slices((dict(data_df), label_df))
if shuffle:
ds = ds.shuffle(1024)
ds = ds.batch(batch_size).repeat(num_epochs)
return ds
return input_function
Then use the code above to create two input functions, one for training and one for validation like this:
train_input_fn = make_input_fn(X_train, y_train)
val_input_fn = make_input_fn(X_val, y_val, num_epochs=1, shuffle=False)
Finally train the classifier you defined and evaluate it using the validation set. Run this pipeline multiple times to tune your hyperparameters.
classifier = tf.estimator.DNNClassifier(
hidden_units=[60, 30, 20],
feature_columns=feature_columns,
n_classes=len(labels),
label_vocabulary=labels,
batch_norm=True,
optimizer=lambda: tf.keras.optimizers.Adam(
learning_rate=tf.compat.v1.train.exponential_decay(
learning_rate=0.1,
global_step=tf.compat.v1.train.get_global_step(),
decay_steps=10000,
decay_rate=0.96)
)
)
# Train Classifier.
classifier.train(train_input_fn)
# Evaluate Classifier.
result = classifier.evaluate(val_input_fn)
print(result)