Constrained Optimization Tensorflow - tensorflow

Question

I have a trained classifier neural network in Keras. Let the neural network be f(x). I want to find the vectors x such that when ||x||^2 = 1, f(x) is maximized. I currently have trained my neural network with Keras
model = Sequential()
model.add(Dense(500, activation='sigmoid'))
model.add(Dense(500, activation='sigmoid'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy', auc])
model.fit(x_train, y_train, validation_data=(x_test, y_test), epochs=2, verbose = 1, callbacks=[earlyStopping])
I want to know if there is a way to solve this constrained optimization problem once my Neural network has already been trained. There is a scipy optimize which can do this for general functions. Is there a way to do this for a neural network. Please include a code sample.

Answer 1

If I understand you correctly you have finished training your neural network and would like to find the input x which maximizes the probability of it being in a certain class (where the output is close to 1.0)
You could write a small function to assess the performance of your network using the predict_proba() method to get classification probabilities on test data, and then optimise this function using scipy:
model = Sequential()
model.add(Dense(500, activation='sigmoid'))
model.add(Dense(500, activation='sigmoid'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy', auc])
model.fit(x_train, y_train, validation_data=(x_test, y_test), epochs=2, verbose = 1, callbacks=[earlyStopping])
def f(x):
prediction = model.predict_proba(x)
return -prediction
a = scipy.optimize.minimize(f, x0=np.random.randn(500))
optimal_x = a.x
optimal_x will be the input x which maximises the certainty with which your classifier puts it in one specific class.