In my Keras custom loss function I would like to know the sample indexes (as in the original input array) for the current y_true, y_pred tensors.
I know it sounds weird, but for calculating loss I need some additional information, what I prepare in an external array, which is not part neither of the input array, nor the expected output array.
The only solution I currently see is to include it to the expected output array as additional columns, so I got it in y_true, but I am not sure how disturbing it would be for the NN and the optimizer to have one extra node in the output layer, which's actual prediction is not correlated with the calculated loss...
Related
I'm building a model using Tensorflow where the input is a slice of the output. Think of the output layer as a 2D array. One row of that array is the input data. The neural network currently tries to connect the input to the output using a mean-squared error loss function. It's doing a fairly good job, but the accuracy needs to be improved a little.
To do that, I'm trying to add another physics-based loss function. If I can have the network place the input slice in its correct location in the output, that would greatly simplify the problem as each row in the output 2D array depends on the two rows above it.
I hope this makes sense.
First, I would like to say that I only have little experience in Keras/Tensorflow and probably lack some understanding on tensors manipulations.
I am using a model which input is an "oversized" matrix (NxN). That is, I feed it with data that can be smaller (ie. (KxK), K <= N) where "missing" data (to fit the NxN shape) is filled with zeros. The output is an encoded version (Nx2) of the input.
I'm using a custom loss function that I would like to be computed only on the (Kx2) first values of the model's output. To do so, I think the solution is to "slice" the y_pred tensor in my loss function since I don't want to simply mask it with a boolean tensor. However, I can't figure out how to pass K as a dynamic argument to my custom loss.
Wrapping the function within another function that takes an argument does not fit my needs since the K value will change on each data sample
Passing K in the model's input and getting it back through a function wrapp (eg. https://stackoverflow.com/a/55445837/6315123) as mentionned in the first point does not work either, since slices cannot be computed from Tensor (as far as I understand); and evaluate the tensor within the loss function doesn't seem possible.
How can I pass such an argument to my loss function ?
Thanks !
I have a setup like this:
model = keras.Model(input,[output1,output2])
My loss function is only a function of output1. How do I tell Keras to ignore output2 for the purposes of computing loss? The best I have come up with is to generate a bogus loss function which always returns 0.0:
model.compile(optimizer=..., loss=[realLossFunction, zeroLossFunction])
I can live with this, but I have to see the statistics and progress of this loss function all over the place and would like to know if there is a more elegant way.
You could just avoid putting this output in the model, and then reusing the weights (or sharing them with the functional API) to add the additional output to the full model.
But using a zero loss is also fine.
I am trying to predict() the output for a single data point d, using my trained Keras model loaded from a file. But I get a ValueError If predicting from data tensors, you should specify the 'step' argument. What does that mean?
I tried setting step=1, but then I get a different error ValueError: Cannot feed value of shape () for Tensor u'input_1:0', which has shape '(?, 600)'.
Here is my code:
d = np.concatenate((hidden[p[i]], hidden[x[i]])).resize((1,600))
hidden[p[i]] = autoencoder.predict(d,steps=)
The model is expecting (?,600) as input. I have concatenated two numpy arrays of shape (300,) each to get (600,), which is resized to (1,600). This (1,600) is my input to predict().
In my case, the input to predict was None (because I had a bug in another part of the code).
In official doc, steps refer to the total number of steps before stopping. So steps=1 means make predictions on one batch instead of making prediction on one record (single data point).
https://keras.io/models/sequential/
-> Define value of steps argument,
d = np.concatenate((hidden[p[i]],
hidden[x[i]])).resize((1,600))
hidden[p[i]] = autoencoder.predict(d,steps=1)
If you are using a test data generator, it is good practice to define the steps, as mentioned in the documentation.
If you are predicting a single instance, no need to define the steps. Just make sure the argument (i.e. instance 'd') is not None, otherwise that error will show up. Some reshaping may also be necessary.
in my case i got the same error, i just reshaped the data to predict with numpy function reshape() to the shape of the data originally used to train the model.
XGBClassifier outputs probabilities if we use the method "predict_proba", however, when I train the model using xgboost.train, I cannot figure out how to get probabilities as output. Here is a chunk of my code:
dtrain=xgb.DMatrix(X_train, label=y)
param = {'max_depth':2, 'eta':1, 'silent':1, 'objective':'binary:logistic'}
modelXG=xgb.train(param,dtrain,xgb_model='xgbmodel')
xgboost.train() returns a xgb.Booster object. The xgb.Booster.predict() call returns probabilities in the case of a classification problem instead of the expected labels, if you are used to the .predict()methods of sklearn models. So modelXG.predict(dtest) call will give you want you need.