I have trained two ml models based on two different datasets. Then I saved them as model1.pkl and model2.pkl . There are two user inputs(not input data for model) like x=0 and x=1 and if x=0 I have to go with model1.pkl for prediction else I have to go with model2.pkl for prediction. I can do them using if condition but my problem is I have to know whether is there any possibilities to save back that as model.pkl including this condition statement. If I combine them and save as a model it will be easy to load in other IDEs.
You can create a class, that has a minimal interface like a model class like this:
# create the test setup
import lightgbm as lgb
import pickle as pkl
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import LabelEncoder
#from sklearn
df['x1']= LabelEncoder().fit_transform(df['x1'])
data= {
'x': [1.0, 1.0, 0.0, 1.0, 1.0, 0.0, 0.0, 1.0, 1.0, 1.0, 0.0, 0.0],
'q': [0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 2.0, 2.0, 2.0, 3.0, 3.0, 3.0],
'b': [1.0, 2.0, 3.0, 1.0, 2.0, 3.0, 1.0, 2.0, 3.0, 1.0, 2.0, 3.0],
'target': [0.0, 2.0, 1.5, 0.0, 5.1, 4.0, 0.0, 1.0, 2.0, 0.0, 2.1, 1.5]
}
df= pd.DataFrame(data)
X, y=df.iloc[:, :-1], df.iloc[:, -1]
X= X.astype('float32')
# create two models
model1= LinearRegression()
model2 = lgb.LGBMRegressor(n_estimators=5, num_leaves=10, min_child_samples=1)
ser_model1= X['x']==0.0
model1.fit(X[ser_model1], y[ser_model1])
model2.fit(X[~ser_model1], y[~ser_model1])
# define a class that mocks the model interface
class CombinedModel:
def __init__(self, model1, model2):
self.model1= model1
self.model2= model2
def predict(self, X, **kwargs):
ser_model1= X['x']==0.0
return pd.concat([
pd.Series(self.model1.predict(X[ser_model1]), index=X.index[ser_model1]),
pd.Series(self.model2.predict(X[~ser_model1]), index=X.index[~ser_model1])
]
).sort_index()
# create a model with the two trained sum models
# and pickle it
model= CombinedModel(model1, model2)
model.predict(X)
with open('model.pkl', 'wb') as fp:
pkl.dump(model, fp)
model= model1= model2= None
# test load it
with open('model.pkl', 'rb') as fp:
model= pkl.load(fp)
model.predict(X)
If you want, of course you can also implement a fit method in the class above, which just calls the two models. If you implement the necessary methods, you could even use this class in a sklearn pipeline.
You can use Ensemble Voting Classifier which will consider the outputs from both the models and give appropriate output.
Link- https://machinelearningmastery.com/voting-ensembles-with-python/
Related
Sorry if this is something it was just asked, but I searched for it without success. I am thinking about to apply for the tensorflow certificate exam. My first question is if, during the exam, the custom activation functions are allowed.
For example: Imagine a question about a regression when the data is:
features = np.array([0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0], dtype=float)
targets = np.array([0.0, 1.0, 4.0, 9.0, 16.0, 25.0, 36.0], dtype=float)
This is clearly a x^2 problem. Could I do something like this?
tf_lpow = lambda x: tf.math.pow(x, 2)
model = tf.keras.models.Sequential([
tf.keras.layers.Dense(units=1, activation=tf_lpow, input_shape=(1,)),
])
Considering that maybe this could not be allowed, I was thinking about another solution:
lr_scheduler = ReduceLROnPlateau(monitor='loss', factor=0.75, patience=50, min_lr=3e-80)
callbacks = [lr_scheduler]
model = tf.keras.models.Sequential([
tf.keras.layers.Dense(units=6, activation='sigmoid', kernel_regularizer=tf.keras.regularizers.l2(0.01), kernel_initializer=tf.keras.initializers.RandomNormal(stddev=0.01), input_shape=(1,)),
tf.keras.layers.Dense(units=1, activation='linear')
])
But even in the case the loss is decreasing, the accuracy is stack in 0.2857, not reaching the goal. In this case, what could I do?
Thanks in advance.
Say I have an vector v(ij) and matrix M(jk).
v = np.array([1.0, 1.0, 0.0, 0.0], dtype=np.float32)
M = np.array([[0.0, 0.0, 0.2, 0.0],
[0.0, 0.0, 0.0, -0.1],
[0.0, 0.0, 0.0, 0.5],
[0.0, 0.0, 0.0, 0.0]], dtype=np.float32)
How can I implement the following operation without for loop?
def matmul_in_place(vector, matrix):
for i in range(vector.shape[0]):
for j in range(matrix.shape[1]):
vector[i] += vector[j] * matrix[j][i]
Result of matmul_in_place(v, M) should be [1.0, 1.0, 0.2, 0.0]
In place is not a requirement, instead, it means that when calculating vector[1], I should use updated vector[0] for vector[j] when j is 0. This also means that I need to calculate the vector from left to right.
Vector v can also be a matrix, but I want to keep the problem simple. As long as two matrices can be multiplied together(M1 # M2), this operation should also work.
I've been looking into it for a while but I'm not familiar with einsum or custom cuda kernel so I don't even know if it's possible.
see the code below for a basic ML test. I ran this, but wanted to see the mathematical formula predicted by the AI using the given xs and ys array set, and the training x value.
import tensorflow as tf
import numpy as np
from tensorflow import keras
model = tf.keras.Sequential([keras.layers.Dense(units=1, input_shape=[1])])
model.compile(optimizer='sgd', loss = 'mean_squared_error')
xs = np.array([-3.0, -2.0, -1.0, 0.0, 1.0, 2.0, 3.0], dtype=float)
ys = np.array([-3.0, -2.0, -1.0, 0.0, 1.0, 2.0, 3.0], dtype=float)
model.fit(xs,ys,epochs=10)
print(model.predict([-2.5]))
while looking at tensorflow examples online , i'm seeing this
xs = np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], dtype=float)
ys = np.array([1.0, 1.5, 2.0, 2.5, 3.0, 3.5], dtype=float)
model = tf.keras.Sequential([keras.layers.Dense(units=1, input_shape=[1])])
model.compile(optimizer='sgd', loss='mean_squared_error')
I was trying to re-write this line so that it uses objects rather than string literals
model.compile(optimizer='sgd', loss='mean_squared_error')
So far i'm able to
model.compile(optimizer=keras.optimizers.SGD(), loss='mean_squared_error')
For mean_squared_error we have keras.losses.mean_squared_error(y_true, y_pred)
I'm unable to understand y_true, y_pred and what values needs to be provided for the example above.
In summary, from example above what is equivalent of
loss='mean_squared_error'
you need to pass simply
model.compile(optimizer=tf.keras.optimizers.SGD(), loss=tf.keras.losses.MeanSquaredError())
y_true and y_pred are handled automatically by keras
In Tensorflow manual, description for labels is like below:
labels: Each row labels[i] must be a valid probability distribution.
Then, does it mean labels can be like below, if I have real probability distributions of classes for each input.
[[0.1, 0.2, 0.05, 0.007 ... ]
[0.001, 0.2, 0.5, 0.007 ... ]
[0.01, 0.0002, 0.005, 0.7 ... ]]
And, is it more efficient than one-hot encoded labels?
Thank you in advance.
In a word, yes, you can use probabilities as labels.
The documentation for tf.nn.softmax_cross_entropy_with_logits says you can:
NOTE: While the classes are mutually exclusive, their probabilities
need not be. All that is required is that each row of labels is
a valid probability distribution. If they are not, the computation of the
gradient will be incorrect.
If using exclusive labels (wherein one and only
one class is true at a time), see sparse_softmax_cross_entropy_with_logits.
Let's have a short example to be sure it works ok:
import numpy as np
import tensorflow as tf
labels = np.array([[0.2, 0.3, 0.5], [0.1, 0.7, 0.2]])
logits = np.array([[5.0, 7.0, 8.0], [1.0, 2.0, 4.0]])
sess = tf.Session()
ce = tf.nn.softmax_cross_entropy_with_logits(
labels=labels, logits=logits).eval(session=sess)
print(ce) # [ 1.24901222 1.86984602]
# manual check
predictions = np.exp(logits)
predictions = predictions / predictions.sum(axis=1, keepdims=True)
ce_np = (-labels * np.log(predictions)).sum(axis=1)
print(ce_np) # [ 1.24901222 1.86984602]
And if you have exclusive labels, it is better to use one-hot encoding and tf.nn.sparse_softmax_cross_entropy_with_logits rather than tf.nn.softmax_cross_entropy_with_logitsand explicit probability representation like [1.0, 0.0, ...]. You can have shorter representation that way.