XGBoost4J Synchronization issue? - xgboost

Question

We are using XGboost4J for ML predictions. We developed predictor using restful webservice so that within platform various components can call ML predictor. e.g from product titles and description finding out product category tree.
Just depicting code in fundamental way we implemented.
// This is done in
initialize method, for every model there is one singleton Booster object loaded.
Class Predictor{
private Booster xgboost;
//init call from Serivice initialization while injecting Predictor
public void init(final String modelFile, final Integer numThreads){
if (!(new File(modelFile).exists())) {
throw new IOException("Modelfile " + modelFile + " does not exist");
}
// we use a util class Params to handle parameters as example
final Iterable<Entry<String, Object>> param = new Params() {
{
put("nthread", numThreads);
}
};
xgboost = new Booster(param, modelFile);
}
//Predict method
public String predict(final String predictionString){
final String dummyLabel = "-1";
final String x_n = dummyLabel + "\t" + x_n_libsvm_idxStr;
final DataLoader.CSRSparseData spData = XGboostSparseData.format(x_n);
final DMatrix x_n_dmatrix = new DMatrix(spData.rowHeaders,
spData.colIndex, spData.data, DMatrix.SparseType.CSR);
final float[][] predict = xgboost.predict(x_n_dmatrix);
// Then there is conversion logic of predict to predicted model result which returns predictions
String prediction = getPrediction(predict);
return prediction
}
}
Above predictor class is singleton injected in webservices Services class
so for every services call thread call's
service.predict(predictionString);
There is problem in tomcat container when multiple concurrent threads calls predict method Boosters method is synchronized
private synchronized float[][] pred(DMatrix data, boolean outPutMargin, long treeLimit, boolean predLeaf) throws XGBoostError {
byte optionMask = 0;
if(outPutMargin) {
optionMask = 1;
}
if(predLeaf) {
optionMask = 2;
}
float[][] rawPredicts = new float[1][];
ErrorHandle.checkCall(XgboostJNI.XGBoosterPredict(this.handle, data.getHandle(), optionMask, treeLimit, rawPredicts));
int row = (int)data.rowNum();
int col = rawPredicts[0].length / row;
float[][] predicts = new float[row][col];
for(int i = 0; i < rawPredicts[0].length; ++i) {
int r = i / col;
int c = i % col;
predicts[r][c] = rawPredicts[0][i];
}
return predicts;
}
This created thread waits and locking because of synchronized block and this is resulting webservices not scalable.
We tried removing synchronized from XGboost4J source code and compiled jar but it crashes within first 1-2 mins. Heap dump showing its crashing at below line while doing native call to XgboostJNI
ErrorHandle.checkCall(XgboostJNI.XGBoosterPredict(this.handle, data.getHandle(), optionMask, treeLimit, rawPredicts));
Anyone knows better way of implementing Xgboost4J for highly scalable webservices approach using Java?

Answer 1

You could use PMML (https://github.com/jpmml/jpmml-xgboost), referring to https://github.com/jpmml/jpmml-xgboost/issues/7#issuecomment-250965282