SMO algorithm running into infinite loop? - optimization

Question

I'm interest in building an SVM multi class classifier, so I am currently implementing
Sequential minimal optimization SMO.
My implementation is based on the pseudo code in
`Fast Training of Support Vector Machines using Sequential Minimal Optimization" by John C. Platt
I observed that for certain training examples. The Smo may diverge and run into an infinite loop
The following loop in the main routine
numChanged = 0;
examineAll = 1;
while (numChanged > 0 || examineAll >0) {…}
may run into an infinite loop.
Is there there clue or criterion to prevent the smo algorithm routine from running into an infinite loop?
I would like to thank you for your answer.
Regards

Answer 1

You can add a max iteration condition if you want:
while ((numChanged > 0 || examineAll) && iter < MaxIter)
but for most cases it shouldn't run into an infinite loop, this is the full Platt's pseudocode:
while (numChanged > 0 || examineAll)
{
numChanged = 0;
// Adding curly brackets for better readability
if (examineAll)
{
loop I over all training examples
numChanged += examineExample(I);
}
else
{
loop I over examples where alpha is not 0 & not C
numChanged += examineExample(I);
}
if (examineAll == 1)
{
examineAll = 0;
}
else
{
examineAll = 1;
}
}
Notice that what it is doing is performing an iteration to examine the example and the next one do the same just to those examples where alpha is not 0 or C. If nothing changes after the "examine all" iteration, the while loop condition will be false hence stopping the loop.
So, for that to be in a infinite loop there must be a corner case (probably a numerical error) that introduce oscillations making examples to change during the examine all phase but not changing in the "examine only alpha == 0 and C".
Usually if the data is normalized in [-1,1] or [0,1] and the parameters of the algorithm have reasonable values, those corner cases would be rare. In any case, if you want to be extra careful you can put the max-iter safety net.