Yolov4 Darknet Training Error on Macbook Pro M1 - tensorflow

I am creating a custom Yolov4 to detect characters & digits in an image. I have installed Darknet on my Macbook M1 referring to this repo: https://github.com/AlexeyAB/darknet
The annotated dataset is ready for training. However, when the training beings, an error is shown saying that the GPU and OpenCV are not being used and the training stops abruptly.
Here is the error in the terminal:
GPU isn't used
OpenCV isn't used - data augmentation will be slow
valid: Using default 'data/train.txt'
yolov4-obj
mini_batch = 4, batch = 64, time_steps = 1, train = 1
layer filters size/strd(dil) input output
0 conv 32 3 x 3/ 1 416 x 416 x 3 -> 416 x 416 x 32 0.299 BF
1 conv 64 3 x 3/ 2 416 x 416 x 32 -> 208 x 208 x 64 1.595 BF
2 conv 64 1 x 1/ 1 208 x 208 x 64 -> 208 x 208 x 64 0.354 BF
3 route 1 -> 208 x 208 x 64
4 conv 64 1 x 1/ 1 208 x 208 x 64 -> 208 x 208 x 64 0.354 BF
5 conv 32 1 x 1/ 1 208 x 208 x 64 -> 208 x 208 x 32 0.177 BF
6 conv 64 3 x 3/ 1 208 x 208 x 32 -> 208 x 208 x 64 1.595 BF
......
......
Total BFLOPS 59.817
avg_outputs = 494379
Loading weights from yolov4.conv.137...
seen 64, trained: 0 K-images (0 Kilo-batches_64)
Done! Loaded 137 layers from weights-file
Learning Rate: 0.001, Momentum: 0.949, Decay: 0.0005
Detection layer: 139 - type = 28
Detection layer: 150 - type = 28
Detection layer: 161 - type = 28
Resizing, random_coef = 1.40
608 x 608
Create 64 permanent cpu-threads
mosaic=1 - compile Darknet with OpenCV for using mosaic=1
mosaic=1 - compile Darknet with OpenCV for using mosaic=1
Opencv is installed using brew install opencv#2 and M1 GPU works fine with TensorFlow. But due to some reasons, it is simply not working here.
Any help on this issue will be greatly appreciated.
Thank you in advance!

Related

nonlinear ODEs optimization with leastsq

[UPDATED] I'm working on a nonlinear ODEs system optimization and fitting it to experimental data. I have a system of 5 model ODEs which must be optimized by 17 parameters. My approach is to calculate the differences between solved ODEs and experimental data - function Differences, then use leastsq solver to minimize diferences and find the optimal parameters, as below code:
//RHSs of ODEs to be fitted:
function dx=model3(t,x,Kap,Ksa,Ko,Ks,Kia,Kis,p_Amax,q_Amax,qm,q_Smax,Yas,Yoa,Yxa,Yem,Yos,Yxsof,H)
X=x(1);
S=x(2);
A=x(3);
DO=x(4);
V=x(5);`
qs=((q_Smax*S/(S+Ks))*Kia/(Kia+A));
qsof=(p_Amax*qs/(qs+Kap));
qsox=(qs-qsof)*DO/(DO+Ko);
qsa=(q_Amax*A/(A+Ksa))*(Kis/(qs+Kis));
pa=qsof*Yas;
qa=pa-qsa;
qo=(qsox-qm)*Yos+qsa*Yoa;
u=(qsox-qm)*Yem+qsof*Yxsof+qsa*Yxa;
dx(1)=u*X-F*X/V;
dx(2)=(F*(Sf-S)/V)-qs*X;
dx(3)=qsa*X-(F*A/V);
dx(4)=200*(100-DO)-qo*X*H;
dx(5)=F;
endfunction
//experimental data:
//Dat=fscanfMat('dane_exper_III_etap.txt');
Dat = [
0 30 1.4 24.1 99 6884.754
1 35 0.2 23.2 89 6959.754
2 40 0.1 21.6 80 7034.754
3 52 0.1 19.5 67 7109.754
4 61 0.1 18.7 70 7184.754
5 66 0.1 16.4 79 7259.754
6 71 0.1 15 94 7334.754
7 74 0 14.3 100 7409.754
8 76 0 13.8 100 7484.754
9 78 0 13.4 100 7559.754
9.5 79 0 13.2 100 7597.254
10 79 0 13.5 100 7634.754]
t=Dat(:,1);
x_exp(:,1)=Dat(:,2);
x_exp(:,2)=Dat(:,3);
x_exp(:,3)=Dat(:,4);
x_exp(:,4)=Dat(:,5);
x_exp(:,5)=Dat(:,6);
global MYDATA;
MYDATA.t=t;
MYDATA.x_exp=x_exp;
MYDATA.funeval=0;
//calculating differences between calculated values and experimental data:
function f=Differences(k)
global MYDATA
t=MYDATA.t;
x_exp=MYDATA.x_exp;
Kap=k(1); //g/L
Ksa=k(2); //g/L
Ko=k(3); //g/L
Ks=k(4); //g/L
Kia=k(5); //g/L
Kis=k(6); //g/L
p_Amax=k(7); //g/(g*h)
q_Amax=k(8); //g/(g*h)
qm=k(9);
q_Smax=k(10);
Yas=k(11); //g/g
Yoa=k(12);
Yxa=k(13);
Yem=k(14);
Yos=k(15);
Yxsof=k(16);
H=k(17);
x0=x_exp(1,:);
t0=0;
F=75;
Sf=500;
%ODEOPTIONS=[1,0,0,%inf,0,2,10000,12,5,0,-1,-1]
x_calc=ode('rk',x0',t0,t,list(model3,Kap,Ksa,Ko,Ks,Kia,Kis,p_Amax,q_Amax,qm,q_Smax,Yas,Yoa,Yxa,Yem,Yos,Yxsof,H));
diffmat=x_calc'-x_exp;
//column vector of differences (concatenates 4 columns of the difference matrix)
f=diffmat(:);
MYDATA.funeval=MYDATA.funeval+1;
endfunction
// Initial guess
Kap=0.3; //g/L
Ksa=0.05; //g/L
Ko=0.1; //g/L
Ks=0.5; //g/L
Kia=0.5; //g/L
Kis=0.05; //g/L
p_Amax=0.4; //g/(g*h)
q_Amax=0.8; //g/(g*h)
qm=0.2;
q_Smax=0.6;
Yas=0.5; //g/g
Yoa=0.5;
Yxa=0.5;
Yem=0.5;
Yos=1.5;
Yxsof=0.22;
H=1000;
y0=[Kap;Ksa;Ko;Ks;Kia;Kis;p_Amax;q_Amax;qm;q_Smax;Yas;Yoa;Yxa;Yem;Yos;Yxsof;H];
yinf=[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,100];
ysup=[%inf,%inf,%inf,%inf,%inf,%inf,3,3,3,3,3,3,3,3,3,3,10000];
[fopt,xopt,gopt]=leastsq(Differences,'b',yinf,ysup,y0);
Now result is:
0.2994018
0.0508325
0.0999987
0.4994088
0.5081272
0.
0.4004560
0.7050746
0.2774195
0.6068328
0.5
0.4926150
0.4053860
0.5255006
1.5018725
0.2193901
1000.0000
33591.642
Running this script causes such an error:
lsoda-- caution... t (=r1) and h (=r2) are
such that t + h = t at next step
(h = pas). integration continues
where r1 is : 0.5658105345269D+01 and r2 : 0.1884898700920D-17
lsoda-- previous message precedent given i1 times
will no more be repeated
where i1 is : 10
lsoda-- at t (=r1), mxstep (=i1) steps
needed before reaching tout
where i1 is : 500000
where r1 is : 0.5658105345270D+01
Excessive work done on this call (perhaps wrong jacobian type).
at line 27 of function Differences
I understand that problem is on ODEs solving step. Thus, I have tried changing the mxstep, as also solving method type to 'adams','rk', and 'stiff' - none of this solved the problem. Using 'fix' method in ode I get this error:
ode: rksimp exit with state 3.
Please advise how to solve this?
P.S. Experimental data in file 'dane_exper_III_etap.txt':
0 30 1.4 24.1 99 6884.754
1 35 0.2 23.2 89 6959.754
2 40 0.1 21.6 80 7034.754
3 52 0.1 19.5 67 7109.754
4 61 0.1 18.7 70 7184.754
5 66 0.1 16.4 79 7259.754
6 71 0.1 15 94 7334.754
7 74 0 14.3 100 7409.754
8 76 0 13.8 100 7484.754
9 78 0 13.4 100 7559.754
9.5 79 0 13.2 100 7597.254
10 79 0 13.5 100 7634.754
In Scilab leastsq (based on optim) is very poor and doesn't have global convergence properties, unlike ipopt which is available as an atoms module. Install it like this:
--> atomsInstall sci_ipopt
I have modified your script in the following way
keep classical use of ode for this kind of biological kinetics, i.e. "stiff" (which uses BDF method). The Runge-Kutta you were using is very poor as it is an explicit method only for gentle ODEs.
use ipopt instead of leastsq
use a try/catch/end block around the computation of the residual in order to catch failing calls to the ode solver.
use some weight for the residual. You should play with it in order to improve the fit
use a strictly positive lower bound instead of 0, as very low value of some parameters make the ode solver fail.
add a drawing callback that also saves current value of parameters in case where you stop the optimization with ctrl-C.
//RHSs of ODEs to be fitted:
function dx=model3(t,x,Kap,Ksa,Ko,Ks,Kia,Kis,p_Amax,q_Amax,qm,q_Smax,Yas,Yoa,Yxa,Yem,Yos,Yxsof,H)
X=x(1);
S=x(2);
A=x(3);
DO=x(4);
V=x(5);
qs=((q_Smax*S/(S+Ks))*Kia/(Kia+A));
qsof=(p_Amax*qs/(qs+Kap));
qsox=(qs-qsof)*DO/(DO+Ko);
qsa=(q_Amax*A/(A+Ksa))*(Kis/(qs+Kis));
pa=qsof*Yas;
qa=pa-qsa;
qo=(qsox-qm)*Yos+qsa*Yoa;
u=(qsox-qm)*Yem+qsof*Yxsof+qsa*Yxa;
dx(1)=u*X-F*X/V;
dx(2)=(F*(Sf-S)/V)-qs*X;
dx(3)=qsa*X-(F*A/V);
dx(4)=200*(100-DO)-qo*X*H;
dx(5)=F;
endfunction
//calculating differences between calculated values and experimental data:
function [f,x_calc]=Differences(k, t, x_exp)
Kap=k(1); //g/L
Ksa=k(2); //g/L
Ko=k(3); //g/L
Ks=k(4); //g/L
Kia=k(5); //g/L
Kis=k(6); //g/L
p_Amax=k(7); //g/(g*h)
q_Amax=k(8); //g/(g*h)
qm=k(9);
q_Smax=k(10);
Yas=k(11); //g/g
Yoa=k(12);
Yxa=k(13);
Yem=k(14);
Yos=k(15);
Yxsof=k(16);
H=k(17);
x0=x_exp(1,:);
t0=0;
F=75;
Sf=500;
[x_calc]=ode("stiff",x0',t0,t,list(model3,Kap,Ksa,Ko,Ks,Kia,Kis,p_Amax,q_Amax,qm,q_Smax,Yas,Yoa,Yxa,Yem,Yos,Yxsof,H));
diffmat=(x_calc'-x_exp)*residual_weight;
//column vector of differences (concatenates 4 columns of the difference matrix)
f=diffmat(:);
MYDATA.funeval=MYDATA.funeval+1;
endfunction
function [f,g]=normdiff2(k,new_k,t,x_exp)
try
res = Differences(k,t,x_exp)
if argn(1) == 2
JacRes = numderivative(list(Differences,t,x_exp),k)
g = 2*JacRes'*res;
end
f = sum(res.*res)
catch
f=%inf;
g=%inf*ones(k);
end
endfunction
function out=callback(param)
global MYDATA
if isfield(param,"x")
k = param.x;
MYDATA.k = k;
[f,x_calc]=Differences(k,t,x_exp)
plot_weight = diag(1./max(x_exp,'r'));
drawlater
clf
plot(t,x_exp*plot_weight,'-o')
plot(t,x_calc'*plot_weight,'-x')
legend X S A DO X
drawnow
end
out = %t;
endfunction
//experimental data:
//Dat=fscanfMat('dane_exper_III_etap.txt');
Dat = [
0 30 1.4 24.1 99 6884.754
1 35 0.2 23.2 89 6959.754
2 40 0.1 21.6 80 7034.754
3 52 0.1 19.5 67 7109.754
4 61 0.1 18.7 70 7184.754
5 66 0.1 16.4 79 7259.754
6 71 0.1 15 94 7334.754
7 74 0 14.3 100 7409.754
8 76 0 13.8 100 7484.754
9 78 0 13.4 100 7559.754
9.5 79 0 13.2 100 7597.254
10 79 0 13.5 100 7634.754]
t=Dat(:,1);
x_exp(:,1)=Dat(:,2);
x_exp(:,2)=Dat(:,3);
x_exp(:,3)=Dat(:,4);
x_exp(:,4)=Dat(:,5);
x_exp(:,5)=Dat(:,6);
global MYDATA;
MYDATA.funeval=0;
// Initial guess
Kap=0.3; //g/L
Ksa=0.05; //g/L
Ko=0.1; //g/L
Ks=0.5; //g/L
Kia=0.5; //g/L
Kis=0.05; //g/L
p_Amax=0.4; //g/(g*h)
q_Amax=0.8; //g/(g*h)
qm=0.2;
q_Smax=0.6;
Yas=0.5; //g/g
Yoa=0.5;
Yxa=0.5;
Yem=0.5;
Yos=1.5;
Yxsof=0.22;
H=100;
k0 = [Kap;Ksa;Ko;Ks;Kia;Kis;p_Amax;q_Amax;qm;q_Smax;Yas;Yoa;Yxa;Yem;Yos;Yxsof;H];
residual_weight = diag(1./[79,1.4, 24.1, 100, 7634.754]);
BIG = 1000;
SMALL = 1e-3;
problem = struct();
problem.f = list(normdiff2,t,x_exp);
problem.x0 = k0;
problem.x_lower = [SMALL*ones(16,1);100];
problem.x_upper = [BIG,BIG,BIG,BIG,BIG,BIG,3,3,3,3,3,3,3,3,3,3,10000]';
problem.int_cb = callback;
problem.params = struct("max_iter",200);
//
k = ipopt(problem)
Here is a plot of the results after 100 iterations (you can change the value in the ipopt options). However, don't expect normal termination as
it is almost sure that your parameter set is not identifiable
finite difference gradient with ODEs is very innacurate.
Hope it will help you a little bit...

multi-dimensional indexing warning with pandas

x = df.x_value
y = df.y_value
x = x[:, np.newaxis]
y = y[:, np.newaxis]
polynomial_features= PolynomialFeatures(degree=2)
x_transformed = polynomial_features.fit_transform(x)
The above code is giving following warning...how can I avoid these
FutureWarning: Support for multi-dimensional indexing (e.g. `obj[:, None]`) is deprecated and will be removed in a future version. Convert to a numpy array before indexing instead.
A full working example with solution as suggested by the warning:
In [194]: df
Out[194]:
age rank height weight
0 20 2 155 53
1 15 7 159 60
2 34 6 180 75
3 40 5 163 80
4 60 1 170 49
In [195]: df.height
Out[195]:
0 155
1 159
2 180
3 163
4 170
Name: height, dtype: int64
In [196]: df.height[:,None]
<ipython-input-196-1af0bb09495a>:1: FutureWarning: Support for multi-dimensional indexing (e.g. `obj[:, None]`) is deprecated and will be removed in a future version. Convert to a numpy array before indexing instead.
df.height[:,None]
Out[196]:
array([[155],
[159],
[180],
[163],
[170]])
In [197]: df.height.to_numpy()[:,None]
Out[197]:
array([[155],
[159],
[180],
[163],
[170]])

Right parameters for strip_unused_nodes

Tensorflow Graph Transforms page https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/graph_transforms/README.md shows how to use strip_unused_nodes.
But how to know the right values of X and Y in strip_unused_nodes(type=X, shape="y0,y1,y3,3") for my model?
Output of summarize_graph on my MobileNetV2 model :
Found 1 possible inputs: (name=image_tensor, type=uint8(4), shape=[?,?,?,3])
No variables spotted.
Found 4 possible outputs: (name=detection_boxes, op=Identity) (name=detection_scores, op=Identity) (name=detection_classes, op=Identity) (name=num_detections, op=Identity)
Found 3457096 (3.46M) const parameters, 0 (0) variable parameters, and 623 control_edges
Op types used: 1707 Const, 525 Identity, 277 Mul, 194 Add, 170 Reshape, 147 GatherV2, 133 Sub, 117 Minimum, 98 Slice, 92 Maximum, 77 ConcatV2, 77 Cast, 64 Rsqrt, 60 StridedSlice, 59 Relu6, 55 Conv2D, 54 Pack, 52 Greater, 49 Shape, 46 Split, 46 Where, 45 ExpandDims, 40 Fill, 37 Tile, 33 RealDiv, 33 DepthwiseConv2dNative, 30 Range, 29 Switch, 27 Unpack, 26 Enter, 25 Squeeze, 25 ZerosLike, 23 NonMaxSuppressionV2, 14 Merge, 12 BiasAdd, 12 FusedBatchNorm, 11 TensorArrayV3, 8 NextIteration, 6 TensorArrayWriteV3, 6 TensorArraySizeV3, 6 Sqrt, 6 Exit, 6 TensorArrayGatherV3, 5 TensorArrayScatterV3, 5 TensorArrayReadV3, 3 Rank, 3 Equal, 3 Transpose, 3 Assert, 2 Exp, 2 Less, 2 LoopCond, 1 All, 1 TopKV2, 1 Size, 1 Sigmoid, 1 ResizeBilinear, 1 Placeholder
To use with tensorflow/tools/benchmark:benchmark_model try these arguments:
bazel run tensorflow/tools/benchmark:benchmark_model -- --graph=/home/ubuntu/model-optimization/frozen_inference_graph.pb --show_flops --input_layer=image_tensor --input_layer_type=uint8 --input_layer_shape=-1,-1,-1,3 --output_layer=detection_boxes,detection_scores,detection_classes,num_detections
I believe you should copy the input layer dims, you can find in the .ascii file of your model

Keras (tf backend) memory allocation problems

I am using Keras with Tensorflow backend.
I am facing a batch size limitation due to high memory usage
My data is composed of 4 1D signals treated with a sample size of 801 for each channel. Global sample size is 3204
Input data:
4 channels of N 1D signals of length 7003
Input generated by applying a sliding window on 1D signals
Give input data shape (N*6203, 801, 4)
N is the number of signals used to build one batch
My Model:
Input 801 x 4
Conv2D 5 x 1, 20 channels
MaxPooling 2 x 1
Conv2D 5 x 1, 20 channels
MaxPooling 2 x 1
Conv2D 5 x 1, 20 channels
MaxPooling 2 x 1
Conv2D 5 x 1, 20 channels
Flatten
Dense 2000
Dense 5
With my GPU (Quadro K6000, 12189 MiB) i can fit only N=2 without warning
With N=3 I get a ran out of memory warning
With N=4 I get a ran out of memory error
It sound like batch_size is limitated by the space used by all tensors.
Input 801 x 4 x 1
Conv 797 x 4 x 20
MaxPooling 398 x 4 x 20
Conv 394 x 4 x 20
MaxPooling 197 x 4 x 20
Conv 193 x 4 x 20
MaxPooling 96 x 4 x 20
Conv 92 x 4 x 20
Dense 2000
Dense 5
With a 1D signal of 7001 with 4 channels -> 6201 samples
Total = N*4224 MiB.
N=2 -> 8448 MiB fit in GPU
N=3 -> 12672 MiB work but warning: failed to allocate 1.10 GiB then 3.00 GiB
N=4 -> 16896 MiB fail, only one message: failed to allocate 5.89 GiB
Does it work like that ? Is there any way to reduce the memory usage ?
To give a time estimation: 34 batch run in 40s and I got N total = 10^6
Thank you for your help :)
Example with python2.7: https://drive.google.com/open?id=1N7K_bxblC97FejozL4g7J_rl6-b9ScCn

I want to find the minumum point of any discrete function by using gradient descent method

I want to find the minimum point of any discrete function by using gradient descent method but I don't know how can I evaluate derivative of the function for a specific point. For example
function = [100 81 64 49 36 25 16 9 4 1 0 1 4 9 16 25 36 49 64 81 100]
derivative = [ -19 -17 -15 -13 -11 -9 -7 -5 -3 -1 1 3 5 7 9 11 13 15 17 19]
for gradient descent algorithm
x_next = x_now - alfa * derivative(x_now)
but how can I evaluate derivative(x_now)? I hope you will understand what I mean thanks
Assuming that you know the x values corresponding to your discrete function values, e.g.,
x = [-5 -4 -3 -2 ... ],
function = [100 81 64 49 36 ... ],
derivative = [-19 -17 -15 -13 ...],
the derivative in the gradient descent method can taken as constant over each interval [x_(i), x_(i+1)). I.e., in pseudocode
derivative(x_now) = derivative(x_i), where x_(i) <= x_now < x_(i+1).
As and example, using the x values above:
derivative(-3.5) = -15