I have a dataset of some small fibrous objects on which I tried with YOLOv5OBB method but the results are not satisfying.
The detection result is poor (P=0.3, R=0.28 for total boxes = 1200 in 120 images).
The boxes are not fit well to the objects and often there are overlapped boxes in different orientation.
The considered fibrous objects include fibrous with less than 3 microns in diameters and longer than 5 microns, therefore I can not use scaling and shearing which change the size of the objects. traslate and mosaic will also damage the results as objects are cut.
Do you have any suggestion how to improve it? hyp parameters or even a different method/model?
Some output examples:
1
2
Hyp parameters:
lr0: 0.00258
lrf: 0.017
momentum: 0.779
weight_decay: 0.00058
warmup_epochs: 1.93
warmup_momentum: 0.443
warmup_bias_lr: 0.059
box: 0.0339
cls: 0.33
cls_pw: 0.825
theta: 0.299
theta_pw: 0.825
obj: 0.632
obj_pw: 1.0
iou_t: 0.2
anchor_t: 3.44
anchors: 2.5
fl_gamma: 0.0
hsv_h: 0.0188
hsv_s: 0.354
hsv_v: 0.35
degrees: 0.03
translate: 0.05
scale: 0.02
shear: 0.0
perspective: 0.0
flipud: 0.7
fliplr: 0.7
mosaic: 0.0
mixup: 0.5
copy_paste: 0.4
cls_theta: 180
csl_radius: 2.0
Related
Should I encode a categorical column and use label encoding, then impute NaN values with most frequent value, or are there other ways?
As encoding requires converting dataframe to array, then imputing would require again array to dataframe conversion (all this for a single column, and there are more columns like that).
Fore example, I have the variable BsmtQual which evaluates the height of a basement and has following number of categories:
Ex Excellent (100+ inches)
Gd Good (90-99 inches)
TA Typical (80-89 inches)
Fa Fair (70-79 inches)
Po Poor (<70 inches
NA No Basement
Out of 2919 values in BsmtQual, 81 are NaN values.
For problems you have in the future like this that don't involve coding you should post at https://datascience.stackexchange.com/.
This depends on a few things. First of all, how important is this variable in your exercise? Assuming that you are doing classification, you could try removing all rows without with NaN values, running a few models, then removing the variable and running the same models again. If you haven't seen a dip in accuracy, then you might consider removing the variable completely.
If you do see a dip in accuracy or can't judge impact due to the problem being unsupervised, then there are several other methods you can try. If you just want a quick fix, and if there aren't too many NaNs or categories, then you can just impute with the most frequent value. This shouldn't cause too many problems if the previous conditions are satisfied.
If you want to be more exact, then you could consider using the other variables you have to predict the class of the categorical variable (obviously this will only work if the categorical variable is correlated to some of your other variables). You could use a variety of algorithms for this, including classifiers or clustering. It all depends on the distribution of your categorical variable and how much effort you want to put it in to solve your issue.
(I'm only learning as well, however I think thats most of your options)
"… or there are other ways."
Example:
Ex Excellent (100+ inches) 5 / 5 = 1.0
Gd Good (90-99 inches) 4 / 5 = 0.8
TA Typical (80-89 inches) 3 / 5 = 0.6
Fa Fair (70-79 inches) 2 / 5 = 0.4
Po Poor (<70 inches 1 / 5 = 0.2
NA No Basement 0 / 5 = 0.0
However, labels express less precision (affects accuracy if combined with actual measurements).
Could be solved by either scaling values over category range (e.g. scaling 0 - 69 inches over 0.0 - 0.2), or by approximation value for each category (more linearly accurate). For example, if highest value is 200 inch:
Ex Excellent (100+ inches) 100 / 200 = 0.5000
Gd Good (90-99 inches) ((99 - 90) / 2) + 90 / 200 = 0.4725
TA Typical (80-89 inches) ((89 - 80) / 2) + 80 / 200 = 0.4225
Fa Fair (70-79 inches) ((79 - 70) / 2) + 70 / 200 = 0.3725
Po Poor (<70 inches (69 / 2) / 200 = 0.1725
NA No Basement 0 / 200 = 0.0000
Actual measurement 120 inch 120 / 200 = 0.6000
Produces decent approximation (range mid-point value, except Ex, which is a minimum value). If calculations on such columns produce inaccuracies it is for notation imprecision (labels express ranges rather than values).
I would like to know how to display specific contour levels on the colorbar. For example, as shown in the schematic above taken from pivotalweather, shows a colorbar for precipitation values that are not really equally spaced. I would like to know how to achieve a similar result with GrADS.
PS: I use the cbarn.gs and the xcbar.gs script sometimes.
You need to use the original color set of GRADS for this.
THREE steps:
1). Set the color using the 'set rgb # R G B'. You need the RGB of the colors in your color bar. Since there are 15 default colors in GrADS, you should start the # at 16.
Check this link for details of the colors:
http://cola.gmu.edu/grads/gadoc/colorcontrol.html
2). You need to set the color level as follows:
set clevs 0.01 0.05 0.1 0.02 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.2 1.4 1.6 1.8 2
2.5 3 3.5 4 5 6 8 15
3). You need to specify the colors based on your defined RGBs.
set ccols 16, 17, 18,....etc.
Could anyone help me to find the error in this AMPL's code for a simple least-square error base on the function:
F(X)=1/1+e^-x
param N>=1;# N Number of simulations
param M>=1;# Number of inputs
param simulations {1..N};
param training{1..N,1..M};
var W{1..10};
minimize obj: sum{i in simulations , j in 1..4} (1/(1+exp-(W[9]/(1+exp(-
W[j]/(1+exp(-training[i][j]))))+ W[10]/(1+exp(-W[2*j]/(1+exp(-training[i][j]))))))-training[i][5])^2;
'###### DATA
param N:=6;
param M:=4;
param training:
1 2 3 4 5 :=
1 0.209 0.555 0.644 0.355 0.0
2 0.707 0.450 0.587 0.305 1.0
3 0.579 0.521 0.745 0.394 1.0
4 0.574 0.883 0.211 0.550 1.0
5 0.797 0.055 0.430 0.937 1.0
6 0.782 0.865 0.114 0.317 1.0 ;
Thank you!
A couple of things:
is that quote mark before ###### DATA meant to be there?
You have specified that training has dimension N x M, and your data specifies that N=6, M=4, but you then define training as 6 x 5 and your objective function also refers to column 5.
If that doesn't answer your question, you might want to give more information about what error messages you're getting.
I'm building a DNN predicted (0 or 1) model based on skflow with TF v0.9.
My code with TensorFlowDNNClassifier is like this. I train about 26,000 records and test 6,500 one.
classifier = learn.TensorFlowDNNClassifier(hidden_units=[64, 128, 64], n_classes=2)
classifier.fit(features, labels, steps=50000)
test_pred = classifier.predict(test_features)
print(classification_report(test_labels, test_pred))
It takes about 1 minute and gets a result.
precision recall f1-score support
0 0.77 0.92 0.84 4265
1 0.75 0.47 0.58 2231
avg / total 0.76 0.76 0.75 6496
But I got
WARNING:tensorflow:TensorFlowDNNClassifier class is deprecated.
Please consider using DNNClassifier as an alternative.
So I updated my code with DNNClassifier simply.
classifier = learn.DNNClassifier(hidden_units=[64, 128, 64], n_classes=2)
classifier.fit(features, labels, steps=50000)
It also works well. But result was not the same.
precision recall f1-score support
0 0.77 0.96 0.86 4265
1 0.86 0.45 0.59 2231
avg / total 0.80 0.79 0.76 6496
1 's precision is improved.
Of course this is a good for me, but why it is improved?
And It takes about 2 hours.
This is about 120 times slower than previous example.
Do I have something wrong? or miss some parameters?
Or is DNNClassifier unstable with TF v0.9?
I give the same answer as here. You might experience that because you used the steps parameter instead of max_steps. It was just steps on TensorFlowDNNClassifier that in reality did max_steps. Now you can decide if you really want that in your case 50000 steps or auto abort earlier.
Let's say I have a 5 x 5 array of floating points in a file array.txt:
1.0 1.1 0.0 0.0 0.0
1.2 1.3 1.4 0.0 0.0
0.0 1.5 1.6 1.7 0.0
0.0 0.0 1.8 1.9 1.0
0.0 0.0 0.0 1.1 1.2
I know this is probably a strange thing to do, but I'm just trying to learn the read statements better: I want to create two 3x3 arrays in Fortran, i.e. real, dimension(3,3) :: array1, array2 and try reading in the first 9 values by row into array1 and the following 9 values into array2. That is, I would like arrays to have the form
array1 = 1.0 1.1 0.0
0.0 0.0 1.2
1.3 1.4 0.0
array2 = 0.0 0.0 1.5
1.6 1.7 0.0
0.0 0.0 1.8
Next I want to try to do the same by columns:
array1 = 1.0 1.2 0.0
0.0 0.0 1.1
1.3 1.5 0.0
array2 = 0.0 0.0 1.4
1.6 1.8 0.0
0.0 0.0 1.7
My "closest" attempt for row-wise:
program scratch
implicit none
real, dimension(3,3) :: array1, array2
integer :: i
open(12, file="array.txt")
!read in values
do i = 1,3
read(12,'(3F4.1)', advance="no") array1(i,:)
end do
end program scratch
My questions:
A. How to advance to next record when at the end?
B. How to do the same for reading in column-wise?
C. Why is '(3F4.1)' needed, as opposed to '(3F3.1)'?
Reading by line is easy :
READ(12,*) ((array1(i,j),j=1,3),i=1,3),((array2(i,j),j=1,3),i=1,3)
"advance='no'" is necessary only if you use 2 read statements instead of 1 (and only on the first READ). But this works only with explicit format ...
Reading a file by column is not so obvious, especially because reading a file is usually an expensive task. I suggest you read the file in a larger table and then you distribute the values into your two arrays. For instance :
real :: table(5,5)
integer :: i,j,ii,jj,k
..
read(12,*) ((table(i,j),j=1,5),i=1,5)
k=0
do j=1,3
do i=1,3
k=k+1
jj=(k-1)/5+1
ii=k-(jj-1)*5
array1(i,j)=table(ii,jj)
enddo
enddo
do j=1,3
do i=1,3
k=k+1
jj=(k-1)/5+1
ii=k-(jj-1)*5
array2(i,j)=table(ii,jj)
enddo
enddo
(3F4.1) is better than (3F3.1) because each number occupies 4 bytes in fact (3 for the digits and 1 for the space between numbers). But as you see, I have used * which avoids to think about such detail.
because of the requirement to "assign by columns" i would advise reading the whole works into a 5x5 array:
real tmp(5,5)
read(unit,*)tmp
(note no format specification required)
Then do the assignments you need using array operations.
for this small array, the simplest thing to do seems to be:
real tmp(5,5),flat(25),array1(3,3),array2(3,3)
read(unit,*)tmp
flat=reshape(tmp,shape(flat))
array1=reshape(flat(:9),shape(array1))
array2=reshape(flat(10:18),shape(array2))
then the transposed version is simply:
flat=reshape(transpose(tmp),shape(flat))
array1=reshape(flat(:9),shape(array1))
array2=reshape(flat(10:18),shape(array2))
If it was a really big array I'd think of a way that avoided making an extra copy of the data.
Note you can wrap each of those assignments in transpose if needed depending on
how you really want the data represented, eg.
array1=transpose(reshape(flat(:9),shape(array1)))