I have a data set of:
32
33
34
35
34
32
29
28
27
25
29
32
34
35
36
28
27
28
28
I would like to be able to find out how many numbers in a row are above 32. For example an output like:
5
4
where 5 is the first instance the values are above 32, and 4 is the second instance the values are over 32. I have been trying to do this in awk but so far all I am getting is the collective number i.e. 9 for all value combined above 32.
Any help would be much appreciated.
awk to the rescue! I think your output is not consistent with the input, or I misunderstood the problem. This is computing the chain length of values >31
$ awk '$1>31{c++; next} c{print c; c=0} END{if(c) print c}' file
6
4
END block is required for the case if the last chain contains the last element.
I need to read a file and store column 1 and 4, look in a second file using column one and store column 4 of the second file and then do a subtraction with between column 04 of file 01 and column 04 of file 2 . Can you help me? Column 04 is in seconds.
The two files contain the following headers.
ID, origin, destination, time
I need to get the first ID in file 1, and look in file 2.
For example, take ID 37 from file 1 and look at file 2. When I find it, I need the ID 37 time in the first file to be subtracted from the ID 37 time in file 2
I need the sum of subtraction times.
Wondering if awk is right solution
File 01
37 33 44 602.04
39 32 13 602.20
File 02
37 44 44 602.184852493
39 13 13 602.263704529
Output
0,2
One possibility to consider is splitting the task up into two parts - joining the two files based on that common field, and then doing the math. It avoids having to store part of every line from one file in memory all at once, which is nice if they're big.
The following assumes that a) the files are sorted based on the first column, b) that tabs are used to separate the columns:
$ join -j1 -o '1.4 2.4' file1.txt file2.txt | awk '{total+=$2-$1} END {print total}'
0.208557
The join command merges the two files on common lines and prints out just the numbers you want to subtract, which are piped to awk to do the actual math.
Edit: Or all in awk:
$ awk 'NR==FNR { f1[$1]=$4; next }
$1 in f1 { total += $4 - f1[$1] }
END { print total }' file1.txt file2.txt
0.208557
this stores the ids and times from the first file in an associative array, and then for each line in file 2, if that line's id exists in the array, add the difference of times to the total. Finally, print the total after reading all of the file.
f1.col4 - f2.col4:
awk 'NR==FNR{a[$1]=$4;next}{$4=a[$1]?a[$1]-$4:$4}7' f1 f2
The output looks like:
37 44 44 -0.144852
39 13 13 -0.0637045
41 44 44 -0.0642587
44 13 13 -0.0196296
45 44 44 -0.0145357
47 13 13 -0.014259
If you want the f2.col4 - f1.col4, use $4-a[$1] in the above code, you get:
37 44 44 0.144852
39 13 13 0.0637045
41 44 44 0.0642587
44 13 13 0.0196296
45 44 44 0.0145357
47 13 13 0.0142594
In a DataFrame, I have negative numbers, and also missing values that are given by a - . I want to replace the missing values with an empty cell, but this operation should NOT remove the - in front of the negative numbers.
It looks like:
45 45 45 45 45 45 45 45 45 45
45 45 15 31 43 45 45 45 45 45
44.24 121.55 1.80 0.00% - 97.63 -4.87 -6.02 -20.14 169.19
1 1 7 12 3 1 1 1 1 1
So the missing value cell with the - should be empty, but the -4.87 should stay intact.
Any help would be greatly appreciated.
The problem should have been addressed at the time of loading the file into the DataFrame (by providing the na_values parameter to read_csv() or whatever function you used).
At this point, use operation replace(): it replaces whole words, not individual characters.
df = df.replace("-", np.nan)
I have a file separated by \t.
header text with many lines
V F A B
10 30 26 42
14 33 25 45
16 32 23 43
18 37 22 48
I want to change the 3rd column by the 4th and vice versa. I'm using
awk '
BEGIN {
RS = "\n";
OFS="\t";
record=0;
};
record {
a = $4;
$4 = $3;
$3 = a;
};
$1=="V" {
record=1
};
{
print $0
};
'
}
Instead of just changing the position of the columns, column 3 also has the line break of the original 4th column:
header text with many lines
V F A B
10 30 42
26
14 33 45
25
16 32 43
23
18 37 48
22
How can I prevent this in order to get?
header text with many lines
V F A B
10 30 42 26
14 33 45 25
16 32 43 23
18 37 48 22
Could you please try following, using usual method of storing 1 field's value to a variable and then exchanging the value of 4th field to 3rd field, at last putting 4th field value as variable value(could say swapping values using a variable).
awk 'FNR==1{print;next} {val=$3;$3=$4;$4=val} 1' OFS="\t" Input_file
Or, this messy sed:
sed -E 's/([[:digit:]]+)([[:blank:]]+)([[:digit:]]+)([[:space:]]*)$/\3\2\1\4/' file
# ^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^
# 3rd column tab 4th column optional whitespce
Tensorflow Graph Transforms page https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/graph_transforms/README.md shows how to use strip_unused_nodes.
But how to know the right values of X and Y in strip_unused_nodes(type=X, shape="y0,y1,y3,3") for my model?
Output of summarize_graph on my MobileNetV2 model :
Found 1 possible inputs: (name=image_tensor, type=uint8(4), shape=[?,?,?,3])
No variables spotted.
Found 4 possible outputs: (name=detection_boxes, op=Identity) (name=detection_scores, op=Identity) (name=detection_classes, op=Identity) (name=num_detections, op=Identity)
Found 3457096 (3.46M) const parameters, 0 (0) variable parameters, and 623 control_edges
Op types used: 1707 Const, 525 Identity, 277 Mul, 194 Add, 170 Reshape, 147 GatherV2, 133 Sub, 117 Minimum, 98 Slice, 92 Maximum, 77 ConcatV2, 77 Cast, 64 Rsqrt, 60 StridedSlice, 59 Relu6, 55 Conv2D, 54 Pack, 52 Greater, 49 Shape, 46 Split, 46 Where, 45 ExpandDims, 40 Fill, 37 Tile, 33 RealDiv, 33 DepthwiseConv2dNative, 30 Range, 29 Switch, 27 Unpack, 26 Enter, 25 Squeeze, 25 ZerosLike, 23 NonMaxSuppressionV2, 14 Merge, 12 BiasAdd, 12 FusedBatchNorm, 11 TensorArrayV3, 8 NextIteration, 6 TensorArrayWriteV3, 6 TensorArraySizeV3, 6 Sqrt, 6 Exit, 6 TensorArrayGatherV3, 5 TensorArrayScatterV3, 5 TensorArrayReadV3, 3 Rank, 3 Equal, 3 Transpose, 3 Assert, 2 Exp, 2 Less, 2 LoopCond, 1 All, 1 TopKV2, 1 Size, 1 Sigmoid, 1 ResizeBilinear, 1 Placeholder
To use with tensorflow/tools/benchmark:benchmark_model try these arguments:
bazel run tensorflow/tools/benchmark:benchmark_model -- --graph=/home/ubuntu/model-optimization/frozen_inference_graph.pb --show_flops --input_layer=image_tensor --input_layer_type=uint8 --input_layer_shape=-1,-1,-1,3 --output_layer=detection_boxes,detection_scores,detection_classes,num_detections
I believe you should copy the input layer dims, you can find in the .ascii file of your model