awk mixed trailing and leading underscore - awk

To keep lines which have numbers below 100(hundred) on their third field, how can I deal with trailing and leading underscores using awk script.
Here is sample input:
asd 5.0 _84_ 34
sds 3.8 _340_ 56
sds 3.8 99 56
sds 3.8 _340_ 56
ddd 288 _26_ 45
sdf 321 _374_ 53
tyu 321 _39_ 89
sds 3.8 120 56
wer 3.8 _28_ 345
sds 3.8 45 56
tre 230 _265_ 90
rtu 389 _372_ 45
bnm 965 _270_ 745
fgh 856 _44_ 346
sds 3.8 107 56
expected output will be:
asd 5.0 _84_ 34
sds 3.8 99 56
ddd 288 _26_ 45
tyu 321 _39_ 89
wer 3.8 _28_ 345
sds 3.8 45 56
fgh 856 _44_ 346


awk '$3 ~ /^_?[[:digit:]]{1,2}_?$/'
That will match an optional underscore, one or two digits followed by an optional underscore.


try this awk one-liner:
kent$ awk -F'_| +' 'NF==4&&$3<100||NF>=6&&$4<100' file
asd 5.0 _84_ 34
sds 3.8 99 56
ddd 288 _26_ 45
tyu 321 _39_ 89
wer 3.8 _28_ 345
sds 3.8 45 56
fgh 856 _44_ 346


Using gawk:
awk 'match($3, /([0-9]+)/, a) {if (a[1]<100) print $0}' input


$ awk '{line=$0; gsub(/_/,"")} $3 < 100{print line}' file
asd 5.0 _84_ 34
sds 3.8 99 56
ddd 288 _26_ 45
tyu 321 _39_ 89
wer 3.8 _28_ 345
sds 3.8 45 56
fgh 856 _44_ 346

Related

Printf in awk with the same space between columns

I have a data file with five columns, when I use the printf command in awk, the output isn't aligned.
118 96 105 106 0
119 97 106 107 0
120 98 107 108 0
121 99 108 109 0
122 100 109 110 0
123 101 110 111 0
124 102 111 23 0
125 11 12 112 0
126 103 112 113 0
127 104 113 114 0
128 105 114 115 0
I need to use all columns with the same space separator, independent is the number are tens, hundreds, or thousands like this:
118 96 105 106 0
119 97 106 107 0
120 98 107 108 0
121 99 108 109 0
122 100 109 110 0
123 101 110 111 0
124 102 111 23 0
125 11 12 112 0
126 103 112 113 0
127 104 113 114 0
128 105 114 115 0
How can I do this using printf in awk?
I'm using this:
awk '{printf "%d %s %d %s %d %s %d %s %d\n", $1,"",$2,"",$3,"",$4,"",$5}' test

Use column on the output rather than trying to format it with awk:
$ column -t -R'1,2,3,4,5' file
118 96 105 106 0
119 97 106 107 0
120 98 107 108 0
121 99 108 109 0
122 100 109 110 0
123 101 110 111 0
124 102 111 23 0
125 11 12 112 0
126 103 112 113 0
127 104 113 114 0
128 105 114 115 0
Your version of column may already support -R0 which means "right align all columns" so you don't need to list them, see https://github.com/util-linux/util-linux/issues/1306.
As #glennjackman pointed out in comments:
The BSD-derived column on MacOS does not have the -R option. Have to do rev file | column -t | rev on a mac

Most printf formats allow the width to be supplied as an argument to a * format specifier, eg:
printf "%*s", 5 "abc"
Is evaulated as:
printf "%5s, "abc"
One awk idea making use of this printf/* feature:
awk '
FNR==NR { for (i=1;i<=NF;i++)
w[i]= length($i) > w[i] ? length($i) : w[i] # find max width for each column
next
}
{ pfx=""
for (i=1;i<=NF;i++) {
printf "%s%*s", pfx, w[i], $i
pfx=" " # (aligned) column delimiter == 2 spaces for columns 2 to NF
}
print "" # terminate current line
}
' five.dat five.dat
NOTES:
requires 2 passes of the input file (could be rewritten to use a single pass but will need to store the entire file in memory)
assumes the minimum delimiter between (aligned) columns is 2 spaces
This generates:
118 96 105 106 0
119 97 106 107 0
120 98 107 108 0
121 99 108 109 0
122 100 109 110 0
123 101 110 111 0
124 102 111 23 0
125 11 12 112 0
126 103 112 113 0
127 104 113 114 0
128 105 114 115 0

The 3 easiest solution are: 1) pipe the output to column -t, 2) use a tab separator (doesn't completely align the text, but for the sample input is sufficient, and 3) print each column on a fixed width.
$ cat input
118 96 105 106 0
119 97 106 107 0
120 98 107 108 0
121 99 108 109 0
122 100 109 110 0
123 101 110 111 0
124 102 111 23 0
125 11 12 112 0
126 103 112 113 0
127 104 113 114 0
128 105 114 115 0
$ awk '($1=$1) || 1' OFS=\\t input
118 96 105 106 0
119 97 106 107 0
120 98 107 108 0
121 99 108 109 0
122 100 109 110 0
123 101 110 111 0
124 102 111 23 0
125 11 12 112 0
126 103 112 113 0
127 104 113 114 0
128 105 114 115 0
$ awk '{printf "%5s%5s%5s%5s%5s\n", $1, $2, $3, $4, $5}' input
118 96 105 106 0
119 97 106 107 0
120 98 107 108 0
121 99 108 109 0
122 100 109 110 0
123 101 110 111 0
124 102 111 23 0
125 11 12 112 0
126 103 112 113 0
127 104 113 114 0
128 105 114 115 0

Here's one that requires two passes of the data (hence the file file in the end):
$ awk 'NR==FNR { # first pass
for(i=1;i<=NF;i++)
if(m[i]=="" || m[i]<length($i)) # get max field widths
m[i]=length($i)
next
}
{ # second pass
for(i=1;i<=NF;i++)
printf "%" m[i] "s%s",$i,(i==NF?ORS:" ") # output two spaces in between
}' file file # two passes, twice the file
Output:
118 96 105 106 0
119 97 106 107 0
120 98 107 108 0
121 99 108 109 0
122 100 109 110 0
123 101 110 111 0
124 102 111 23 0
125 11 12 112 0
126 103 112 113 0
127 104 113 114 0
128 105 114 115 0

Here is an awk to do that:
awk 'FNR==NR{for(i=1;i<=NF;i++) if (w[i]<length($i)) w[i]=length($i); next}
{for(i=1;i<=NF;i++) printf("%*s%s", w[i], $i, i<NF ? OFS : ORS)}
' file file
Prints:
118 96 105 106 0
119 97 106 107 0
120 98 107 108 0
121 99 108 109 0
122 100 109 110 0
123 101 110 111 0
124 102 111 23 0
125 11 12 112 0
126 103 112 113 0
127 104 113 114 0
128 105 114 115 0
Then if you want to field size to increase, just add that:
awk -v wp=5 'FNR==NR{for(i=1;i<=NF;i++) if (w[i]<length($i)+wp) w[i]=length($i)+wp; next}
{for(i=1;i<=NF;i++) printf("%*s%s", w[i], $i, i<NF ? OFS : ORS)}
' file file
118 96 105 106 0
119 97 106 107 0
120 98 107 108 0
121 99 108 109 0
122 100 109 110 0
123 101 110 111 0
124 102 111 23 0
125 11 12 112 0
126 103 112 113 0
127 104 113 114 0
128 105 114 115 0

How to pick a specific columns of header of even number only?

I want to ask a question on how to call on the specific columns that only contains an even numbers.
On my previous questions :How to make all the rows data drop the similar data and multiplying float numbers.
df2 =df['hlogUs_dB'].str.split('[,:]',expand = True)
df2 = data.drop(["0"])
df2
0 1 2 3 4 5 6 7 8 9 ... 276 277 278 279 280 281 282 283 284 285
0 109 -3.4 110 -3.4 111 -3.4 112 -3.5 113 -3.5 ... 343 -4.3 344 -4.3 345 -4.2 346 -4.2 347 -4.2
1 109 -3.5 110 -3.5 111 -3.4 112 -3.4 113 -3.4 ... 343 -4.1 344 -4.2 345 -4.4 346 -4.4 347 -4.2
2 109 -3.7 110 -3.7 111 -3.8 112 -3.8 113 -3.8 ... 343 -4.2 344 -4.3 345 -4.3 346 -4.3 347 -4.3
3 109 -3.5 110 -3.6 111 -3.6 112 -3.6 113 -3.7 ... 343 -4.1 344 -4.1 345 -4.1 346 -4.1 347 -4.1
4 109 -3.7 110 -3.8 111 -3.8 112 -3.8 113 -3.8 ... 343 -4.2 344 -4.2 345 -4.2 346 -4.2 347 -4.3
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
165 109 -5.2 110 -5.3 111 -5.5 112 -5.7 113 -5.9 ... 343 -5.4 344 -5.3 345 -5.2 346 -5.1 347 -5.1
166 109 -5.5 110 -5.6 111 -5.8 112 -6.1 113 -6.3 ... 343 -5.5 344 -5.4 345 -5.3 346 -5.2 347 -5.2
167 109 -6.0 110 -6.2 111 -6.4 112 -6.7 113 -7.1 ... 343 -4.9 344 -4.9 345 -4.9 346 -4.9 347 -4.9
168 109 -5.4 110 -5.5 111 -5.7 112 -5.9 113 -6.2 ... 343 -5.9 344 -5.7 345 -5.7 346 -5.6 347 -5.6
169 109 -5.9 110 -6.1 111 -6.4 112 -6.6 113 -7.0 ... 343 -5.7 344 -5.7 345 -5.7 346 -5.6 347 -5.6
170 rows × 286 columns
My question is how to called out on even number without using a manual way of typing all the even number of the head of columns.
such as:
df2[[0,2,4]]*= 2
I am currently stuck on the ideas on making the conditional on the header columns. I want to call on even numbers only. I hope to find a suitable solutions on the questions. Thank you in advance.

We can select all columns based on whose modulus 2 is 0 (even):
even_cols = df.columns[(df.columns % 2) == 0]
even_cols:
Int64Index([ 0, 2, 4, 6, 8, 10, 12, 14, 16, 18,
...
266, 268, 270, 272, 274, 276, 278, 280, 282, 284],
dtype='int64', length=143)
Then operations can use the newly created index:
df[even_cols] *= 2
df:
0 1 2 3 4 5 6 ... 279 280 281 282 283 284 285
0 198 78 122 16 146 8 124 ... 8 102 61 168 52 148 25
1 18 31 78 59 44 80 116 ... 75 124 51 4 96 38 7
2 152 66 112 0 114 31 172 ... 18 186 19 84 29 36 0
3 80 99 152 25 34 31 106 ... 59 190 33 68 31 66 83
4 192 95 48 95 130 14 8 ... 6 74 79 40 46 198 65
.. ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
165 170 58 110 75 196 11 50 ... 54 46 53 146 62 30 48
166 54 8 148 25 174 40 114 ... 6 180 32 94 44 142 16
167 42 48 48 31 126 60 86 ... 11 128 10 162 67 142 13
168 54 37 70 2 128 38 134 ... 85 166 40 142 57 54 52
169 164 41 146 40 64 44 28 ... 83 90 86 188 23 38 35
If we need every other column instead of columns based on numeric value, we can set the step of slicing to create a list of columns:
every_other_column = df.columns[::2]
df[every_other_column] *= 2
Or simply modify the DataFrame without creating a list of columns:
df.loc[:, ::2] *= 2
Sample DataFrame:
import numpy as np
import pandas as pd
np.random.seed(5)
df = pd.DataFrame(np.random.randint(0, 100, (170, 286)))
print(df)
df:
0 1 2 3 4 5 6 ... 279 280 281 282 283 284 285
0 99 78 61 16 73 8 62 ... 8 51 61 84 52 74 25
1 9 31 39 59 22 80 58 ... 75 62 51 2 96 19 7
2 76 66 56 0 57 31 86 ... 18 93 19 42 29 18 0
3 40 99 76 25 17 31 53 ... 59 95 33 34 31 33 83
4 96 95 24 95 65 14 4 ... 6 37 79 20 46 99 65
.. ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
165 85 58 55 75 98 11 25 ... 54 23 53 73 62 15 48
166 27 8 74 25 87 40 57 ... 6 90 32 47 44 71 16
167 21 48 24 31 63 60 43 ... 11 64 10 81 67 71 13
168 27 37 35 2 64 38 67 ... 85 83 40 71 57 27 52
169 82 41 73 40 32 44 14 ... 83 45 86 94 23 19 35

How to make all the rows data drop the similar data and multiplying float numbers

How to separate all of the columns?
df= df[['hlogUs_dB','hlogDs_dB']]
df
hlogUs_dB hlogDs_dB
0 109:-3.4,110:-3.4,111:-3.4,112:-3.5,113:-3.5,1... 5:-2.5,6:-2.5,7:-2.1,8:-2.0,9:-2.0,10:-2.0,11:...
1 109:-3.5,110:-3.5,111:-3.4,112:-3.4,113:-3.4,1... 5:-2.1,6:-2.0,7:-1.8,8:-1.8,9:-1.8,10:-1.8,11:...
2 109:-3.7,110:-3.7,111:-3.8,112:-3.8,113:-3.8,1... 5:-2.1,6:-2.0,7:-1.8,8:-1.8,9:-1.8,10:-1.8,11:...
3 109:-3.5,110:-3.6,111:-3.6,112:-3.6,113:-3.7,1... 5:-2.5,6:-2.5,7:-2.1,8:-2.0,9:-2.0,10:-2.0,11:...
4 109:-3.7,110:-3.8,111:-3.8,112:-3.8,113:-3.8,1... 5:-2.5,6:-2.5,7:-2.1,8:-2.1,9:-2.0,10:-2.1,11:...
... ... ...
165 109:-5.2,110:-5.3,111:-5.5,112:-5.7,113:-5.9,1... 5:-2.5,6:-2.5,7:-2.1,8:-2.1,9:-2.1,10:-2.2,11:...
166 109:-5.5,110:-5.6,111:-5.8,112:-6.1,113:-6.3,1... 5:-2.8,6:-2.7,7:-2.5,8:-2.5,9:-2.3,10:-2.5,11:...
167 109:-6.0,110:-6.2,111:-6.4,112:-6.7,113:-7.1,1... 5:-2.6,6:-2.5,7:-2.2,8:-2.2,9:-2.2,10:-2.3,11:...
168 109:-5.4,110:-5.5,111:-5.7,112:-5.9,113:-6.2,1... 5:-3.0,6:-3.0,7:-2.6,8:-2.5,9:-2.5,10:-2.5,11:...
169 109:-5.9,110:-6.1,111:-6.4,112:-6.6,113:-7.0,1... 5:-2.7,6:-2.5,7:-2.3,8:-2.2,9:-2.3,10:-2.3,11:...
170 rows × 2 columns
<After that I split using delimiter for only hlogUs_dB/>
df2 =df['hlogUs_dB'].str.split('[,:]',expand = True)
df2 = data.drop(["0"])
df2
The result :
0 1 2 3 4 5 6 7 8 9 ... 276 277 278 279 280 281 282 283 284 285
0 109 -3.4 110 -3.4 111 -3.4 112 -3.5 113 -3.5 ... 343 -4.3 344 -4.3 345 -4.2 346 -4.2 347 -4.2
1 109 -3.5 110 -3.5 111 -3.4 112 -3.4 113 -3.4 ... 343 -4.1 344 -4.2 345 -4.4 346 -4.4 347 -4.2
2 109 -3.7 110 -3.7 111 -3.8 112 -3.8 113 -3.8 ... 343 -4.2 344 -4.3 345 -4.3 346 -4.3 347 -4.3
3 109 -3.5 110 -3.6 111 -3.6 112 -3.6 113 -3.7 ... 343 -4.1 344 -4.1 345 -4.1 346 -4.1 347 -4.1
4 109 -3.7 110 -3.8 111 -3.8 112 -3.8 113 -3.8 ... 343 -4.2 344 -4.2 345 -4.2 346 -4.2 347 -4.3
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
165 109 -5.2 110 -5.3 111 -5.5 112 -5.7 113 -5.9 ... 343 -5.4 344 -5.3 345 -5.2 346 -5.1 347 -5.1
166 109 -5.5 110 -5.6 111 -5.8 112 -6.1 113 -6.3 ... 343 -5.5 344 -5.4 345 -5.3 346 -5.2 347 -5.2
167 109 -6.0 110 -6.2 111 -6.4 112 -6.7 113 -7.1 ... 343 -4.9 344 -4.9 345 -4.9 346 -4.9 347 -4.9
168 109 -5.4 110 -5.5 111 -5.7 112 -5.9 113 -6.2 ... 343 -5.9 344 -5.7 345 -5.7 346 -5.6 347 -5.6
169 109 -5.9 110 -6.1 111 -6.4 112 -6.6 113 -7.0 ... 343 -5.7 344 -5.7 345 -5.7 346 -5.6 347 -5.6
170 rows × 286 columns
After that I want to drop the same number that appear only on even columns. I manage to found the solutions but somehow or rather, it does not suit my preference.
df2.drop(columns=[0,2,4,6,8,9,10,12,14,16,18,20,22,24,26,28,30,32,34,36])
df2
the output:
0 1 2 3 4 5 6 7 8 9 ... 276 277 278 279 280 281 282 283 284 285
0 109 -3.4 110 -3.4 111 -3.4 112 -3.5 113 -3.5 ... 343 -4.3 344 -4.3 345 -4.2 346 -4.2 347 -4.2
1 109 -3.5 110 -3.5 111 -3.4 112 -3.4 113 -3.4 ... 343 -4.1 344 -4.2 345 -4.4 346 -4.4 347 -4.2
2 109 -3.7 110 -3.7 111 -3.8 112 -3.8 113 -3.8 ... 343 -4.2 344 -4.3 345 -4.3 346 -4.3 347 -4.3
3 109 -3.5 110 -3.6 111 -3.6 112 -3.6 113 -3.7 ... 343 -4.1 344 -4.1 345 -4.1 346 -4.1 347 -4.1
4 109 -3.7 110 -3.8 111 -3.8 112 -3.8 113 -3.8 ... 343 -4.2 344 -4.2 345 -4.2 346 -4.2 347 -4.3
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
165 109 -5.2 110 -5.3 111 -5.5 112 -5.7 113 -5.9 ... 343 -5.4 344 -5.3 345 -5.2 346 -5.1 347 -5.1
166 109 -5.5 110 -5.6 111 -5.8 112 -6.1 113 -6.3 ... 343 -5.5 344 -5.4 345 -5.3 346 -5.2 347 -5.2
167 109 -6.0 110 -6.2 111 -6.4 112 -6.7 113 -7.1 ... 343 -4.9 344 -4.9 345 -4.9 346 -4.9 347 -4.9
168 109 -5.4 110 -5.5 111 -5.7 112 -5.9 113 -6.2 ... 343 -5.9 344 -5.7 345 -5.7 346 -5.6 347 -5.6
169 109 -5.9 110 -6.1 111 -6.4 112 -6.6 113 -7.0 ... 343 -5.7 344 -5.7 345 -5.7 346 -5.6 347 -5.6
170 rows × 286 columns
still show the same as before, I just want the odd columns to be multiply with 8 and float 4.3125. Then later the data will replace on the same columns, that was my roughly ideas.
df2*4.3125
the results contain error.
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
~\anaconda3\lib\site-packages\pandas\core\ops\array_ops.py in na_arithmetic_op(left, right, op, str_rep)
148 try:
--> 149 result = expressions.evaluate(op, str_rep, left, right)
150 except TypeError:
~\anaconda3\lib\site-packages\pandas\core\computation\expressions.py in evaluate(op, op_str, a, b, use_numexpr)
207 if use_numexpr:
--> 208 return _evaluate(op, op_str, a, b)
209 return _evaluate_standard(op, op_str, a, b)
~\anaconda3\lib\site-packages\pandas\core\computation\expressions.py in _evaluate_numexpr(op, op_str, a, b)
120 if result is None:
--> 121 result = _evaluate_standard(op, op_str, a, b)
122
~\anaconda3\lib\site-packages\pandas\core\computation\expressions.py in _evaluate_standard(op, op_str, a, b)
69 with np.errstate(all="ignore"):
---> 70 return op(a, b)
71
TypeError: can't multiply sequence by non-int of type 'float'
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
<ipython-input-24-424060d3aad6> in <module>
----> 1 df2*4.3125
~\anaconda3\lib\site-packages\pandas\core\ops\__init__.py in f(self, other, axis, level, fill_value)
717 self = self.fillna(fill_value)
718
--> 719 new_data = dispatch_to_series(self, other, op, str_rep)
720 return self._construct_result(new_data)
721
~\anaconda3\lib\site-packages\pandas\core\ops\__init__.py in dispatch_to_series(left, right, func, str_rep, axis)
376 # Get the appropriate array-op to apply to each block's values.
377 array_op = get_array_op(func, str_rep=str_rep)
--> 378 bm = left._data.apply(array_op, right=right)
379 return type(left)(bm)
380
~\anaconda3\lib\site-packages\pandas\core\internals\managers.py in apply(self, f, filter, **kwargs)
438
439 if callable(f):
--> 440 applied = b.apply(f, **kwargs)
441 else:
442 applied = getattr(b, f)(**kwargs)
~\anaconda3\lib\site-packages\pandas\core\internals\blocks.py in apply(self, func, **kwargs)
388 """
389 with np.errstate(all="ignore"):
--> 390 result = func(self.values, **kwargs)
391
392 if is_extension_array_dtype(result) and result.ndim > 1:
~\anaconda3\lib\site-packages\pandas\core\ops\array_ops.py in arithmetic_op(left, right, op, str_rep)
195 else:
196 with np.errstate(all="ignore"):
--> 197 res_values = na_arithmetic_op(lvalues, rvalues, op, str_rep)
198
199 return res_values
~\anaconda3\lib\site-packages\pandas\core\ops\array_ops.py in na_arithmetic_op(left, right, op, str_rep)
149 result = expressions.evaluate(op, str_rep, left, right)
150 except TypeError:
--> 151 result = masked_arith_op(left, right, op)
152
153 return missing.dispatch_fill_zeros(op, left, right, result)
~\anaconda3\lib\site-packages\pandas\core\ops\array_ops.py in masked_arith_op(x, y, op)
110 if mask.any():
111 with np.errstate(all="ignore"):
--> 112 result[mask] = op(xrav[mask], y)
113
114 result, _ = maybe_upcast_putmask(result, ~mask, np.nan)
TypeError: can't multiply sequence by non-int of type 'float'
I am stuck at the area and have searched on the Stack Overflow, youtube about the basic for multiplying in terms of float but, I think my keywords is not on par with the ideas.

Sorting a string of numbers into a grid

I'm needing to sort a long list of ID numbers into 'grids' of 8 ID numbers down (8 cells/rows), 6 ID numbers across (or 6 columns long etc), sorted from smallest to largest ID number. When one 'grid' is 'full', the numbers which cannot fit in the first grid should go on to form a second one and so on. The last 4 cells of the last row should be blank. (This is a template for a lab procedure).
ie this is the data I have:
column of ID numbers
and this how I want it to be (but like, 6 of these)
example 'grid'

Here's one method.
Sample data
import pandas as pd
import numpy as np
# Sorted list of string IDs
l = np.arange(0, 631, 1).astype('str')
Code
N = 44
# Ensure we can reshape last group
data = np.concatenate((l, np.repeat('', N-len(l)%N)))
# Split array, make a separate `DataFrame` for each grid.
data = [
pd.DataFrame(np.concatenate((x, np.repeat('', 4))).reshape(8,6))
for x in np.array_split(data, np.arange(N, len(l), N))
]
df = pd.concat(data, ignore_index=True) # If want a single df in the end
Output df:
0 1 2 3 4 5
0 0 1 2 3 4 5
1 6 7 8 9 10 11
2 12 13 14 15 16 17
3 18 19 20 21 22 23
4 24 25 26 27 28 29
5 30 31 32 33 34 35
6 36 37 38 39 40 41
7 42 43
8 44 45 46 47 48 49
9 50 51 52 53 54 55
10 56 57 58 59 60 61
11 62 63 64 65 66 67
12 68 69 70 71 72 73
13 74 75 76 77 78 79
14 80 81 82 83 84 85
15 86 87
16 88 89 90 91 92 93
...
110 608 609 610 611 612 613
111 614 615
112 616 617 618 619 620 621
113 622 623 624 625 626 627
114 628 629 630
115
116
117
118
119

func = lambda lst,n: np.pad(lst, (0,n*(1+len(lst)//n) - len(lst)), 'constant')
rows, cols = 8, 6
arr = np.arange(1, 283, 1) ##np.array(df.A)
new_df = pd.DataFrame(func(arr, rows*cols).reshape(-1,cols))
new_df
0 1 2 3 4 5
0 1 2 3 4 5 6
1 7 8 9 10 11 12
2 13 14 15 16 17 18
3 19 20 21 22 23 24
4 25 26 27 28 29 30
5 31 32 33 34 35 36
6 37 38 39 40 41 42
7 43 44 45 46 47 48
8 49 50 51 52 53 54
9 55 56 57 58 59 60
10 61 62 63 64 65 66
11 67 68 69 70 71 72
12 73 74 75 76 77 78
13 79 80 81 82 83 84
14 85 86 87 88 89 90
15 91 92 93 94 95 96
16 97 98 99 100 101 102
17 103 104 105 106 107 108
18 109 110 111 112 113 114
19 115 116 117 118 119 120
20 121 122 123 124 125 126
21 127 128 129 130 131 132
22 133 134 135 136 137 138
23 139 140 141 142 143 144
24 145 146 147 148 149 150
25 151 152 153 154 155 156
26 157 158 159 160 161 162
27 163 164 165 166 167 168
28 169 170 171 172 173 174
29 175 176 177 178 179 180
30 181 182 183 184 185 186
31 187 188 189 190 191 192
32 193 194 195 196 197 198
33 199 200 201 202 203 204
34 205 206 207 208 209 210
35 211 212 213 214 215 216
36 217 218 219 220 221 222
37 223 224 225 226 227 228
38 229 230 231 232 233 234
39 235 236 237 238 239 240
40 241 242 243 244 245 246
41 247 248 249 250 251 252
42 253 254 255 256 257 258
43 259 260 261 262 263 264
44 265 266 267 268 269 270
45 271 272 273 274 275 276
46 277 278 279 280 281 282
47 0 0 0 0 0 0
I think it's better to save this dataframe into an excel worksheet and then remove the last padded zeros manually. Hope this helped

Removing from the end of the file until you see a number or letter with sed/awk

I have a file ending with a number, character or a comma:
file1.txt
1 101 111 BCX A#WWW 123
1 101 111 BCX A#WWW 123;;;;;;
1 298 306 CCC A#QQQ 234-ck
1 298 306 CCC A#QQQ 234-ck;
1 298 306 CCC A#QQQ 234-ck ;;
1 299 308 CCD A#QQQ 234-cJ
1 299 309 DDD A#ZZZ 345;678
1 299 309 DDD A#ZZZ 345;678
The output should be :
1 101 111 BCX A#WWW 123
1 101 111 BCX A#WWW 123
1 298 306 CCC A#QQQ 234-ck
1 298 306 CCC A#QQQ 234-ck
1 298 306 CCC A#QQQ 234-ck
1 299 308 CCD A#QQQ 234-cJ
1 299 309 DDD A#ZZZ 345;678
1 299 309 DDD A#ZZZ 345;678
What I do only removes one comma from the end:
cat file1.txt | sed 's/;$//g'
1 101 111 BCX A#WWW 123
1 101 111 BCX A#WWW 123;;;;;
1 298 306 CCC A#QQQ 234-ck
1 298 306 CCC A#QQQ 234-ck
1 298 306 CCC A#QQQ 234-ck ;
1 299 308 CCD A#QQQ 234-cJ
1 299 309 DDD A#ZZZ 345;678
1 299 309 DDD A#ZZZ 345;678
How can I remove all of the ";"s from the end of the file until I see a letter/number?

Modify your sed, no need of cat
sed 's/;*$//' infile
using awk
awk '{sub(/;*$/,"")}1' infile