Related
I have a single column file.
A
A
A
B
B
B
C
C
D
I want to use this file and want to make a new one as below
command="A" "B" "C" "D"
TYPE=1 1 1 2 2 2 3 3 4,
These A B C D are random alphabets and varies file to file.
I tried to overcome the solution with below shell script
#!/bin/bash
NQ=$(cat RLP.txt | wc -l)
ELEMENT='element='
echo "$ELEMENT" > element.txt
TYPE='types='
echo "$TYPE" > types.txt
for i in `seq 1 1 $NQ`
do
RLP=$(echo "$i" | tail -n 1)
cat RLP.txt | head -n "$RLP" | tail -n 1 > el.$RLP.txt
done
paste element.txt el.*.txt
paste types.txt
The output of paste element.txt el.*.txt is element= A A A B B B C C D
I could not remove the repeated alphabets and put the reaming alphabets in "".
and cold not move forward for with second command to get
TYPE=1 1 1 2 2 2 3 3 4,
which represents that the 1st alphabets repeated three times, 2nd alphabets repeated three times, 3rd alphabets repeated two times and so on..
$ cat tst.awk
!seen[$1]++ {
cmd = cmd sep "\"" $1 "\""
cnt++
}
{
type = type sep cnt
sep = OFS
}
END {
print "command=" cmd
print "TYPE=" type ","
}
$ awk -f tst.awk file
command="A" "B" "C" "D"
TYPE=1 1 1 2 2 2 3 3 4,
Instead of using multiple text processing tools in a pipeline, this can be achieved by one awk command as below
awk '
{
unique[$0]
}
prev !~ $0 {
alpha[NR] = idx++
}
{
prev = $0
alpha[NR] = idx
}
END {
for (i in unique) {
str = str ? (str " " "\"" i "\"") : "\"" i "\""
}
first = "command=" str
str = ""
for (i = 1; i <= NR; i++) {
str = str ? (str " " alpha[i]) : alpha[i]
}
second = "TYPE=" str ","
print(first "\n" second) > "types.txt"
close("types.txt")
}' RLP.txt
The command works as follows
Each unique line in the file is saved as an index in into the array unique
The array alpha keeps track of the unique value counter, i.e. every time a value in the file changes, the counter is incremented at the corresponding line number NR
The END block is all about constructing the output from the array to a string value and writing the result to the new file "types.txt"
Pure bash implementation. Requires at least Bash version 4 for the associative array
#!/bin/bash
outfile="./RLP.txt"
infile="./infile"
declare -A map
while read line; do
(( map["$line"]++ ))
done < "$infile"
command="command="
command+=$(printf "\"%s\" " "${!map[#]}")
type="$(
for i in "${map[#]}"; do
((k++))
for (( j=0; j < i; j++ )); do
printf " %d" "$k"
done
done
),"
echo "$command" >> "$outfile"
echo "TYPE=${type#* }" >> "$outfile"
Using c++14. I have read many posts regarding the problem.
If I run this code below, it jumps over the getline lines.
#include <iostream>
#include "main_menu.h"
void MainMenu::AddTest()
{
std::string courseName = "";
std::string testName = "";
std::string date = "";
std::cout << "Enter course name: " << std::endl;
std::getline(std::cin, courseName);
std::cout << "Enter test name: " << std::endl;
std::getline(std::cin, testName);
std::cout << "Enter test date: " << std::endl;
std::getline(std::cin, date);
Test test(courseName, testName, date);
tests.Add(test);
std::cout << "Test registered : " << std::endl;
tests.Print(test.id);
}
If I add cin ignore after each getline lines (example below how I implement it), it deletes some characters from the input strings and uses wrong variables to store them. Note that I have strings with whitespaces.
std::getline(std::cin, courseName);
std::cin.ignore();
This is what I get:
Enter course name:
History 2
Enter test name:
History 2 exam
Enter test date:
2017.01.02
Test registered :
test id = 2, course name = , test name = istory 2, date = istory 2 exam
I also tried to flush cout, didn't help.
My Print function works like a charm, if I add courses manually from main, I get the expected output, so the problem is definitely the cin / getline.
Test registered :
test id = 1, course name = History 2, test name = History 2 exam , date = 01.02.2017
I use getline as explained here: http://www.cplusplus.com/reference/string/string/getline/?kw=getline
Any help would be much appreciated, thank you.
By using cin.ignore you are messing with the input itself. If you want to get rid of \n character you don't have to! getline will just do that automatically. So just don't use ignore function and the code will be fine.
This works:
#include<iostream>
using namespace std;
int main()
{
string courseName = "";
string testName = "";
string date = "";
cout << "Enter course name: " << std::endl;
getline(std::cin, courseName);
cout << "Enter test name: " << std::endl;
getline(std::cin, testName);
cout << "Enter test date: " << std::endl;
getline(std::cin, date);
cout << courseName << endl;
cout << testName << endl;
cout << date << endl;
return 0;
}
I'm answering an ancient question, but try clearing the input stream before you use all the getline()'s. It is possible that you have some extra returns in the buffer before you ask for input.
cin.clear();
cin.ignore(INT_MAX);
"Using awk to bin values in a list of numbers" provide a solution to average each set of 3 points in a column using awk.
How is it possible to extend it to an indefinite number of columns mantaining the format? For example:
2457135.564106 13.249116 13.140903 0.003615 0.003440
2457135.564604 13.250833 13.139971 0.003619 0.003438
2457135.565067 13.247932 13.135975 0.003614 0.003432
2457135.565576 13.256441 13.146996 0.003628 0.003449
2457135.566039 13.266003 13.159108 0.003644 0.003469
2457135.566514 13.271724 13.163555 0.003654 0.003476
2457135.567011 13.276248 13.166179 0.003661 0.003480
2457135.567474 13.274198 13.165396 0.003658 0.003479
2457135.567983 13.267855 13.156620 0.003647 0.003465
2457135.568446 13.263761 13.152515 0.003640 0.003458
averaging values every 5 lines, should output something like
2457135.564916 13.253240 13.143976 0.003622 0.003444
2457135.567324 13.270918 13.161303 0.003652 0.003472
where the first result is the average of the first 1-5 lines, and the second result is the average of the 6-10 lines.
The accepted answer to Using awk to bin values in a list of numbers is:
awk '{sum+=$1} NR%3==0 {print sum/3; sum=0}' inFile
The obvious extension to average all the columns is:
awk 'BEGIN { N = 3 }
{ for (i = 1; i <= NF; i++) sum[i] += $i }
NR % N == 0 { for (i = 1; i <= NF; i++)
{
printf("%.6f%s", sum[i]/N, (i == NF) ? "\n" : " ")
sum[i] = 0
}
}' inFile
The extra flexibility here is that if you want to group blocks of 5 rows, you simply change one occurrence of 3 into 5. This ignores blocks of up to N-1 rows at the end of the file. If you want to, you can add an END block that prints a suitable average if NR % N != 0.
For the sample input data, the output I got from the script above was:
2457135.564592 13.249294 13.138950 0.003616 0.003437
2457135.566043 13.264723 13.156553 0.003642 0.003465
2457135.567489 13.272767 13.162732 0.003655 0.003475
You can make the code much more complex if you want to analyze what the output formats should be. I've simply used %.6f to ensure 6 decimal places.
If you want N to be a command-line parameter, you can use the -v option to relay the variable setting to awk:
awk -v N="${variable:-3}" \
'{ for (i = 1; i <= NF; i++) sum[i] += $i }
NR % N == 0 { for (i = 1; i <= NF; i++)
{
printf("%.6f%s", sum[i]/N, (i == NF) ? "\n" : " ")
sum[i] = 0
}
}' inFile
When invoked with $variable set to 5, the output generated from the sample data is:
2457135.565078 13.254065 13.144591 0.003624 0.003446
2457135.567486 13.270757 13.160853 0.003652 0.003472
what would be an opposite of split() in awk?
Imagine I have array containig characters/integers.
What I've tried:
color = "#FFFF00";
printf("color original: %s\n", color);
split(color, chars, "");
joined = "";
for (i=1; i <= length(chars); i++) {
joined = joined + chars[i];
}
printf("color joined: %s\n", joined);
however the output is:
color original: #FFFF00
color joined: 0
that is of course incorrect.
UPDATE:
cool, ended up with the following code (inspired by join function present in answers):
color = "#FFFF00";
printf("color original: %s\n", color);
split(color, chars, "");
joined = "";
for (i=1; i <= length(chars); i++) {
joined = joined "" chars[i];
}
printf("color joined: %s\n", joined);
the trick was not to use + sign when joining things back
Here's a solution that doesn't rely on gawk or knowing the length of the array and lets you put a separator (space in this case) string between each array element if you like:
color = "#FFFF00"
printf "color original: %s\n", color
split(color, chars, "")
joined = sep = ""
for (i=1; i in chars; i++) {
joined = joined sep chars[i]
sep = " " # populate sep here with whatever string you want between elements
}
printf "color joined: %s\n", joined
I also cleaned up the incorrect use of printf and the spurious semi-colons.
In the above script split(color, chars, "") relies on having a version of awk that will split a string into an array given a null field separator, which is undefined behavior per POSIX, but that's not what this answer is about - the question is how to join array elements not how to split them.
Here is a way with POSIX Awk:
br = "red,orange,yellow,green,blue"
ch = split(br, de, ",")
print "original: " br
printf "joined: "
for (ec in de) printf ec == ch ? de[ec] "\n" : de[ec] "-"
Output:
original: red,orange,yellow,green,blue
joined: red-orange-yellow-green-blue
What you want (in your loop) is string concatenation.
Similar to previous answers and less elegant, but easy and short:
color = "#FFFF00"
printf "color original: %s\n", color
split(color, chars, "")
for (i=1; i<=length(chars); i++) {
(i==1) ? joined = chars[i] : joined = joined" "chars[i] # Define separator here
}
printf "color joined: %s\n", joined
Knowing that the opposite of split() is join(), a mere Google Search gives me this page, which seems to contain the solution : http://www.gnu.org/software/gawk/manual/html_node/Join-Function.html . It joins all the elements of an array together, and returns the corresponding string.
['f','o','o'] => "foo"
Have fun
Here's my array (gawk script) :
myArray["peter"] = 32
myArray["bob"] = 5
myArray["john"] = 463
myArray["jack"] = 11
After sort, I need the following result :
bob 5
jack 11
peter 32
john 463
When i use "asort", indices are lost. How to sort by array value without losing indices ? (I need ordered indices based on their values)
(I need to obtain this result with awk/gawk only, not shell script, perl, etc)
If my post isn't clear enough, here is an other post explaining the same issue : http://www.experts-exchange.com/Programming/Languages/Scripting/Shell/Q_26626841.html )
Thanks in advance
Update :
Thanks to you both, but i need to sort by values, not indices (i want ordered indices according to their values).
In other terms, i need this result :
bob 5
jack 11
peter 32
john 463
not :
bob 5
jack 11
john 463
peter 32
(I agree, my example is confusing, the chosen values are pretty bad)
From the code of Catcall, I wrote a quick implementation that works, but it's rather ugly (I concatenate keys & values before sort and split during comparison). Here's what it looks like :
function qsort(A, left, right, i, last) {
if (left >= right)
return
swap(A, left, left+int((right-left+1)*rand()))
last = left
for (i = left+1; i <= right; i++)
if (getPart(A[i], "value") < getPart(A[left], "value"))
swap(A, ++last, i)
swap(A, left, last)
qsort(A, left, last-1)
qsort(A, last+1, right)
}
function swap(A, i, j, t) {
t = A[i]; A[i] = A[j]; A[j] = t
}
function getPart(str, part) {
if (part == "key")
return substr(str, 1, index(str, "#")-1)
if (part == "value")
return substr(str, index(str, "#")+1, length(str))+0
return
}
BEGIN { }
{ }
END {
myArray["peter"] = 32
myArray["bob"] = 5
myArray["john"] = 463
myArray["jack"] = 11
for (key in myArray)
sortvalues[j++] = key "#" myArray[key]
qsort(sortvalues, 0, length(myArray));
for (i = 1; i <= length(myArray); i++)
print getPart(sortvalues[i], "key"), getPart(sortvalues[i], "value")
}
Of course I'm interested if you have something more clean...
Thanks for your time
Edit:
Sort by values
Oh! To sort the values, it's a bit of a kludge, but you can create a temporary array using a concatenation of the values and the indices of the original array as indices in the new array. Then you can asorti() the temporary array and split the concatenated values back into indices and values. If you can't follow that convoluted description, the code is much easier to understand. It's also very short.
# right justify the integers into space-padded strings and cat the index
# to create the new index
for (i in myArray) tmpidx[sprintf("%12s", myArray[i]),i] = i
num = asorti(tmpidx)
j = 0
for (i=1; i<=num; i++) {
split(tmpidx[i], tmp, SUBSEP)
indices[++j] = tmp[2] # tmp[2] is the name
}
for (i=1; i<=num; i++) print indices[i], myArray[indices[i]]
Edit 2:
If you have GAWK 4, you can traverse the array by order of values without performing an explicit sort:
#!/usr/bin/awk -f
BEGIN {
myArray["peter"] = 32
myArray["bob"] = 5
myArray["john"] = 463
myArray["jack"] = 11
PROCINFO["sorted_in"] = "#val_num_asc"
for (i in myArray) {
{print i, myArray[i]}}
}
}
There are settings for traversing by index or value, ascending or descending and other options. You can also specify a custom function.
Previous answer:
Sort by indices
If you have an AWK, such as gawk 3.1.2 or greater, which supports asorti():
#!/usr/bin/awk -f
BEGIN {
myArray["peter"] = 32
myArray["bob"] = 5
myArray["john"] = 463
myArray["jack"] = 11
num = asorti(myArray, indices)
for (i=1; i<=num; i++) print indices[i], myArray[indices[i]]
}
If you don't have asorti():
#!/usr/bin/awk -f
BEGIN {
myArray["peter"] = 32
myArray["bob"] = 5
myArray["john"] = 463
myArray["jack"] = 11
for (i in myArray) indices[++j] = i
num = asort(indices)
for (i=1; i<=num; i++) print i, indices[i], myArray[indices[i]]
}
Use the Unix sort command with the pipe, keeps Awk code simple and follow Unix philosophy
Create a input file with values seperated by comma
peter,32
jack,11
john,463
bob,5
Create a sort.awk file with the code
BEGIN { FS=","; }
{
myArray[$1]=$2;
}
END {
for (name in myArray)
printf ("%s,%d\n", name, myArray[name]) | "sort -t, -k2 -n"
}
Run the program, should give you the output
$ awk -f sort.awk data
bob,5
jack,11
peter,32
john,463
PROCINFO["sorted_in"] = "#val_num_desc";
Before iterating an array, use the above statement. But, it works in awk version 4.0.1. It does not work in awk version 3.1.7.
I am not sure in which intermediate version, it got introduced.
And the simple answer...
function sort_by_myArray(i1, v1, i2, v2) {
return myArray[i2] < myArray[i1];
}
BEGIN {
myArray["peter"] = 32;
myArray["bob"] = 5;
myArray["john"] = 463;
myArray["jack"] = 11;
len = length(myArray);
asorti(myArray, k, "sort_by_myArray");
# Print result.
for(n = 1; n <= len; ++n) {
print k[n], myArray[k[n]]
}
}
Use asorti:
#!/usr/bin/env -S gawk -f
{
score[$1] = $0;
array[sprintf("%3s",$2) $1] = $1;
}
END {
asorti(array, b)
for(i in b)
{
name = array[b[i]]
print score[name]
}
}
The following function works in Gawk 3.1.7 and doesn't resort to any of the workarounds described above (no external sort command, no array subscripts, etc.) It's just a basic implementation of the insertion sort algorithm adapted for associative arrays.
You pass it an associative array to sort on the values and an empty array to populate with the corresponding keys.
myArray["peter"] = 32;
myArray["bob"] = 5;
myArray["john"] = 463;
myArray["jack"] = 11;
len = resort( myArray, result );
for( i = 1; i <= len; i++ ) {
key = result[ i ];
print i ": " key " = " myArray[ key ];
}
Here is the implementation:
function resort( data, list,
key, len, i, j, v )
{
len = 0;
for( key in data ) {
list[ ++len ] = key;
}
# insertion sort algorithm adapted for
# one-based associative arrays in gawk
for( i = 2; i <= len; i++ )
{
v = list[ i ];
for( j = i - 1; j >= 1; j-- )
{
if( data[ list[ j ] ] <= data[ v ] )
break;
list[ j + 1 ] = list[ j ];
}
list[ j + 1 ] = v;
}
return len;
}
You can "reverse sort" by simply iterating the resulting array in descending order.