How do I use Perl to parse the output of the sqlplus command? - sql

Question

I have an SQL file which will give me an output like below:
10|1
10|2
10|3
11|2
11|4
.
.
.
I am using this in a Perl script like below:
my #tmp_cycledef = `sqlplus -s $connstr \#DLCycleState.sql`;
after this above statement, since #tmp_cycledef has all the output of the SQL query,
I want to show the output as:
10 1,2,3
11 2,4
How could I do this using Perl?
EDIT:
I am using the following code:
foreach my $row (#tmp_cycledef)
{
chomp $row;
my ($cycle_code,$cycle_month)= split /\s*\|\s*/, $row;
print "$cycle_code, $cycle_month\n";
$hash{$cycle_code}{$cycle_month}=1
}
foreach my $num ( sort keys %hash )
{
my $h = $hash{$num};
print join(',',sort keys %$h),"\n";
}
the fist print statement prints:
2, 1
2, 10
2, 11
2, 12
3, 1
3, 10
3, 11
but the out is always
1,10,11,12
1,10,11,12
1,10,11,12
1,10,11,12
1,10,11,12
1,10,11,12
1,10,11,12

Answer 1

Well, this one is actually how you might do it in perl:
# two must-have pragmas for perl development
use strict;
use warnings;
Perl allows for variables to be created as they are used, $feldman = some_function() means that you now have the variable $feldman in your local namespace. But the bad part about this is that you can type $fldman and take a long time finding out why what you thought was $feldman has no value. Turning on strictures means that your code fails to compile if it encounters an undeclared variable. You declare a variable with a my or our statement (or in older Perl code a use vars statement.
Turning on warnings just warns you when you're not getting values you expect. Often warnings tends to be too touchy, but they are generally a good thing to develop code with.
my %hash; # the base object for the data
Here, I've declared a hash variable that I creatively called %hash. The sigil (pronounced "sijil") "%" tells that it is a map of name-value pairs. This my statement declared the variable and makes it legal for the compiler. The compiler will warn me about any use of %hsh.
The next item is a foreach loop (which can be abbreviated "for"). The loop will process the list of lines in #tmp_cycledef assigning each one in turn to $row. ( my $row).
We chomp the line first, removing the end-of-line character for that platform.
We split the line on the '|' character, creating a list of strings that had been separated by a pipe.
And then we store it in a two-layered hash. Since we want to group them by at least the first number. We could do this by array, and create an array at the location in the hash like so: push #{$hash{$key}}, $val, but I typically want to collapse duplicates (not that there were any duplicates in your sample.)
Here:
foreach my $row ( #tmp_cycledef ) {
chomp $row; # removes the end-of-line character when present.
my ( $key, $val ) = split /\|/, $row;
# One of the best ways to merge lists is a presence-of idea
# with the hash holding whether the value is present
$hash{$key}{$val} = 1;
}
Once we have the data in the structure, we need to iterate both level of hash keys. You wanted to separate the "top level" numbers by lines, but you wanted the second numbers concatenated on the same line. So we print a line for each of the first numbers and join the list of strings stored for each number on the same line, delimited by commas. We also sort the list: { $a <=> $b } just takes to keys and numerically compares them. So you get a numeric order.
# If they were alpha keys our sort routine, we would just likely say sort keys %hash
foreach my $num ( sort { $a <=> $b } keys %hash ) {
my $h = $hash{$num};
print "$num ", join( ',', sort { $a <=> $b } keys %$h ), "\n";
}
As I said in the comments, sort, by default, sorts in character order so you can just say sort keys %hash.
To help you out, you really need to read some of these:
strictures
warnings
perldata
perlfunc -- especially my, foreach, chomp, split, keys, sort and join
And the data structure tutorial

Answer 2

Use a hash of arrays to collect all the values for a single key together, then print them out:
init hash
for each line:
parse into key|value
append value to hash[key]
for each key in hash: # you can sort it, if needed
print out key, list of values

Answer 3

If your input is sorted (as it is in the provided sample), you don't actually need to bother with the hash of arrays/hashes. The code is a bit longer, but doesn't require you to understand references and should run faster for large datasets:
#!/usr/bin/perl
use strict;
use warnings;
my #tmp_cycledef = <DATA>;
my $last_key;
my #values;
for (#tmp_cycledef) {
chomp;
my ($key, $val) = split '\|';
# Seed $last_key with the first key value on the first pass
$last_key = $key unless defined $last_key;
# The key has changed, so it's time to print out the values associated
# with the previous key, then reset everything for the new one
if ($key != $last_key) {
print "$last_key " . join(',', #values) . "\n";
$last_key = $key;
#values = ();
}
# Add the current value to the list of values for this key
push #values, $val;
}
# Don't forget to print out the final key when you're done!
print "$last_key " . join(',', #values) . "\n";
__DATA__
10|1
10|2
10|3
11|2
11|4