RPG IV Files processing issue - sql
I am still learning RPG and AS/400. coming from MS C#.NET.
we are currently moving to iSeries machine and I am trying to pickup the IBM RPG and CL programming as it is more needed for the company.
the iSeries version if I am reading it right is
dspdtaara QSS1MRI: ==> "V7R1M000 2924"
DSPSFWRSC+ F11 : ==>
5770999 *BASE 5050 *CODE QSYS V7R1M0 L00
5770SS1 *BASE 5050 *CODE QSYS V7R1M0 L00
5770SS1 *BASE 2924 *LNG QSYS V7R1M0 L00
...
5770WDS 56 5101 *CODE QDEVTOOLS V7R1M0
5770WDS 60 5050 *CODE QDEVTOOLS V7R1M0
here is my issue.
I have a flat file that is generated on the old AS400 machine and is copied to the new iSeries
file is a delimited data (using ";" ) that I need to reformat and export to FTP for further processing by outside company. each record have either 29 or 28 columns. thus first thing I do is using an SQLRPGLE program add an extra delimiter to the records that are missing one. in the flat file itself
C/EXEC SQL
C+ SELECT
C+ MAX((length(trim(F00001))- length(REPLACE(trim(F00001),';',''))))
C+ , MIN((length(trim(F00001))- length(REPLACE(trim(F00001),';',''))))
C+ INTO :MaxCount, :#DelimCount
C+ FROM QGPL.fIncomming
C/END-EXEC
**/* using the delimiter could fix data by inserting a delimiter in the proper place */
C/EXEC SQL
C+ UPDATE QGPL.fIncomming
C+ SET F00001 = INSERT(F00001,225,0,';')
C+ WHERE ( :MaxCount
C+ - ( length(trim(F00001))
C+ - length(REPLACE(trim(F00001),';','')) )
C+ ) > 0
C/END-EXEC
then I created 2 tables (PF) on iSeries
Table1, which is an SQL table with 29 columns based on the incoming file format, (All columns are named and set to length of the data, but all columns are text (A) type )
and the second table (Table2) which have exactly the same layout as table (Table1) but each columns is of specific data type as needed.
i.e. id is INTEGER and dateof is DATE etc.
no the data is not clean. thus a date field might have spaces in it or money field might have text in it.
I need the best way to move the data to table2 cleaning up and validating it during the transfer.
the SQL way would be the best but the statement is so big I can not truly validate it properly.
can some one suggest (with an example if possible) other ways or better way to write SQL for it
I basically have this but get a lot of errors: and if I try to run it in STRSQL not all the code fits in a screen.
SELECT
TRIM(ORDERID) as ORDERID
...,LINENUM) <> ''
THEN TRIM(LINENUM)
ELSE 9999 END as LINENUM
, TRIM(CUSTNUM) as CUSTNUM
, TRIM(PONUM) as PONUM
, CASE WHEN TRIM(REPLACE(ORDDATE,0,'')) <> ''
THEN CAST(INSERT(INSERT(ORDDATE,5,0,'/'),3,0,'/') as Date)
...
from Table1
AS REQUESTED:
This is the original flat file data [preceded first by column headings c1-c29 for records with 29 effective columns of delimited data, then second by column headings c1-c28 with a *Err-> pointing out where the expected delimiter and 30 blanks are missing for the secondary records because each record should share the same layout, and third by an embedded ruler-line to show the 225th position where the semicolon would be added with the prior SQL code and optionally 30 blanks could be added to effect maintaining the fixed-length layout.]:
---c1----c2---c3----c4---------c5-------c6-------c7-------c8----------c9-----------c10-------c11--------------c12--------------c13----c14-c15---c16---c17---c18------c19---------c20------c21-----c22-------------c23-----------------------c24------------------c25----c26----c27------c28-----c29---
---c1----c2---c3----c4---------c5-------c6-------c7-------c8----------c9-----------c10-------c11--------------c12--------------c13----c14-c15---c16---c17---c18------c19---------c20------c21-----c22-------------c23-----*ERR->---c24--c25----c26-------c27-----c28-- **missing one column!**
....+....1....+....2....+....3....+....4....+....5....+....6....+....7....+....8....+....9....+...10....+...11....+...12....+...13....+...14....+...15....+...16....+...17....+...18....+...19....+...20....+...21....+...22....+...23....+...24....+...25....+...26....+...27....+...28....+...29....+
1596555;001;10010;TEST5 ;01062015; 1213.00; 1219.00; 17.000;NET 30 DAYS ; ;543534241;TOYYO1/5 ; 14OZ ;T; 5.00; .500; 1; 560.00; ; 560.00;01292015;5379602;** 2ND DAY ** ;5XDFSDFFGFGHGHGH16 ; ; ; ; ;
1596555; ;10010; ; ; ; ; ; ; ; ; ; ; ; ; . ; ; ; ; ;01292015;5379602; ; 16.60;FRT; ; ;
1598556;001;10021;TEST ;02112015; 1237.00; 1207.00; 17.000;NET 30 DAYS ; ;567860502;45GGH/4019 ; 10OZ ;R;12.50; .000; 1; 105.42; ; 105.42;02122015;5380313;** 2ND DAY ** ;3HGFH5456GFHFG5G27 ; ; ; ; ;
1598556; ;10021; ; ; ; ; ; ; ; ; ; ; ; ; . ; ; ; ; ;02122015;5380313; ; 13.19;FRT; ; ;
1598557;001;10067;020415 ;02042015; 1283.00; 1238.00; 18.000;NET 30 DAYS ; ;657870142;FTKG061/11 ; 14OZ ;R; ; .330; 1; 358.00; ; 358.00;02092015;5380071;** 2ND DAY ** ;3NHJYJ64646GHJGHJ8 ; ; ; ; ;
1598557; ;10067; ; ; ; ; ; ; ; ; ; ; ; ; . ; ; ; ; ;02092015;5380071; ; 15.09;FRT; ; ;
And this is the desired outcome that is currently generate by MS SQL process [preceded by an embedded ruler-line]:
....+....1....+....2....+....3....+....4....+....5....+....6....+....7....+....8....+....9....+...10....+...11....+...12....+...13....+...14....+...15....+...16....+...17....+...18....+...19....+...20....+...21....+
"1596555",1,"10010","TEST5",01/06/2015, 1213.00, 1219.00, 17.00,"NET 30 DAYS",,"543534241","TOYYO1/5","14OZ","T",5.00,0.50,1, 560.00,, 560.00,01/29/2015,"5379602","** 2ND DAY **","5XDFSDFFGFGHGHGH16",,,,,
"1596555",9999,"10010",,,,,,,,,,,,,,,,,,01/29/2015,"5379602",,,"16.60","FRT",,,
"1598556",1,"10021","TEST",02/11/2015, 1237.00, 1207.00, 17.00,"NET 30 DAYS",,"567860502","45GGH/4019","10OZ","R",12.50,0.00,1, 105.42,, 105.42,02/12/2015,"5380313","** 2ND DAY **","3HGFH5456GFHFG5G27",,,,,
"1598556",9999,"10021",,,,,,,,,,,,,,,,,,02/12/2015,"5380313",,,"13.19","FRT",,,
"1598557",1,"10067","020415",02/04/2015, 1283.00, 1238.00, 18.00,"NET 30 DAYS",,"657870142","FTKG061/11","14OZ","R",,0.33,1, 358.00,, 358.00,02/09/2015,"5380071","** 2ND DAY **","3NHJYJ64646GHJGHJ8",,,,,
"1598557",9999,"10067",,,,,,,,,,,,,,,,,,02/09/2015,"5380071",,,"15.09","FRT",,,
I would read the FIncomming directly into the RPG program, and not worry about that missing delimiter. I am guessing it is at the end of the record anyway (you didn't really tell us that). Parse out the fileds as you would in C/C++, the standard C library is available to RPGIV. For each record validate the fields, and if all are valid, write directly to Table 2 mentioned in your question. Records with errors would be written to Table 1 for remediation.
Now you mention that the data needs to be FTP'd to an outside company, that is another issue altogether. Scott Klement has an API that you can use to FTP files directly from within RPG. His web site is http://www.scottklement.com.
And one more thing, given that you are new to RPG, you might want to learn free format rather than the fixed format variant as that is the modern way to code RPG. That SQL statement in free format would look like:
exec sql
UPDATE qgpl.fIncomming
SET F00001 = INSERT(F00001,225,0,';')
WHERE :MaxCount - length(trim(F00001) - length(REPLACE(trim(F00001),';',''))) > 0;
There are many ways to generate a comma seperated file from a database file. The easiest is to use CL command CPYTOIMPF, prompt it, and you can see the options. That command can also drop the file directly into the IFS with translation to ASCII for transfer to MS-SQL.
Edit March 3, 2017
Wow, looks like some folks are using some really old stuff. So if you are on a release of i5/OS, or OS/400 prior to V5R4, the code looks like this
c/exec sql
c+ UPDATE qgpl.fIncomming
c+ SET F00001 = INSERT(F00001,225,0,';')
c+ WHERE :MaxCount - length(trim(F00001) - length(REPLACE(trim(F00001),';',''))) > 0;
c/end-exec
This is fixed format, notice the c spec. If you are coding in free format on this old release, and need the SQL precompiler to understand what you are doing, you will need to add /end-free and /free compiler directives around the fixed form portions of your code. #Dam included a link in the comments.
I am not authorized [yet] to "comment", so I am using the "answer" feature instead, to solicit more information :-(
Regarding the statement in the OP that "I created 2 tables (PF) on iSeries", the SQL DDL for those two TABLE objects was not included. Having that information would better allow reviewers of the scenario to propose some ideas, using the already-defined TABLEs, rather than the reviewer proposing something else that might possibly be unacceptable within the constraints mandated\established for the scenario; we can only presume that those two tables are a requirement, given the SELECT and UPDATE statements apparently reference those [yet to be described] tables.?
FWiW, when presenting an issue\scenario, offering all of the relevant details [as much as reasonably possible anyhow] is best, to elicit worthwhile feedback; otherwise a topic ends up receiving many comments asking for additional clarification of the scenario. That appears to have transpired already with this topic.
Related
Writing SAS dates to SQL Server databse
How to write SAS dates to Microsoft SQL Server 2016 Date data type in database?
I got SAS data with a sas date DataEndDay and I want to write that into a database. The following bit is in use (buffer is just to speed up the testing-failing) :
libname valu oledb provider=sqloledb schema="dbo" INSERTBUFF=100
properties=("User ID"="&username." Password="&pw."
"data source" = &database.
"initial catalog"=&catalog.);
proc sql noprint;
insert into valu.Data_upload_from_me
( <some_columns...>,
<more-columns...>
,DataEndDay
)
select
<some_columns_source...>,
<more-columns_source...>
,DataEndDay
from work.SAS_data_to_publish
;quit;
Of course because SAS dates are numbers, direct writing is going to fail. What works is if I hard-code this as:
select
<some_columns_source...>,
<more-columns_source...>
,'2018-12-12'
from work.SAS_data_to_publish
;quit;
But If I convert the SAS date to string in SAS datasteps:
data SAS_data_to_publish ;
set SAS_data_to_publish ;
dataEndday0 = put(DataEndDay, yymmddd10.);
DataEndDay1 = quote(dataEndday0, "'") ;
run;
and try to write either of these, I get conversion error:
ERROR: ICommand::Execute failed. : Conversion failed when converting date and/or time from character string.
When I select the string it looks pretty ok:
proc sql; select DataEndDay1 from SAS_data_to_publish; quit;
'2018-12-12'
previously I've managed to write dateTimes with similar trick, which works:
proc format;
picture sjm
. = .
other='%Y-%0m-%0d %0H:%0M:%0S:000' (datatype=datetime)
;run;
data to_be_written;
set save.raw_data_to_be_written;
DataEndDay0 = put(dhms(DataEndDay,0,0,0), sjm. -L);
run;
Anyone ran into similar issues? How could I write the dates?
I could ask them to change the column to dateTime, maybe....
Thank you in advance.
Edit:
I managed to develop a work-around, which works but is ugly and -frankly- I don't like it. It so happens that my date is same for all rows, so I can assing it to macro variable and then use it in database writing.
data _NULL_;
set SAS_data_to_publish;
call symput('foobar', quote( put (DataEndDay , yymmddd10. -L), "'") ) ;
run;
....
select
<some_columns_source...>,
<more-columns_source...>
,&foobar.
from work.SAS_data_to_publish
;quit;
Of course this would fail immediately should DataEndDay vary, but maybe demonstrates that something is off in Proc SQLs select clause....
Edit Edit Pasted the question to SAS forums
I finally managed to crack the issue. The issue was for the missing values. As I am passing the values as strings into the database the parser interpreted missing values as real dots instead of empty strings. The following works: data upload; set upload; CreatedReportdate2 = PUT(CreatedReportdate , yymmddn8.); run; libname uplad_db odbc noprompt = "DRIVER=SQL Server; server=&server.; Uid=&user.;Pwd=&pw.; DATABASE=&db.;" INSERTBUFF=32767; proc sql; insert into uplad_db.upload_table (.... ) select case when CreatedReportdate2 ='.' then '' else CreatedReportdate2 end, ... from upload; quit;
SAS does not really properly support the SQL server DATE data type. I imagine this is due to the fact that it's newer, but for whatever reason you have to pass the data as strings. For missing values, it's important to have a blank string, not a . character. The easiest workaround here is to set: options missing=' '; That will allow you to insert data properly. You can then return it to . if you wish. In a production application that might be used by others, I'd consider storing aside the option value temporarily then resetting to that, in order to do no harm.
Normally I just use PROC APPEND to insert observations into a remote database. proc append base=valu.Data_upload_from_me force data=work.SAS_data_to_publish ; run; Make sure your date variable in your SAS dataset use the same data type as the corresponding variable names in your target database table. So if your MS SQL database uses TIMESTAMP fields for date values then make sure your SAS dataset uses DATETIME values. If you want to use constants then make sure to use SAS syntax in your SAS code and MS SQL syntax in any pass through code. data test; date = '01JAN2017'd ; datetime = '01JAN2017:00:00'dt ; run; proc sql ; connect to oledb .... ; execute ( ... date = '2017-01-01' .... datetime='2017-01-01 00:00' ...) by oledb; quit;
Foxpro String Variable combination in Forloop
As in title, there is an error in my first code in FOR loop: Command contains unrecognized phrase. I am thinking if the method string+variable is wrong. ALTER TABLE table1 ADD COLUMN prod_n c(10) ALTER TABLE table1 ADD COLUMN prm1 n(19,2) ALTER TABLE table1 ADD COLUMN rbon1 n(19,2) ALTER TABLE table1 ADD COLUMN total1 n(19,2) There are prm2... until total5, in which the numbers represent the month. FOR i=1 TO 5 REPLACE ALL prm+i WITH amount FOR LEFT(ALLTRIM(a),1)="P" AND batch_mth = i REPLACE ALL rbon+i WITH amount FOR LEFT(ALLTRIM(a),1)="R" AND batch_mth = i REPLACE ALL total+i WITH sum((prm+i)+(rbon+i)) FOR batch_mth = i NEXT ENDFOR Thanks for the help.
There are a number of things wrong with the code you posted above. Cetin has mentioned a number of them, so I apologize if I duplicate some of them. PROBLEM 1 - in your ALTER TABLE commands I do not see where you create fields prm2, prm3, prm4, prm5, rbon2, rbon3, etc. And yet your FOR LOOP would be trying to write to those fields as the FOR LOOP expression i increases from 1 to 5 - if the other parts of your code was correct. PROBLEM 2 - You cannot concatenate a String to an Integer so as to create a Field Name like you attempt to do with prm+i or rbon+1 Cetin's code suggestions would work (again as long as you had the #2, #3, etc. fields defined). However in Foxpro and Visual Foxpro you can generally do a task in a variety of ways. Personally, for readability I'd approach your FOR LOOP like this: FOR i=1 TO 5 * --- Keep in mind that unless fields #2, #3, #4, & #5 are defined --- * --- The following will Fail --- cFld1 = "prm" + STR(i,1) && define the 1st field cFld2 = "rbon" + STR(i,1) && define the 2nd field cFld3 = "total" + STR(i,1) && define the 3rd field REPLACE ALL &cFld1 WITH amount ; FOR LEFT(ALLTRIM(a),1)="P" AND batch_mth = i REPLACE ALL &cFld2 WITH amount ; FOR LEFT(ALLTRIM(a),1)="R" AND batch_mth = i REPLACE ALL &cFld3 WITH sum((prm+i)+(rbon+i)) ; FOR batch_mth = i NEXT NOTE - it might be good if you would learn to use VFP's Debug tools so that you can examine your code execution line-by-line in the VFP Development mode. And you can also use it to examine the variable values. Breakpoints are good, but you have to already have the TRACE WINDOW open for the Break to work. SET STEP ON is the Debug command that I generally use so that program execution will stop and AUTOMATICALLY open the TRACE WINDOW for looking at code execution and/or variable values.
Do you mean you have fields named prm1, prm2, prm3 ... prm12 that represent the months and you want to update them in a loop? If so, you need to understand that a "fieldName" is a "name" and thus you need to use a "name expression" to use it as a variable. That is:
prm+i
would NOT work but:
( 'pro'+ ltrim(str(m.i)) )
would.
For example here is your code revised:
For i=1 To 5
Replace All ('prm'+Ltrim(Str(m.i))) With amount For Left(Alltrim(a),1)="P" And batch_mth = m.i
Replace All ('rbon'+Ltrim(Str(m.i))) With amount For Left(Alltrim(a),1)="R" And batch_mth = m.i
* ????????? REPLACE ALL ('total'+Ltrim(Str(m.i))) WITH sum((prm+i)+(rbon+i)) FOR batch_mth = i
Endfor
However, I must admit, your code doesn't make sense to me. Maybe it would be better if you explained what you are trying to do and give some simple data with the result you expect (as code - you can use FAQ 50 on foxite to create code for data).
Perl: for (min .. max) uses random order, but I want it in order 0,1,2,
As I am a total beginner to perl, oracle sql and everything else. I have to write a script to parse an excel file and write the values into an oracle sql database.
Everything is good so far. But it writes the rows in random order into the database.
for ($row_min .. $row_max) {...insert into db code $sheetValues[$_][col0] etc...}
I don't get it why the rows are inserted in a random order?
And obviously how can I get them in order? excel_row 0 => db_row 0 and so on...
The values in the array are in order! The number of rows is dynamic.
Thanks for your help, I hope you got all the information you need.
Edit:
&parseWrite;
sub parseWrite {
my #sheetValues;
my $worksheet = $workbook->worksheet(0);
my ($row_min, $row_max) = $worksheet->row_range();
print "| Zeile $row_min bis $row_max |";
my ($col_min, $col_max) = $worksheet->col_range();
print " Spalte $col_min bis $col_max |<br>";
for my $row ($row_min .. $row_max) {
for my $col ($col_min .. $col_max) {
my $cell = $worksheet->get_cell ($row,$col);
next unless $cell;
$sheetValues[$row][$col] = $cell->value();
print $sheetValues[$row][$col] .
"(".$row."," .$col.")"."<br>";
}
}
for ($row_min .. $row_max) {
my $sql="INSERT INTO t_excel (
a,b,c,d,e
) VALUES (
'$sheetValues[$_][0 ]',
'$sheetValues[$_][1 ]',
'$sheetValues[$_][2 ]',
'$sheetValues[$_][3 ]',
'$sheetValues[$_][4 ]',
'$sheetValues[$_][5 ]'
)";
$dbh->do($sql);
}
}
With in order I mean that my PL/SQL Developer 8.0.3 (given by my company)
shows with SELECT * FROM t_excel;
pic
But shell = (2,0), maggie = (0,0) and 13 = (1,0) in the array.
The rows are being inserted in the order you expect. I believe the mistaken assumption here is that SELECT will return rows in the same order they're inserted. This is not true. While implementations may make it seem like it does, SELECT has no default order. You're thinking a table is basically like a big list, INSERT is adding to the end of it, and SELECT just iterates through it. That's not a bad approximation, but it can lead you to make bad assumptions. The reality is that you can say little for sure about how a table is stored. SQL is a declarative language which means you tell the computer what you want. This is different from a most other language types where you tell the computer what to do. SELECT * FROM sometable says "give me all the rows and all their columns in the table". Since you didn't give an order, the database can return them in whatever order it likes. Contrast with the procedural meaning which would be "iterate through all the rows in the table" as if the table was some sort of list. Most languages encourage you to take advantage of how data is stored. Declarative languages prevent you from knowing how data is stored. If you want your SELECT to be ordered, you have to give it an ORDER BY.
SAS table string length (limit)
I am creating a SAS table in which one of the fields has to holds a huge string.
Following is my table (TABLE name = MKPLOGS4):
OBS RNID DESCTEXT
--------- -----------
1 123 This is some text which is part of the record. I want this to appear
2 123 concatenated kinda like concat_group() from MYSQL but for some reason
3 123 SAS does not have such functionality. Now I am having trouble with
4 123 String concatenation.
5 124 Hi there old friend of mine, hope you are enjoying the weather
6 124 Are you sure this is not your jacket, okay then. Will give charity.
. . .
. . .
. . .
and I have to get a Table similar to this (table name = MKPLOGSA):
OBS RNID DESCTEXT
--------- -----------
1 123 This is some text which is part of the record. I want this to appear concatenated kinda like concat_group() from MYSQL but for some reason SAS does not have such functionality. Now I am having trouble with String concatenation.
2 124 Hi, there old friend of mine, hope you are enjoying the weather Are you sure this is not your jacket, okay then. Will give charity.
. . .
. . .
. . .
So, after trying unsuccessfully with SQL, I came up with the following SAS code (please note I am very new at SAS):
DATA MKPLOGSA (DROP = DTEMP DTEXT);
SET MKPLOGS4;
BY RNID;
RETAIN DTEXT;
IF FIRST.RNID THEN
DO;
DTEXT = DESCTEXT;
DELETE;
END;
ELSE IF LAST.RNID THEN
DO;
DTEMP = CATX(' ',DTEXT,DESCTEXT);
DESCTEXT = DTEMP;
END;
ELSE
DO;
DTEMP = CATX(' ',DTEXT,DESCTEXT);
DTEXT = DTEMP;
DELETE;
END;
The SAS log is producing this warning message:
WARNING: IN A CALL TO THE CATX FUNCTION, THE BUFFER ALLOCATED
FOR THE RESULT WAS NOT LONG ENOUGH TO CONTAIN THE CONCATENATION
OF ALL THE ARGUMENTS. THE CORRECT RESULT WOULD CONTAIN 229 CHARACTERS,
BUT THE ACTUAL RESULT MAY EITHER BE TRUNCATED TO 200 CHARACTER(S) OR
BE COMPLETELY BLANK, DEPENDING ON THE CALLING ENVIRONMENT. THE
FOLLOWING NOTE INDICATES THE LEFT-MOST ARGUMENT THAT CAUSED TRUNCATION.
Followed by the message (for the SAS data step I posted here):
NOTE: ARGUMENT 3 TO FUNCTION CATX AT LINE 100 COLUMN 15 IS INVALID.
Please note that in my sample data table (MKPLOGS4), each line of string for the field DESCTEXT can be upto 116 characters and there is no limit as to how many lines of description text/recordID.
The output I am getting has only the last line of description:
OBS RNID DESCTEXT
---- --------
1 123 String concatenation.
2 124 Are you sure this is not your jacket, okay then. Will give charity.
. . .
. . .
. . .
I have the following questions:
. is there something wrong with my code?
. is there a limit to SAS string concatenation? Can I override this? If yes, please provide code.
If you have a suggestion, I would really appreciate if you can post your version of code. This is not school work/homework.
Since SAS stores character data as blank-padded fixed length strings, it is usually not a good idea to store a large amount of text in the dataset. However, if you must, then you can create a character type variable with a length of up to 32767 characters. If you don't mind doing some extra I/O, here is an easy way.
/* test data -- same id repeated over multiple observations i.e., in a "long-format" */
data one;
input rnid desctext & :$200.;
cards;
123 This is some text which is part of the record. I want this to appear
123 concatenated kinda like concat_group() from MYSQL but for some reason
123 SAS does not have such functionality. Now I am having trouble with
123 String concatenation.
124 Hi there old friend of mine, hope you are enjoying the weather
124 Are you sure this is not your jacket, okay then. Will give charity.
;
run;
/* re-shape from the long to the wide-format. assumes that data are sorted by rnid. */
proc transpose data=one out=two;
by rnid;
var desctext;
run;
/* concatenate col1, col2, ... vars into single desctext */
data two;
length rnid 8 desctext $1000;
keep rnid desctext;
set two;
desctext = catx(' ', of col:);
run;
The documentation for the catx function specifies that it will (by default) only return 200 characters unless you have already specified a length for the string you are storing the result to.
All you need to do is add either a length or an attrib statement somewhere in your datastep.
Here is how I would have coded it (untested):
data mkplogsa (rename=dtext=desctext);
length dtext $32767 ;
set mkplogs4;
by rnid;
retain dtext;
if first.rnid then do;
dtext = "";
end;
dtext = catx(' ',dtext,desctext);
if last.rnid then do;
output;
end;
keep dtext;
run;
Note that 32767 is the largest string size for a character value in a SAS dataset. If your string is larger than that you're out of luck.
Cheers
Rob
Thanks guys, I was able to solve this problem by using PROC TRANSPOSE and then using concatenation. Here is the code:
/*
THIS TRANSPOSE STEP TAKES THE MKPLOGS4 TABLE AND
CREATES A NEW TEMPORARY TABLE CALLED MKPLOGSA. SINCE
THE DESCRIPTION TEXT IS STORED IN MULTIPLE LINES (OBSERVATIONS)
IN THE ITEXT FILE, IN ORDER TO COMBINE THEM TO A SINGLE ROW,
WE USE TRANSPOSE. HOWEVER, AFTER THIS STEP, THE DESCRIPTION TEXT
SPREAD OVER MULTIPLE LINES ALTHOUGH ON SAME ROW (OBSERVATION)
ARE STILL SEPARATED INTO MULTIPLE COLUMNS (ON THE SAME ROW)
ALL PREFIXED IN THIS CASE BY 'DESCTEXT'. WE DROP THE AUTO-CREATED
COLUMN _NAME_
*/
PROC TRANSPOSE DATA = MKPLOGS4 OUT = MKPLOGSA (DROP = _NAME_)
PREFIX = DESCTEXT;
VAR DESCTEXT;
BY PLOG;
RUN;
/*
THIS DATA STEP CREATES A NEW TABLE CALLED MKPLOGSB WHICH
TAKES ALL THE SEPARATED DESCRIPTION TEXT COLUMNS AND
CONCATENATES THEM INTO A SINGLE COLUMN - LONG_DESCRIPTION.
*/
DATA MKPLOGSB (DROP = DESCTEXT:);
SET MKPLOGSA;
/* CONCATENATED DESC. TEXT SET TO MAX. 27000 CHARS. */
LENGTH LONG_DESCRIPTION $27000;
LONG_DESCRIPTION = CATX(' ',OF DESCTEXT:);
RUN;
How to format Oracle SQL text-only select output
I am using Oracle SQL (in SQLDeveloper, so I don't have access to SQLPLUS commands such as COLUMN) to execute a query that looks something like this: select assigner_staff_id as staff_id, active_flag, assign_date, complete_date, mod_date from work where assigner_staff_id = '2096'; The results it give me look something like this: STAFF_ID ACTIVE_FLAG ASSIGN_DATE COMPLETE_DATE MOD_DATE ---------------------- ----------- ------------------------- ------------------------- ------------------------- 2096 F 25-SEP-08 27-SEP-08 27-SEP-08 02.27.30.642959000 PM 2096 F 25-SEP-08 25-SEP-08 25-SEP-08 01.41.02.517321000 AM 2 rows selected This can very easily produce a very wide and unwieldy textual report when I'm trying to paste the results as a nicely formatted quick-n-dirty text block into an e-mail or problem report, etc. What's the best way to get rid of all tha extra white space in the output columns when I'm using just plain-vanilla Oracle SQL? So far all my web searches haven't turned up much, as all the web search results are showing me how to do it using formatting commands like COLUMN in SQLPLUS (which I don't have).
In your statement, you can specify the type of output you're looking for: select /*csv*/ col1, col2 from table; select /*Delimited*/ col1, col2 from table; there are other formats available such as xml, html, text, loader, etc. You can change the formatting of these particular options under tools > preferences > Database > Utilities > Export Be sure to choose Run Script rather than Run Statement. * this is for Oracle SQL Developer v3.2
What are you using to get the results? The output you pasted looks like it's coming from SQL*PLUS. It may be that whatever tool you are using to generate the results has some method of modifying the output. By default Oracle outputs columns based upon the width of the title or the width of the column data which ever is wider. If you want make columns smaller you will need to either rename them or convert them to text and use substr() to make the defaults smaller. select substr(assigner_staff_id, 8) as staff_id, active_flag as Flag, to_char(assign_date, 'DD/MM/YY'), to_char(complete_date, 'DD/MM/YY'), mod_date from work where assigner_staff_id = '2096';
What you can do with sql is limited by your tool. SQL Plus has commands to format the columns but they are not real easy to use. One quick approach is to paste the output into excel and format it there or just attach the spreadsheet. Some tools will save the output directly as a spreadsheet.
Nice question. I really had to think about it. One thing you could do is change your SQL so that it only returns the narrowest usable columns. e.g. (I'm not very hot on oracle syntax, but something similar should work): select substring( convert(varchar(4), assigner_staff_id), 1, 4 ) as id, active_flag as act, -- use shorter column name -- etc. from work where assigner_staff_id = '2096'; Does that make sense? If you were doing this on unix/linux, I would suggest running it from the command line and piping it through an awk script. If I've miss-understood, then please update your question and I'll have another go :)
If you don't have alot of rows returned I'll often use Tom Kytes print_table function.
SQL> set serveroutput on
SQL> execute print_table('select * from all_objects where rownum < 3');
OWNER : SYS
OBJECT_NAME : /1005bd30_LnkdConstant
SUBOBJECT_NAME :
OBJECT_ID : 27574
DATA_OBJECT_ID :
OBJECT_TYPE : JAVA CLASS
CREATED : 22-may-2008 11:41:13
LAST_DDL_TIME : 22-may-2008 11:41:13
TIMESTAMP : 2008-05-22:11:41:13
STATUS : VALID
TEMPORARY : N
GENERATED : N
SECONDARY : N
-----------------
OWNER : SYS
OBJECT_NAME : /10076b23_OraCustomDatumClosur
SUBOBJECT_NAME :
OBJECT_ID : 22390
DATA_OBJECT_ID :
OBJECT_TYPE : JAVA CLASS
CREATED : 22-may-2008 11:38:34
LAST_DDL_TIME : 22-may-2008 11:38:34
TIMESTAMP : 2008-05-22:11:38:34
STATUS : VALID
TEMPORARY : N
GENERATED : N
SECONDARY : N
-----------------
PL/SQL procedure successfully completed.
SQL>
If its lots of rows, i'll just do the query in SQL Developer and save as xls, businessy types love excel for some reason.
Why not just use the "cast" function? select (cast(assigner_staff_id as VARCHAR2(4)) AS STAFF_ID, (cast(active_flag as VARCHAR2(1))) AS A, (cast(assign_date as VARCHAR2(10))) AS ASSIGN_DATE, (cast(COMPLETE_date as VARCHAR2(10))) AS COMPLETE_DATE, (cast(mod_date as VARCHAR2(10))) AS MOD_DATE from work where assigner_staff_id = '2096';