Dynamic CTE's as part of a SProc in DB2/400 - sql

Question

I'm trying to write a SProc in db2/400 in a V7R2 environment which creates a CTE based on the parameters passed. I then need to perform a recursive query on the CTE.
I'm running issues into creating and executing the dynamic CTE.
According to http://www-01.ibm.com/support/knowledgecenter/ssw_ibm_i_72/db2/rbafzpreph2.htm
the prepare statement does not work with the WITH or SELECT statements directly.
I tried to wrap both the dynamic CTE and dynamic SELECT in a VALUES INTO and manage to successfully prepare the statement. The issue then comes when I try to execute the statement.
I get an error code of SQL0518 which is defined here (CTRL+F for 'SQL0518' to jump down): http://publib.boulder.ibm.com/iseries/v5r2/ic2924/index.htm?info/rzala/rzalamsg.html (NOTE*: This link is for V5R2 but the error code and text portion of my error is exact to the error listed here with the same code. So I'm sure the error code remained the same between versions)
From the 3 recovery suggestions listed, the second seems unlikely to be the case since my execute is the very next line after my prepare. Suggestion 3 also seems unlikely because there is no use of commit or rollback. So I am inclined to believe suggestion 1 applies to my particular case. However, I do not understand how to take the suggested steps.
If &1 identifies a prepared SELECT or DECLARE PROCEDURE statement, a different prepared statement must be named in the EXECUTE statement.
Am I supposed to have two prepare statements for the same execute? Syntactically how would this look?
Here is the code for my SProc for reference:
CREATE OR REPLACE PROCEDURE DLLIB/G_DPIVOT# (
IN TABLE_NAME CHAR(12) CCSID 37 DEFAULT '' ,
IN PIVOT CHAR(12) CCSID 37 DEFAULT '' ,
IN PIVOTFLD CHAR(12) CCSID 37 DEFAULT '' ,
IN "VALUE" DECIMAL(10, 0) DEFAULT 0 ,
INOUT LIST CHAR(5000) CCSID 37 )
LANGUAGE SQL
SPECIFIC DLLIB/G_DPIVOT#
NOT DETERMINISTIC
READS SQL DATA
CALLED ON NULL INPUT
CONCURRENT ACCESS RESOLUTION DEFAULT
SET OPTION ALWBLK = *ALLREAD ,
ALWCPYDTA = *OPTIMIZE ,
COMMIT = *NONE ,
DECRESULT = (31, 31, 00) ,
DFTRDBCOL = *NONE ,
DYNDFTCOL = *NO ,
DYNUSRPRF = *USER ,
SRTSEQ = *HEX
BEGIN
DECLARE STMT1 VARCHAR ( 1000 ) ;
SET STMT1 = 'WITH DETAILS ( ' || TRIM ( PIVOT ) || ' , ' || TRIM ( PIVOTFLD ) || ' , CURR , PREV ) AS ( ' ||
'SELECT ' || TRIM ( PIVOT ) || ' ,' || TRIM ( PIVOTFLD ) || ',' ||
' ROW_NUMBER ( ) OVER ( PARTITION BY ' || TRIM ( PIVOT ) || ' ORDER BY ' || TRIM ( PIVOTFLD ) || ' ) AS CURR ,' ||
' ROW_NUMBER ( ) OVER ( PARTITION BY ' || TRIM ( PIVOT ) || ' ORDER BY ' || TRIM ( PIVOTFLD ) || ' ) - 1 AS PREV' ||
' FROM ' || TRIM ( TABLE_NAME ) ||
' WHERE ' || TRIM ( PIVOT ) || ' = ' || TRIM ( VALUE ) ||
' GROUP BY ' || TRIM ( PIVOT ) || ' , ' || TRIM ( PIVOTFLD ) || ' )' ||
' VALUES( SELECT MAX ( TRIM ( L '','' FROM CAST ( SYS_CONNECT_BY_PATH ( ' || TRIM ( PIVOTFLD ) || ' , '','' ) AS CHAR ( 5000 ) ) ) )' ||
' FROM DETAILS ' ||
' START WITH CURR = 1 ' ||
' CONNECT BY NOCYCLE ' || TRIM ( PIVOT ) || ' = PRIOR ' || TRIM ( PIVOT ) || ' AND PREV = PRIOR CURR) INTO ?' ;
--SET LIST = STMT1; -- If I execute the value of LIST in interactive SQL everything is as expected (minus the VALUES INTO ofcourse)
PREPARE S1 FROM STMT1 ;
EXECUTE S1 USING LIST; -- If I comment this I don't get an error, but I also don't get a return value in LIST)
END ;
Any assistance is appreciated.
EDIT 1: I am trying to create a SProc (which I will use to create a UDF) which has 5 parameters. I am trying to pivot a single field spanning across multiple records so the values are returned as a comma delimited string. I want to make this dynamic though so I can re-use it for many situations. An example call would be: CALL DLLIB.G_DPIVOT#(TABLE, PIVOT, PIVOTFLD, VALUE, LIST); Where TABLE is the name of the table I want to pivot, PIVOT is the commonality between records (FK), PIVOTFLD is the field I want to condense to a single string, VALUE is the FK value I want to use to pivot on, and LIST is the OUT parameter which would contain the resulting string. You can read more about a non-dynamic implementation here: http://www.mcpressonline.com/sql/techtip-combining-multiple-row-values-into-a-single-row-with-sql-in-db2-for-i.html
The use is for when I have a header table which has a one-to-many relationship with another table. I'll then be able to summarize all the values of a particular field in the "many" table based on the PK/FK relationship.
EDIT 2:
Here is a recent attempt which I think I manage to successfully create the CTE using EXECUTE IMMEDIATE and am now trying to just perform a simple select on it. I'm trying to make use of DB2 cursors but get at error at the "C2" on the line DECLARE C2 CURSOR FOR S2;. I don't have much experience with DB2 cursors but believe I am using them in the correct way.
DECLARE STMT1 VARCHAR ( 1000 ) ;
DECLARE STMT2 VARCHAR ( 1000 ) ;
SET STMT1 = 'WITH DETAILS ( ' || TRIM ( PIVOT ) || ' , ' || TRIM ( PIVOTFLD ) || ' , CURR , PREV ) AS ( ' ||
'SELECT ' || TRIM ( PIVOT ) || ' ,' || TRIM ( PIVOTFLD ) || ',' ||
' ROW_NUMBER ( ) OVER ( PARTITION BY ' || TRIM ( PIVOT ) || ' ORDER BY ' || TRIM ( PIVOTFLD ) || ' ) AS CURR ,' ||
' ROW_NUMBER ( ) OVER ( PARTITION BY ' || TRIM ( PIVOT ) || ' ORDER BY ' || TRIM ( PIVOTFLD ) || ' ) - 1 AS PREV' ||
' FROM ' || TRIM ( TABLE_NAME ) ||
' WHERE ' || TRIM ( PIVOT ) || ' = ' || TRIM ( VALUE ) ||
' GROUP BY ' || TRIM ( PIVOT ) || ' , ' || TRIM ( PIVOTFLD ) || ' )';
EXECUTE IMMEDIATE STMT1;
SET STMT2 = "SELECT * FROM DETAILS";
PREPARE S2 FROM STMT2;
DECLARE C2 CURSOR FOR S2;
OPEN C2;
FETCH C2 INTO LIST;
CLOSE C2;
Does anyone see anything wrong with these changes?
Here is the exact error message (excluding suggestion text):
SQL State: 42601
Vendor Code: -104
Message: [SQL0104] Token C2 was not valid. Valid tokens: GLOBAL.
EDIT 3 (Final SProc):
#user2338816 for all of the help. See his post for an explanation of the solution, but here is the final SProc for reference:
CREATE PROCEDURE DLLIB/G_DPIVOT# (
IN TABLE_NAME CHAR(12) CCSID 37 DEFAULT '' ,
IN PIVOT CHAR(12) CCSID 37 DEFAULT '' ,
IN PIVOTFLD CHAR(12) CCSID 37 DEFAULT '' ,
IN "VALUE" DECIMAL(10, 0) DEFAULT 0 ,
INOUT LIST CHAR(5000) CCSID 37 )
LANGUAGE SQL
SPECIFIC DLLIB/G_DPIVOT#
NOT DETERMINISTIC
READS SQL DATA
CALLED ON NULL INPUT
CONCURRENT ACCESS RESOLUTION DEFAULT
SET OPTION ALWBLK = *ALLREAD ,
ALWCPYDTA = *OPTIMIZE ,
COMMIT = *NONE ,
DECRESULT = (31, 31, 00) ,
DFTRDBCOL = *NONE ,
DYNDFTCOL = *NO ,
DYNUSRPRF = *USER ,
SRTSEQ = *HEX
BEGIN
DECLARE STMT1 VARCHAR ( 1000 ) ;
DECLARE C1 CURSOR FOR S1 ;
SET STMT1 = 'WITH DETAILS ( ' || TRIM ( PIVOT ) || ' , ' || TRIM ( PIVOTFLD ) || ' , CURR , PREV ) AS ( ' ||
'SELECT ' || TRIM ( PIVOT ) || ' ,' || TRIM ( PIVOTFLD ) || ',' ||
' ROW_NUMBER ( ) OVER ( PARTITION BY ' || TRIM ( PIVOT ) || ' ORDER BY ' || TRIM ( PIVOTFLD ) || ' ) AS CURR ,' ||
' ROW_NUMBER ( ) OVER ( PARTITION BY ' || TRIM ( PIVOT ) || ' ORDER BY ' || TRIM ( PIVOTFLD ) || ' ) - 1 AS PREV' ||
' FROM ' || TRIM ( TABLE_NAME ) ||
' WHERE ' || TRIM ( PIVOT ) || ' = ' || TRIM ( VALUE ) ||
' GROUP BY ' || TRIM ( PIVOT ) || ' , ' || TRIM ( PIVOTFLD ) || ' )' ||
' SELECT MAX ( TRIM ( L '','' FROM CAST ( SYS_CONNECT_BY_PATH ( ' || TRIM ( PIVOTFLD ) || ' , '','' ) AS CHAR ( 5000 ) ) ) ) ' ||
' FROM DETAILS ' ||
' START WITH CURR = 1 ' ||
' CONNECT BY NOCYCLE ' || TRIM ( PIVOT ) || ' = PRIOR ' || TRIM ( PIVOT ) || ' AND PREV = PRIOR CURR' ;
PREPARE S1 FROM STMT1 ;
OPEN C1 ;
FETCH C1 INTO LIST ;
CLOSE C1 ;
END ;

Answer 1

The basic problem is in the EXECUTE. You can't "execute" the prepared SELECT. Instead, you'll need to DECLARE CURSOR for S1 and FETCH rows from the CURSOR. Note that 'executing' a SELECT statement wouldn't actually do anything if it was allowed; it would just "SELECT", so EXECUTE doesn't make much sense. (A SELECT INTO statement can be different, but it's not clear if that's appropriate here.)
It might be possible to OPEN a CURSOR and return a result set rather than FETCHing rows. With more definition of how you actually want to use this, some elaboration should be possible.
Edit:
Second problem:
I've created more readable versions of your original CTE and the CTE in your edited question. The original:
WITH DETAILS ( PIVOT , PIVOTFLD , CURR , PREV ) AS (
SELECT PIVOT , PIVOTFLD ,
ROW_NUMBER ( ) OVER ( PARTITION BY PIVOT ORDER BY PIVOTFLD ) AS CURR ,
ROW_NUMBER ( ) OVER ( PARTITION BY PIVOT ORDER BY PIVOTFLD ) - 1 AS PREV
FROM TABLE_NAME
WHERE PIVOT = VALUE
GROUP BY PIVOT , PIVOTFLD )
VALUES( SELECT MAX ( CAST ( SYS_CONNECT_BY_PATH ( PIVOTFLD , ',' ) AS CHAR ( 5000 ) ) ) )
FROM DETAILS
START WITH CURR = 1
CONNECT BY NOCYCLE PIVOT = PRIOR PIVOT AND PREV = PRIOR CURR) INTO ? ;
You have a VALUE INTO statement after the CTE. AFAIK, that's not valid.
And your edited example:
WITH DETAILS ( PIVOT , PIVOTFLD , CURR , PREV ) AS (
SELECT PIVOT ,
PIVOTFLD ,
ROW_NUMBER ( ) OVER ( PARTITION BY PIVOT ORDER BY PIVOTFLD ) AS CURR ,
ROW_NUMBER ( ) OVER ( PARTITION BY PIVOT ORDER BY PIVOTFLD ) - 1 AS PREV
FROM TABLE_NAME
WHERE PIVOT = VALUE
GROUP BY PIVOT , PIVOTFLD );
Well, it's just a bare CTE that has no associated SELECT referencing it. You do try to PREPARE a SELECT statement later, but the two need to go together. You can't EXECUTE the CTE by itself.
Try putting them together as a single statement and see if a CURSOR creates over the result. Variable STMT1 would then look something like this:
WITH DETAILS ( PIVOT , PIVOTFLD , CURR , PREV ) AS (
SELECT PIVOT ,
PIVOTFLD ,
ROW_NUMBER ( ) OVER ( PARTITION BY PIVOT ORDER BY PIVOTFLD ) AS CURR ,
ROW_NUMBER ( ) OVER ( PARTITION BY PIVOT ORDER BY PIVOTFLD ) - 1 AS PREV
FROM TABLE_NAME
WHERE PIVOT = VALUE
GROUP BY PIVOT , PIVOTFLD )
SELECT * FROM DETAILS ;
Note that the statement includes the SELECT at the end. The WITH ... clause is followed by the SELECT ... in a single statement that is PREPAREd. The CURSOR would then be OPENed over that statement.
Edit 2:
I have modified a sample CTE that I've had for a while to fit into a stored proc and to return a value. It was compiled and run on my i 6.1 system. The CTE is PREPAREd from a string placed into a VARCHAR, then a CURSOR is opened over it. Rows are FETCHed in a WHILE-loop.
The CTE generates summary rows that are then UNIONed with detail rows from QIWS/QCUSTCDT. The summary is by STATE to provided a sub-total of BALDUE. The WHILE-loop is kind of meaningless; it only shows FETCHing and processing of rows. The only action is to count the number of rows that are not summary rows out of the CTE. This is essentially the same as the number of rows in the base table. The row count is returned in the rowCnt OUT parameter.
The source code is copy/pasted, but comes from two sources. First, the CREATE PROCEDURE statement is taken from iNavigator's 'Run SQL scripts' utility after generating the SQL from the compiled stored procedure. And second, the BEGIN ... END compound statement body is from the original I typed into the iNavigator New-> Procedure function. Although the two would have logical equivalence, I wanted to preserve the actual lines that were input. You can copy/paste the entire source into 'Run SQL Scripts' or go through the utility to create the procedure and only copy/paste the BEGIN ... END compound statement after entering values into the first two tabs of the New-> Procedure function.
I have a schema named SQLEXAMPLE that I build things like this into. You'll need to adjust the schema and procedure names to fit your environment. The QIWS/QCUSTCDT table should exist on nearly all AS/400-series systems.
CREATE PROCEDURE SQLEXAMPLE.CTE_CustCDT (
OUT rowCnt INTEGER )
LANGUAGE SQL
SPECIFIC SQLEXAMPLE.CTECUSTCDT
NOT DETERMINISTIC
READS SQL DATA
CALLED ON NULL INPUT
SET OPTION ALWBLK = *ALLREAD ,
ALWCPYDTA = *OPTIMIZE ,
COMMIT = *NONE ,
CLOSQLCSR = *ENDMOD ,
DECRESULT = (31, 31, 00) ,
DFTRDBCOL = *NONE ,
DYNDFTCOL = *NO ,
DYNUSRPRF = *USER ,
SRTSEQ = *HEX
BEGIN
DECLARE sumRows INTEGER DEFAULT 0 ;
DECLARE cusNum INTEGER ;
DECLARE lstNam CHAR(10) ;
DECLARE state CHAR(2) ;
DECLARE balDue DECIMAL(7, 2) ;
DECLARE stmt1 VARCHAR(512) ;
DECLARE at_end INT DEFAULT 0 ;
DECLARE not_found
CONDITION FOR '02000';
DECLARE c1 CURSOR FOR c1Stmt ;
DECLARE CONTINUE HANDLER FOR not_found
SET at_end = 1 ;
SET stmt1 = 'with t1 As(
SELECT 0 ,''Tot'' , state , sum( balDue )
FROM qiws.qcustcdt
GROUP BY state
ORDER BY state
)
select cusNum , lstNam , state, balDue
from qiws.qcustcdt
union
select *
from t1
order by state FOR FETCH ONLY' ;
PREPARE c1Stmt FROM stmt1 ;
OPEN c1 ;
FETCH C1 INTO cusNum , lstNam , state , balDue ;
WHILE at_end = 0 DO
IF cusNum <> 0 THEN SET sumRows = sumRows + 1 END IF ;
FETCH C1 INTO cusNum , lstNam , state , balDue ;
END WHILE ;
SET rowCnt = sumRows ;
CLOSE c1 ;
END
When the CTE is run by itself in STRSQL, the first few lines of output look like this:
....+....1....+....2....+....3....+....4....+....5....+....6....+....7....+.
CUSNUM LSTNAM STATE BALDUE
475,938 Doe CA 250.00
0 Tot CA 250.00
389,572 Stevens CO 58.75
0 Tot CO 58.75
938,485 Johnson GA 3,987.50
0 Tot GA 3,987.50
846,283 Alison MN 10.00
583,990 Abraham MN 500.00
0 Tot MN 510.00
The summary rows should easily be recognized. And when the stored proc is CALLed from 'Run SQL Scripts', the resulting output is:
Connected to relational database TISI on Tisi as Toml - 090829/Quser/Qzdasoinit
> call SQLEXAMPLE.CTE_CustCDT( 0 )
Return Code = 0
Output Parameter #1 = 12
Statement ran successfully (570 ms)
The QIWS/QCUSTCDT table on that system has 12 rows, and that matches the value returned.
It's not exactly the same as your desired CTE, but it should demonstrate that a dynamic CTE can be used. It also shows how FETCHes might pull rows from the CTE for whatever purpose is needed.