Pivoting a table in iSeries DB2 dynamically - sql

So what I'm trying to do is convert rows into columns. This has been covered before on this site in a verity of ways, the way I like the best is as follows:
SELECT
FRNPCM032.WFP.WFORD#,
MAX(CASE WHEN FRNPCM032.WFP.WFSEGN = 'COLOR1' THEN WFVAL END) AS COLOR,
MAX(CASE WHEN FRNPCM032.WFP.WFSEGN = 'OSKVA' THEN WFVAL END) AS KVA,
MAX(CASE WHEN FRNPCM032.WFP.WFSEGN = 'OSSWITCH' THEN WFVAL END) AS LBSWITCH
FROM FRNPCM032.WFP
GROUP BY FRNPCM032.WFP.WFORD#;
This is a very simple way, but I need something less manual than this because new rows might be added later and I dont want to have to go back to keep maintaining a query or view.
Is there a way I can dynamically do this? I can think of a way to do it with loops, but i cant do that in a query or view.

Unfortunately, DB2 for IBM i doesn't have a PIVOT() function...
However, you can build a procedure that dynamically looks at the current contents of the table and builds an SQL statement that pivots the data.
The following code was taken from the article, An SQL Pivot Procedure
If you wanted you could modify the code to build a view, rather than returning a result set. But you'd have to rebuild the view if the pivot values changed.
CREATE PROCEDURE DO_PIVOT
(IN FOR_SCHEMA CHARACTER (10) ,
IN FOR_TABLE CHARACTER (10) ,
IN PIVOT_COLUMN VARCHAR (250) ,
IN VALUE_COLUMN VARCHAR (250) ,
IN AGG_FUNCTION VARCHAR (5) DEFAULT 'SUM' ,
IN GROUP_COLUMN VARCHAR (250) DEFAULT NULL )
LANGUAGE SQL
MODIFIES SQL DATA
PROGRAM TYPE SUB
CONCURRENT ACCESS RESOLUTION DEFAULT
DYNAMIC RESULT SETS 1
OLD SAVEPOINT LEVEL COMMIT ON RETURN NO
BEGIN
DECLARE SQLCODE INTEGER DEFAULT 0 ;
DECLARE SQL_STATEMENT VARCHAR ( 5000 ) ;
DECLARE PIVOT_VALUE VARCHAR ( 20 ) ;
DECLARE PAD CHAR ( 2 ) DEFAULT ' ' ;
DECLARE C1 CURSOR FOR D1 ;
DECLARE C2 CURSOR WITH RETURN FOR D2 ;
SET SCHEMA = FOR_SCHEMA ;
-- Get the list of values available for the pivot column
-- Each value will be a column in the return set
SET SQL_STATEMENT = 'select distinct '
|| PIVOT_COLUMN
|| ' from '
|| FOR_TABLE
|| ' order by 1' ;
PREPARE D1 FROM SQL_STATEMENT ;
OPEN C1 ;
-- Construct a dynamic select statement for the pivot
SET SQL_STATEMENT = 'select ' ;
-- If requested, add the Group By Column
-- to the select clause
IF GROUP_COLUMN IS NOT NULL THEN
SET SQL_STATEMENT = SQL_STATEMENT || GROUP_COLUMN ;
SET PAD = ', ' ;
END IF ;
-- For each possible value for the Pivot Column,
-- add a case statement to perform the requested
-- aggregate function on the Value Column
FETCH NEXT FROM C1 INTO PIVOT_VALUE ;
WHILE ( SQLCODE >= 0 AND SQLCODE <> 100 ) DO
SET SQL_STATEMENT = SQL_STATEMENT
|| PAD
|| AGG_FUNCTION
|| '(CASE WHEN '
|| PIVOT_COLUMN
|| ' = '''
|| PIVOT_VALUE
|| ''' THEN '
|| VALUE_COLUMN
|| ' END) AS '
|| PIVOT_VALUE ;
SET PAD = ', ' ;
FETCH NEXT FROM C1 INTO PIVOT_VALUE ;
END WHILE ;
CLOSE C1 ;
-- Specify the table to select from
SET SQL_STATEMENT = SQL_STATEMENT
|| ' from '
|| FOR_TABLE ;
-- If requested, add the Group By Column
-- to the select clause
IF GROUP_COLUMN IS NOT NULL THEN
SET SQL_STATEMENT = SQL_STATEMENT
|| ' group by '
|| GROUP_COLUMN
|| ' order by '
|| GROUP_COLUMN;
END IF ;
PREPARE D2 FROM SQL_STATEMENT ;
OPEN C2 ;
END ;
LABEL ON ROUTINE DO_PIVOT
( CHAR(), CHAR(), VARCHAR(), VARCHAR(), VARCHAR(), VARCHAR() )
IS 'Perform a General Purpose Pivot';
COMMENT ON PARAMETER ROUTINE DO_PIVOT
( CHAR(), CHAR(), VARCHAR(), VARCHAR(), VARCHAR(), VARCHAR() )
(FOR_SCHEMA IS 'Schema for Table' ,
FOR_TABLE IS 'For Table' ,
PIVOT_COLUMN IS 'Name of Column to be Pivoted' ,
VALUE_COLUMN IS 'Column to be Aggregated for Pivot' ,
AGG_FUNCTION IS 'Use Aggregate Function' ,
GROUP_COLUMN IS 'Group on Column' ) ;

Related

Subselect statement to set integrity

I am trying to set integrity to about 1000 tables which error out because of Pending State. I cannot go 1 by 1 as there are too many tables. The subquery returns the name of all the tables. This is the query that I use which is not working right now:
SET INTEGRITY FOR TABSCHEMA.TABNAME IMMEDIATE CHECKED IN
( SELECT TABNAME
FROM SYSCAT.TABLES
WHERE ( CONST_CHECKED LIKE '%N%' AND TABSCHEMA = 'FINANCE')
WITH ur
)
Any idea?
The main problem of such a massive SET INTEGRITYs is, that if you have a parent-child pair in check-pending state, that you have to either include both tables into a single SET INTEGRITY command or run it on the parent table first, and on the child table with a subsequent command. You get an error, if you run SET INTEGRITY on a child table only, if the corresponding parent table is in the check pending state.
It's quite a non-trivial task to split all tables in check pending to distinct non-relative groups to run a single SET INTEGRITY on each such a group of tables.
This is why it's better to run a script like below:
--#SET TERMINATOR #
SET SERVEROUTPUT ON#
DECLARE GLOBAL TEMPORARY TABLE SESSION.BAD_TABLES
(
TABSCHEMA VARCHAR (128) NOT NULL
, TABNAME VARCHAR (128) NOT NULL
) WITH REPLACE ON COMMIT PRESERVE ROWS NOT LOGGED#
BEGIN
--DECLARE L_ITER INT DEFAULT 0;
DECLARE L_PROCESSED INT;
DECLARE L_TABSCHEMA VARCHAR (128);
DECLARE L_TABNAME VARCHAR (128);
DECLARE CONTINUE HANDLER FOR SQLSTATE '23514'
BEGIN
INSERT INTO SESSION.BAD_TABLES (TABSCHEMA, TABNAME) VALUES (L_TABSCHEMA, L_TABNAME);
END;
-- Ordinal tables processing
L1: LOOP
--SET L_ITER = L_ITER + 1;
--CALL DBMS_OUTPUT.PUT_LINE ('Iteration ' || L_ITER);
SET L_PROCESSED = 0;
FOR V AS C1 INSENSITIVE CURSOR WITH HOLD FOR
SELECT
'SET INTEGRITY FOR "' || T.TABSCHEMA || '"."' || T.TABNAME || '" IMMEDIATE CHECKED' AS CMD
, T.TABSCHEMA
, T.TABNAME
FROM SYSCAT.TABLES T
WHERE T.TYPE = 'T' AND T.STATUS = 'C'
AND NOT EXISTS
(
SELECT 1
FROM SYSCAT.REFERENCES R
JOIN SYSCAT.TABLES P ON P.TABSCHEMA = R.REFTABSCHEMA AND P.TABNAME = R.REFTABNAME
WHERE R.TABSCHEMA = T.TABSCHEMA AND R.TABNAME = T.TABNAME
AND P.STATUS = 'C'
)
AND NOT EXISTS
(
SELECT 1 FROM SESSION.BAD_TABLES B WHERE B.TABSCHEMA = T.TABSCHEMA AND B.TABNAME = T.TABNAME
)
DO
SET (L_TABSCHEMA, L_TABNAME) = (V.TABSCHEMA, V.TABNAME);
EXECUTE IMMEDIATE V.CMD;
COMMIT;
CALL DBMS_OUTPUT.PUT_LINE (V.CMD);
SET L_PROCESSED = L_PROCESSED + 1;
END FOR;
CALL DBMS_OUTPUT.PUT_LINE ('Tables processed: ' || L_PROCESSED);
IF L_PROCESSED = 0 THEN LEAVE L1; END IF;
END LOOP L1;
-- MQTs processing
SET L_PROCESSED = 0;
FOR V AS C1 INSENSITIVE CURSOR WITH HOLD FOR
SELECT
'SET INTEGRITY FOR "' || T.TABSCHEMA || '"."' || T.TABNAME || '" IMMEDIATE CHECKED' AS CMD
, T.TABSCHEMA
, T.TABNAME
FROM SYSCAT.TABLES T
WHERE T.TYPE = 'S' AND T.STATUS = 'C'
DO
SET (L_TABSCHEMA, L_TABNAME) = (V.TABSCHEMA, V.TABNAME);
EXECUTE IMMEDIATE V.CMD;
COMMIT;
CALL DBMS_OUTPUT.PUT_LINE (V.CMD);
SET L_PROCESSED = L_PROCESSED + 1;
END FOR;
CALL DBMS_OUTPUT.PUT_LINE ('MQTs processed: ' || L_PROCESSED);
END
#
SET SERVEROUTPUT OFF#
Ordinal tables are processed iteratively. Each iteration processes a table, if it doesn't have its parent table in check pending at the moment.
MQTs are processed afterwards.
The table name is inserted into a session table, if SET INTEGRITY failed on it.

How to display column name and column comment, when try DML

I'm try use DML operations on table, when insert or update. I need to show column name and column comment when the operation failed. For example code:
CREATE TABLE test_test(col1 VARCHAR2(10), col2 VARCHAR2(100) not null);
DECLARE
ex_insert_null EXCEPTION;
PRAGMA EXCEPTION_INIT(ex_insert_null, -1400);
ex_value_too_large EXCEPTION;
PRAGMA EXCEPTION_INIT(ex_value_too_large, -12899);
BEGIN
INSERT INTO test_test
(col1
,col2)
SELECT CASE
WHEN LEVEL = 8 THEN
(LEVEL + 1) || 'qqqqqqqqqqqq'
ELSE
(LEVEL + 2) || 'qqq'
END AS col1
,CASE
WHEN LEVEL = 7 THEN
NULL
ELSE
(LEVEL + 3) || 'wwwwwww'
END AS col2
FROM dual
CONNECT BY LEVEL <= 10;
COMMIT;
EXCEPTION
WHEN ex_insert_null THEN
ROLLBACK;
dbms_output.put_line('ex_insert_null at ' || ' ' /* || column_name || ' ' || column_comment */);
WHEN ex_value_too_large THEN
ROLLBACK;
dbms_output.put_line('ex_value_too_large at ' || ' ' /* || column_name || ' ' || column_comment */);
END;
/
As APC has pointed out, you could use "existing Oracle exceptions" eg if you had something like ...
procedure insert_( col1 varchar2, col2 varchar2 )
is
v_errorcode varchar2(64) ;
v_errormsg varchar2(128) ;
begin
insert into t ( c1, c2 ) values ( col1, col2 ) ;
exception
when others then
if sqlcode = -1400 or sqlcode = -12899 then
v_errorcode := sqlcode;
v_errormsg := substr( sqlerrm, 1, 128 );
dbms_output.put_line( v_errorcode || ' ' || v_errormsg ) ;
raise;
end if;
end insert_ ;
... you could get error messages such as these:
-1400 ORA-01400: cannot insert NULL into ("MYSCHEMA"."T"."C2")
-12899 ORA-12899: value too large for column "MYSCHEMA"."T"."C1" (actual: 13, maximum: 10)
If this is enough information for you, fine. However, you also want to see the COMMENTS for the columns. Although we could get the column names from the SQLERRM strings, it may be more reliable to use user-defined exceptions (as you have hinted).
As a starting point, the following DDL and PACKAGE code may be of use for you. ( see also: dbfiddle here )
Tables:
drop table t cascade constraints ;
drop table errorlog cascade constraints ;
create table t (
c1 varchar2(10)
, c2 varchar2(64) not null
) ;
comment on column t.c1 is 'this is the column comment for c1';
comment on column t.c2 is 'this is the column comment for c2';
create table errorlog (
when_ timestamp
, msg varchar2(4000)
) ;
Package spec
create or replace package P is
-- insert into T, throwing exceptions
procedure insert_( col1 varchar2, col2 varchar2 );
-- use your example SELECT, call the insert_ procedure
procedure insert_test ;
-- retrieve the column comments from user_col_comments
function fetch_comment( table_ varchar2, col_ varchar2 ) return varchar2 ;
end P ;
/
Package body
create or replace package body P is
procedure insert_( col1 varchar2, col2 varchar2 )
is
ex_value_too_large exception ; -- T.c1: varchar2(10)
ex_insert_null exception ; -- T.c2: cannot be null
v_errorcol varchar2(32) := '' ;
v_comment varchar2(128) := '' ;
v_tablename constant varchar2(32) := upper('T') ;
begin
if length( col1 ) > 10 then
v_errorcol := upper('C1') ;
raise ex_value_too_large ;
end if;
if col2 is null then
v_errorcol := upper('C2') ;
raise ex_insert_null ;
end if ;
insert into t ( c1, c2 ) values ( col1, col2 ) ;
exception
when ex_value_too_large then
dbms_output.put_line( ' ex_value_too_large # '
|| v_errorcol || ' (' || fetch_comment( v_tablename, v_errorcol ) || ')' );
when ex_insert_null then
dbms_output.put_line( ' ex_insert_null # '
|| v_errorcol || ' (' || fetch_comment( v_tablename, v_errorcol ) || ')' );
when others then
raise ;
end insert_ ;
procedure insert_test
is
begin
for rec_ in (
select
case
when level = 8 then ( level + 1 ) || 'qqqqqqqqqqqq'
else ( level + 2 ) || 'qqq'
end as col1
, case
when level = 7 then null
else ( level + 3 ) || 'wwwwwww'
end as col2
from dual
connect by level <= 10
) loop
insert_( rec_.col1, rec_.col2 ) ;
end loop;
commit;
end insert_test;
function fetch_comment( table_ varchar2, col_ varchar2 ) return varchar2
is
v_comment varchar2(4000) ; -- same datatype as in user_tab_comments
begin
select comments into v_comment
from user_col_comments
where table_name = table_
and column_name = col_ ;
return v_comment ;
end fetch_comment ;
end P ;
/
For testing the package code, execute the following anonymous block:
begin
P.insert_test ;
end;
/
-- output
ex_insert_null # C2 (this is the column comment for c2)
ex_value_too_large # C1 (this is the column comment for c1)
-- Table T contains:
SQL> select * from T;
C1 C2
3qqq 4wwwwwww
4qqq 5wwwwwww
5qqq 6wwwwwww
6qqq 7wwwwwww
7qqq 8wwwwwww
8qqq 9wwwwwww
11qqq 12wwwwwww
12qqq 13wwwwwww
In the dbfiddle, all output will be written to T and ERRORLOG, respectively. You can also use dbms_output.put_line (which is commented out in the dbfiddle) if needed. Notice that the cursor for loop in the insert_test procedure is inefficient (we could use BULK operations). Also, you need to decide where and how the exceptions are handled. As mentioned, this example is just a starting point - which will probably need lots of refinements.

Pivoting DB2 Data - where number of columns and and rows are not fixed

I've seen a few similar Q/A's on here, but the cases I've seen were variations on the "sales by year across quarter" variety, so DECODE is used, with 4 categories.
In my case, I don't know in advance how many rows or columns the pivot will have.
| Pay | Age | Value |
|-----|-----|-------|
| 1 | 1 | 10 |
| 1 | 2 | 20 |
| 1 | 3 | 30 |
| 2 | 1 | 90 |
| 2 | 2 | 80 |
| 2 | 3 | 70 |
and we want the result set as
PAYGROUP Millennials GenX Boomers
1 10 20 30
2 90 80 70
This would be easy with a PIVOT statement, i.e.
Transform Max(VALUE) AS V
SELECT PAYGROUP
FROM table
GROUP BY PAYGROUP
PIVOT AGEGROUP;
but my DB2 has no PIVOT function.
The number of pay groups and age groups may vary from case to case, e.g., the data can have different numbers of pay and age groupings for different cases.
It can be done, but only within an SQL procedure or other HLL.
You used one SQL statement to find out what/how many distinct values then build a dynamic SQL statement using those values.
Can probably point you to an example if you add your DB2 platform and version to the question.
Here's a pure SQL procedure (originally for DB2 for IBM i but should work for LUW ) found here: https://www.itjungle.com/2015/04/21/fhg042115-story01/
SET SCHEMA = WHER_YOU_WANT_IT;
CREATE PROCEDURE DO_PIVOT
(IN FOR_SCHEMA CHARACTER (10) ,
IN FOR_TABLE CHARACTER (10) ,
IN PIVOT_COLUMN VARCHAR (250) ,
IN VALUE_COLUMN VARCHAR (250) ,
IN AGG_FUNCTION VARCHAR (5) DEFAULT 'SUM' ,
IN GROUP_COLUMN VARCHAR (250) DEFAULT NULL )
LANGUAGE SQL
MODIFIES SQL DATA
PROGRAM TYPE SUB
CONCURRENT ACCESS RESOLUTION DEFAULT
DYNAMIC RESULT SETS 1
OLD SAVEPOINT LEVEL COMMIT ON RETURN NO
BEGIN
DECLARE SQLCODE INTEGER DEFAULT 0 ;
DECLARE SQL_STATEMENT VARCHAR ( 5000 ) ;
DECLARE PIVOT_VALUE VARCHAR ( 20 ) ;
DECLARE PAD CHAR ( 2 ) DEFAULT ' ' ;
DECLARE C1 CURSOR FOR D1 ;
DECLARE C2 CURSOR WITH RETURN FOR D2 ;
SET SCHEMA = FOR_SCHEMA ;
-- Get the list of values available for the pivot column
-- Each value will be a column in the return set
SET SQL_STATEMENT = 'select distinct '
|| PIVOT_COLUMN
|| ' from '
|| FOR_TABLE
|| ' order by 1' ;
PREPARE D1 FROM SQL_STATEMENT ;
OPEN C1 ;
-- Construct a dynamic select statement for the pivot
SET SQL_STATEMENT = 'select ' ;
-- If requested, add the Group By Column
-- to the select clause
IF GROUP_COLUMN IS NOT NULL THEN
SET SQL_STATEMENT = SQL_STATEMENT || GROUP_COLUMN ;
SET PAD = ', ' ;
END IF ;
-- For each possible value for the Pivot Column,
-- add a case statement to perform the requested
-- aggregate function on the Value Column
FETCH NEXT FROM C1 INTO PIVOT_VALUE ;
WHILE ( SQLCODE >= 0 AND SQLCODE <> 100 ) DO
SET SQL_STATEMENT = SQL_STATEMENT
|| PAD
|| AGG_FUNCTION
|| '(CASE WHEN '
|| PIVOT_COLUMN
|| ' = '''
|| PIVOT_VALUE
|| ''' THEN '
|| VALUE_COLUMN
|| ' END) AS '
|| PIVOT_VALUE ;
SET PAD = ', ' ;
FETCH NEXT FROM C1 INTO PIVOT_VALUE ;
END WHILE ;
CLOSE C1 ;
-- Specify the table to select from
SET SQL_STATEMENT = SQL_STATEMENT
|| ' from '
|| FOR_TABLE ;
-- If requested, add the Group By Column
-- to the select clause
IF GROUP_COLUMN IS NOT NULL THEN
SET SQL_STATEMENT = SQL_STATEMENT
|| ' group by '
|| GROUP_COLUMN
|| ' order by '
|| GROUP_COLUMN;
END IF ;
PREPARE D2 FROM SQL_STATEMENT ;
OPEN C2 ;
END ;
LABEL ON ROUTINE DO_PIVOT
( CHAR(), CHAR(), VARCHAR(), VARCHAR(), VARCHAR(), VARCHAR() )
IS 'Perform a General Purpose Pivot';
COMMENT ON PARAMETER ROUTINE DO_PIVOT
( CHAR(), CHAR(), VARCHAR(), VARCHAR(), VARCHAR(), VARCHAR() )
(FOR_SCHEMA IS 'Schema for Table' ,
FOR_TABLE IS 'For Table' ,
PIVOT_COLUMN IS 'Name of Column to be Pivoted' ,
VALUE_COLUMN IS 'Column to be Aggregated for Pivot' ,
AGG_FUNCTION IS 'Use Aggregate Function' ,
GROUP_COLUMN IS 'Group on Column' ) ;
I think you could also use the XML functions to pivot the data...but I've yet to find a good example of that

Count the number of null values into an Oracle table?

I need to count the number of null values of all the columns in a table in Oracle.
For instance, I execute the following statements to create a table TEST and insert data.
CREATE TABLE TEST
( A VARCHAR2(20 BYTE),
B VARCHAR2(20 BYTE),
C VARCHAR2(20 BYTE)
);
Insert into TEST (A) values ('a');
Insert into TEST (B) values ('b');
Insert into TEST (C) values ('c');
Now, I write the following code to compute the number of null values in the table TEST:
declare
cnt number :=0;
temp number :=0;
begin
for r in ( select column_name, data_type
from user_tab_columns
where table_name = upper('test')
order by column_id )
loop
if r.data_type <> 'NOT NULL' then
select count(*) into temp FROM TEST where r.column_name IS NULL;
cnt := cnt + temp;
END IF;
end loop;
dbms_output.put_line('Total: '||cnt);
end;
/
It returns 0, when the expected value is 6.
Where is the error?
Thanks in advance.
Counting NULLs for each column
In order to count NULL values for all columns of a table T you could run
SELECT COUNT(*) - COUNT(col1) col1_nulls
, COUNT(*) - COUNT(col2) col2_nulls
,..
, COUNT(*) - COUNT(colN) colN_nulls
, COUNT(*) total_rows
FROM T
/
Where col1, col2, .., colN should be replaced with actual names of columns of T table.
Aggregate functions -like COUNT()- ignore NULL values, so COUNT(*) - COUNT(col) will give you how many nulls for each column.
Summarize all NULLs of a table
If you want to know how many fields are NULL, I mean every NULL of every record you can
WITH d as (
SELECT COUNT(*) - COUNT(col1) col1_nulls
, COUNT(*) - COUNT(col2) col2_nulls
,..
, COUNT(*) - COUNT(colN) colN_nulls
, COUNT(*) total_rows
FROM T
) SELECT col1_nulls + col1_nulls +..+ colN_null
FROM d
/
Summarize all NULLs of a table (using Oracle dictionary tables)
Following is an improvement in which you need to now nothing but table name and it is very easy to code a function based on it
DECLARE
T VARCHAR2(64) := '<YOUR TABLE NAME>';
expr VARCHAR2(32767);
q INTEGER;
BEGIN
SELECT 'SELECT /*+FULL(T) PARALLEL(T)*/' || COUNT(*) || ' * COUNT(*) OVER () - ' || LISTAGG('COUNT(' || COLUMN_NAME || ')', ' + ') WITHIN GROUP (ORDER BY COLUMN_ID) || ' FROM ' || T
INTO expr
FROM USER_TAB_COLUMNS
WHERE TABLE_NAME = T;
-- This line is for debugging purposes only
DBMS_OUTPUT.PUT_LINE(expr);
EXECUTE IMMEDIATE expr INTO q;
DBMS_OUTPUT.PUT_LINE(q);
END;
/
Due to calculation implies a full table scan, code produced in expr variable was optimized for parallel running.
User defined function null_fields
Function version, also includes an optional parameter to be able to run on other schemas.
CREATE OR REPLACE FUNCTION null_fields(table_name IN VARCHAR2, owner IN VARCHAR2 DEFAULT USER)
RETURN INTEGER IS
T VARCHAR2(64) := UPPER(table_name);
o VARCHAR2(64) := UPPER(owner);
expr VARCHAR2(32767);
q INTEGER;
BEGIN
SELECT 'SELECT /*+FULL(T) PARALLEL(T)*/' || COUNT(*) || ' * COUNT(*) OVER () - ' || listagg('COUNT(' || column_name || ')', ' + ') WITHIN GROUP (ORDER BY column_id) || ' FROM ' || o || '.' || T || ' t'
INTO expr
FROM all_tab_columns
WHERE table_name = T;
EXECUTE IMMEDIATE expr INTO q;
RETURN q;
END;
/
-- Usage 1
SELECT null_fields('<your table name>') FROM dual
/
-- Usage 2
SELECT null_fields('<your table name>', '<table owner>') FROM dual
/
Thank you #Lord Peter :
The below PL/SQL script works
declare
cnt number :=0;
temp number :=0;
begin
for r in ( select column_name, nullable
from user_tab_columns
where table_name = upper('test')
order by column_id )
loop
if r.nullable = 'Y' then
EXECUTE IMMEDIATE 'SELECT count(*) FROM test where '|| r.column_name ||' IS NULL' into temp ;
cnt := cnt + temp;
END IF;
end loop;
dbms_output.put_line('Total: '||cnt);
end;
/
The table name test may be replaced the name of table of your interest.
I hope this solution is useful!
The dynamic SQL you execute (this is the string used in EXECUTE IMMEDIATE) should be
select sum(
decode(a,null,1,0)
+decode(b,null,1,0)
+decode(c,null,1,0)
) nullcols
from test;
Where each summand corresponds to a NOT NULL column.
Here only one table scan is necessary to get the result.
Use the data dictionary to find the number of NULL values almost instantly:
select sum(num_nulls) sum_num_nulls
from all_tab_columns
where owner = user
and table_name = 'TEST';
SUM_NUM_NULLS
-------------
6
The values will only be correct if optimizer statistics were gathered recently and if they were gathered with the default value for the sample size.
Those may seem like large caveats but it's worth becoming familiar with your database's statistics gathering process anyway. If your database is not automatically gathering statistics or if your database is not using the default sample size those are likely huge problems you need to be aware of.
To manually gather stats for a specific table a statement like this will work:
begin
dbms_stats.gather_table_stats(user, 'TEST');
end;
/
select COUNT(1) TOTAL from table where COLUMN is NULL;

Dynamic CTE's as part of a SProc in DB2/400

I'm trying to write a SProc in db2/400 in a V7R2 environment which creates a CTE based on the parameters passed. I then need to perform a recursive query on the CTE.
I'm running issues into creating and executing the dynamic CTE.
According to http://www-01.ibm.com/support/knowledgecenter/ssw_ibm_i_72/db2/rbafzpreph2.htm
the prepare statement does not work with the WITH or SELECT statements directly.
I tried to wrap both the dynamic CTE and dynamic SELECT in a VALUES INTO and manage to successfully prepare the statement. The issue then comes when I try to execute the statement.
I get an error code of SQL0518 which is defined here (CTRL+F for 'SQL0518' to jump down): http://publib.boulder.ibm.com/iseries/v5r2/ic2924/index.htm?info/rzala/rzalamsg.html (NOTE*: This link is for V5R2 but the error code and text portion of my error is exact to the error listed here with the same code. So I'm sure the error code remained the same between versions)
From the 3 recovery suggestions listed, the second seems unlikely to be the case since my execute is the very next line after my prepare. Suggestion 3 also seems unlikely because there is no use of commit or rollback. So I am inclined to believe suggestion 1 applies to my particular case. However, I do not understand how to take the suggested steps.
If &1 identifies a prepared SELECT or DECLARE PROCEDURE statement, a different prepared statement must be named in the EXECUTE statement.
Am I supposed to have two prepare statements for the same execute? Syntactically how would this look?
Here is the code for my SProc for reference:
CREATE OR REPLACE PROCEDURE DLLIB/G_DPIVOT# (
IN TABLE_NAME CHAR(12) CCSID 37 DEFAULT '' ,
IN PIVOT CHAR(12) CCSID 37 DEFAULT '' ,
IN PIVOTFLD CHAR(12) CCSID 37 DEFAULT '' ,
IN "VALUE" DECIMAL(10, 0) DEFAULT 0 ,
INOUT LIST CHAR(5000) CCSID 37 )
LANGUAGE SQL
SPECIFIC DLLIB/G_DPIVOT#
NOT DETERMINISTIC
READS SQL DATA
CALLED ON NULL INPUT
CONCURRENT ACCESS RESOLUTION DEFAULT
SET OPTION ALWBLK = *ALLREAD ,
ALWCPYDTA = *OPTIMIZE ,
COMMIT = *NONE ,
DECRESULT = (31, 31, 00) ,
DFTRDBCOL = *NONE ,
DYNDFTCOL = *NO ,
DYNUSRPRF = *USER ,
SRTSEQ = *HEX
BEGIN
DECLARE STMT1 VARCHAR ( 1000 ) ;
SET STMT1 = 'WITH DETAILS ( ' || TRIM ( PIVOT ) || ' , ' || TRIM ( PIVOTFLD ) || ' , CURR , PREV ) AS ( ' ||
'SELECT ' || TRIM ( PIVOT ) || ' ,' || TRIM ( PIVOTFLD ) || ',' ||
' ROW_NUMBER ( ) OVER ( PARTITION BY ' || TRIM ( PIVOT ) || ' ORDER BY ' || TRIM ( PIVOTFLD ) || ' ) AS CURR ,' ||
' ROW_NUMBER ( ) OVER ( PARTITION BY ' || TRIM ( PIVOT ) || ' ORDER BY ' || TRIM ( PIVOTFLD ) || ' ) - 1 AS PREV' ||
' FROM ' || TRIM ( TABLE_NAME ) ||
' WHERE ' || TRIM ( PIVOT ) || ' = ' || TRIM ( VALUE ) ||
' GROUP BY ' || TRIM ( PIVOT ) || ' , ' || TRIM ( PIVOTFLD ) || ' )' ||
' VALUES( SELECT MAX ( TRIM ( L '','' FROM CAST ( SYS_CONNECT_BY_PATH ( ' || TRIM ( PIVOTFLD ) || ' , '','' ) AS CHAR ( 5000 ) ) ) )' ||
' FROM DETAILS ' ||
' START WITH CURR = 1 ' ||
' CONNECT BY NOCYCLE ' || TRIM ( PIVOT ) || ' = PRIOR ' || TRIM ( PIVOT ) || ' AND PREV = PRIOR CURR) INTO ?' ;
--SET LIST = STMT1; -- If I execute the value of LIST in interactive SQL everything is as expected (minus the VALUES INTO ofcourse)
PREPARE S1 FROM STMT1 ;
EXECUTE S1 USING LIST; -- If I comment this I don't get an error, but I also don't get a return value in LIST)
END ;
Any assistance is appreciated.
EDIT 1: I am trying to create a SProc (which I will use to create a UDF) which has 5 parameters. I am trying to pivot a single field spanning across multiple records so the values are returned as a comma delimited string. I want to make this dynamic though so I can re-use it for many situations. An example call would be: CALL DLLIB.G_DPIVOT#(TABLE, PIVOT, PIVOTFLD, VALUE, LIST); Where TABLE is the name of the table I want to pivot, PIVOT is the commonality between records (FK), PIVOTFLD is the field I want to condense to a single string, VALUE is the FK value I want to use to pivot on, and LIST is the OUT parameter which would contain the resulting string. You can read more about a non-dynamic implementation here: http://www.mcpressonline.com/sql/techtip-combining-multiple-row-values-into-a-single-row-with-sql-in-db2-for-i.html
The use is for when I have a header table which has a one-to-many relationship with another table. I'll then be able to summarize all the values of a particular field in the "many" table based on the PK/FK relationship.
EDIT 2:
Here is a recent attempt which I think I manage to successfully create the CTE using EXECUTE IMMEDIATE and am now trying to just perform a simple select on it. I'm trying to make use of DB2 cursors but get at error at the "C2" on the line DECLARE C2 CURSOR FOR S2;. I don't have much experience with DB2 cursors but believe I am using them in the correct way.
DECLARE STMT1 VARCHAR ( 1000 ) ;
DECLARE STMT2 VARCHAR ( 1000 ) ;
SET STMT1 = 'WITH DETAILS ( ' || TRIM ( PIVOT ) || ' , ' || TRIM ( PIVOTFLD ) || ' , CURR , PREV ) AS ( ' ||
'SELECT ' || TRIM ( PIVOT ) || ' ,' || TRIM ( PIVOTFLD ) || ',' ||
' ROW_NUMBER ( ) OVER ( PARTITION BY ' || TRIM ( PIVOT ) || ' ORDER BY ' || TRIM ( PIVOTFLD ) || ' ) AS CURR ,' ||
' ROW_NUMBER ( ) OVER ( PARTITION BY ' || TRIM ( PIVOT ) || ' ORDER BY ' || TRIM ( PIVOTFLD ) || ' ) - 1 AS PREV' ||
' FROM ' || TRIM ( TABLE_NAME ) ||
' WHERE ' || TRIM ( PIVOT ) || ' = ' || TRIM ( VALUE ) ||
' GROUP BY ' || TRIM ( PIVOT ) || ' , ' || TRIM ( PIVOTFLD ) || ' )';
EXECUTE IMMEDIATE STMT1;
SET STMT2 = "SELECT * FROM DETAILS";
PREPARE S2 FROM STMT2;
DECLARE C2 CURSOR FOR S2;
OPEN C2;
FETCH C2 INTO LIST;
CLOSE C2;
Does anyone see anything wrong with these changes?
Here is the exact error message (excluding suggestion text):
SQL State: 42601
Vendor Code: -104
Message: [SQL0104] Token C2 was not valid. Valid tokens: GLOBAL.
EDIT 3 (Final SProc):
#user2338816 for all of the help. See his post for an explanation of the solution, but here is the final SProc for reference:
CREATE PROCEDURE DLLIB/G_DPIVOT# (
IN TABLE_NAME CHAR(12) CCSID 37 DEFAULT '' ,
IN PIVOT CHAR(12) CCSID 37 DEFAULT '' ,
IN PIVOTFLD CHAR(12) CCSID 37 DEFAULT '' ,
IN "VALUE" DECIMAL(10, 0) DEFAULT 0 ,
INOUT LIST CHAR(5000) CCSID 37 )
LANGUAGE SQL
SPECIFIC DLLIB/G_DPIVOT#
NOT DETERMINISTIC
READS SQL DATA
CALLED ON NULL INPUT
CONCURRENT ACCESS RESOLUTION DEFAULT
SET OPTION ALWBLK = *ALLREAD ,
ALWCPYDTA = *OPTIMIZE ,
COMMIT = *NONE ,
DECRESULT = (31, 31, 00) ,
DFTRDBCOL = *NONE ,
DYNDFTCOL = *NO ,
DYNUSRPRF = *USER ,
SRTSEQ = *HEX
BEGIN
DECLARE STMT1 VARCHAR ( 1000 ) ;
DECLARE C1 CURSOR FOR S1 ;
SET STMT1 = 'WITH DETAILS ( ' || TRIM ( PIVOT ) || ' , ' || TRIM ( PIVOTFLD ) || ' , CURR , PREV ) AS ( ' ||
'SELECT ' || TRIM ( PIVOT ) || ' ,' || TRIM ( PIVOTFLD ) || ',' ||
' ROW_NUMBER ( ) OVER ( PARTITION BY ' || TRIM ( PIVOT ) || ' ORDER BY ' || TRIM ( PIVOTFLD ) || ' ) AS CURR ,' ||
' ROW_NUMBER ( ) OVER ( PARTITION BY ' || TRIM ( PIVOT ) || ' ORDER BY ' || TRIM ( PIVOTFLD ) || ' ) - 1 AS PREV' ||
' FROM ' || TRIM ( TABLE_NAME ) ||
' WHERE ' || TRIM ( PIVOT ) || ' = ' || TRIM ( VALUE ) ||
' GROUP BY ' || TRIM ( PIVOT ) || ' , ' || TRIM ( PIVOTFLD ) || ' )' ||
' SELECT MAX ( TRIM ( L '','' FROM CAST ( SYS_CONNECT_BY_PATH ( ' || TRIM ( PIVOTFLD ) || ' , '','' ) AS CHAR ( 5000 ) ) ) ) ' ||
' FROM DETAILS ' ||
' START WITH CURR = 1 ' ||
' CONNECT BY NOCYCLE ' || TRIM ( PIVOT ) || ' = PRIOR ' || TRIM ( PIVOT ) || ' AND PREV = PRIOR CURR' ;
PREPARE S1 FROM STMT1 ;
OPEN C1 ;
FETCH C1 INTO LIST ;
CLOSE C1 ;
END ;
The basic problem is in the EXECUTE. You can't "execute" the prepared SELECT. Instead, you'll need to DECLARE CURSOR for S1 and FETCH rows from the CURSOR. Note that 'executing' a SELECT statement wouldn't actually do anything if it was allowed; it would just "SELECT", so EXECUTE doesn't make much sense. (A SELECT INTO statement can be different, but it's not clear if that's appropriate here.)
It might be possible to OPEN a CURSOR and return a result set rather than FETCHing rows. With more definition of how you actually want to use this, some elaboration should be possible.
Edit:
Second problem:
I've created more readable versions of your original CTE and the CTE in your edited question. The original:
WITH DETAILS ( PIVOT , PIVOTFLD , CURR , PREV ) AS (
SELECT PIVOT , PIVOTFLD ,
ROW_NUMBER ( ) OVER ( PARTITION BY PIVOT ORDER BY PIVOTFLD ) AS CURR ,
ROW_NUMBER ( ) OVER ( PARTITION BY PIVOT ORDER BY PIVOTFLD ) - 1 AS PREV
FROM TABLE_NAME
WHERE PIVOT = VALUE
GROUP BY PIVOT , PIVOTFLD )
VALUES( SELECT MAX ( CAST ( SYS_CONNECT_BY_PATH ( PIVOTFLD , ',' ) AS CHAR ( 5000 ) ) ) )
FROM DETAILS
START WITH CURR = 1
CONNECT BY NOCYCLE PIVOT = PRIOR PIVOT AND PREV = PRIOR CURR) INTO ? ;
You have a VALUE INTO statement after the CTE. AFAIK, that's not valid.
And your edited example:
WITH DETAILS ( PIVOT , PIVOTFLD , CURR , PREV ) AS (
SELECT PIVOT ,
PIVOTFLD ,
ROW_NUMBER ( ) OVER ( PARTITION BY PIVOT ORDER BY PIVOTFLD ) AS CURR ,
ROW_NUMBER ( ) OVER ( PARTITION BY PIVOT ORDER BY PIVOTFLD ) - 1 AS PREV
FROM TABLE_NAME
WHERE PIVOT = VALUE
GROUP BY PIVOT , PIVOTFLD );
Well, it's just a bare CTE that has no associated SELECT referencing it. You do try to PREPARE a SELECT statement later, but the two need to go together. You can't EXECUTE the CTE by itself.
Try putting them together as a single statement and see if a CURSOR creates over the result. Variable STMT1 would then look something like this:
WITH DETAILS ( PIVOT , PIVOTFLD , CURR , PREV ) AS (
SELECT PIVOT ,
PIVOTFLD ,
ROW_NUMBER ( ) OVER ( PARTITION BY PIVOT ORDER BY PIVOTFLD ) AS CURR ,
ROW_NUMBER ( ) OVER ( PARTITION BY PIVOT ORDER BY PIVOTFLD ) - 1 AS PREV
FROM TABLE_NAME
WHERE PIVOT = VALUE
GROUP BY PIVOT , PIVOTFLD )
SELECT * FROM DETAILS ;
Note that the statement includes the SELECT at the end. The WITH ... clause is followed by the SELECT ... in a single statement that is PREPAREd. The CURSOR would then be OPENed over that statement.
Edit 2:
I have modified a sample CTE that I've had for a while to fit into a stored proc and to return a value. It was compiled and run on my i 6.1 system. The CTE is PREPAREd from a string placed into a VARCHAR, then a CURSOR is opened over it. Rows are FETCHed in a WHILE-loop.
The CTE generates summary rows that are then UNIONed with detail rows from QIWS/QCUSTCDT. The summary is by STATE to provided a sub-total of BALDUE. The WHILE-loop is kind of meaningless; it only shows FETCHing and processing of rows. The only action is to count the number of rows that are not summary rows out of the CTE. This is essentially the same as the number of rows in the base table. The row count is returned in the rowCnt OUT parameter.
The source code is copy/pasted, but comes from two sources. First, the CREATE PROCEDURE statement is taken from iNavigator's 'Run SQL scripts' utility after generating the SQL from the compiled stored procedure. And second, the BEGIN ... END compound statement body is from the original I typed into the iNavigator New-> Procedure function. Although the two would have logical equivalence, I wanted to preserve the actual lines that were input. You can copy/paste the entire source into 'Run SQL Scripts' or go through the utility to create the procedure and only copy/paste the BEGIN ... END compound statement after entering values into the first two tabs of the New-> Procedure function.
I have a schema named SQLEXAMPLE that I build things like this into. You'll need to adjust the schema and procedure names to fit your environment. The QIWS/QCUSTCDT table should exist on nearly all AS/400-series systems.
CREATE PROCEDURE SQLEXAMPLE.CTE_CustCDT (
OUT rowCnt INTEGER )
LANGUAGE SQL
SPECIFIC SQLEXAMPLE.CTECUSTCDT
NOT DETERMINISTIC
READS SQL DATA
CALLED ON NULL INPUT
SET OPTION ALWBLK = *ALLREAD ,
ALWCPYDTA = *OPTIMIZE ,
COMMIT = *NONE ,
CLOSQLCSR = *ENDMOD ,
DECRESULT = (31, 31, 00) ,
DFTRDBCOL = *NONE ,
DYNDFTCOL = *NO ,
DYNUSRPRF = *USER ,
SRTSEQ = *HEX
BEGIN
DECLARE sumRows INTEGER DEFAULT 0 ;
DECLARE cusNum INTEGER ;
DECLARE lstNam CHAR(10) ;
DECLARE state CHAR(2) ;
DECLARE balDue DECIMAL(7, 2) ;
DECLARE stmt1 VARCHAR(512) ;
DECLARE at_end INT DEFAULT 0 ;
DECLARE not_found
CONDITION FOR '02000';
DECLARE c1 CURSOR FOR c1Stmt ;
DECLARE CONTINUE HANDLER FOR not_found
SET at_end = 1 ;
SET stmt1 = 'with t1 As(
SELECT 0 ,''Tot'' , state , sum( balDue )
FROM qiws.qcustcdt
GROUP BY state
ORDER BY state
)
select cusNum , lstNam , state, balDue
from qiws.qcustcdt
union
select *
from t1
order by state FOR FETCH ONLY' ;
PREPARE c1Stmt FROM stmt1 ;
OPEN c1 ;
FETCH C1 INTO cusNum , lstNam , state , balDue ;
WHILE at_end = 0 DO
IF cusNum <> 0 THEN SET sumRows = sumRows + 1 END IF ;
FETCH C1 INTO cusNum , lstNam , state , balDue ;
END WHILE ;
SET rowCnt = sumRows ;
CLOSE c1 ;
END
When the CTE is run by itself in STRSQL, the first few lines of output look like this:
....+....1....+....2....+....3....+....4....+....5....+....6....+....7....+.
CUSNUM LSTNAM STATE BALDUE
475,938 Doe CA 250.00
0 Tot CA 250.00
389,572 Stevens CO 58.75
0 Tot CO 58.75
938,485 Johnson GA 3,987.50
0 Tot GA 3,987.50
846,283 Alison MN 10.00
583,990 Abraham MN 500.00
0 Tot MN 510.00
The summary rows should easily be recognized. And when the stored proc is CALLed from 'Run SQL Scripts', the resulting output is:
Connected to relational database TISI on Tisi as Toml - 090829/Quser/Qzdasoinit
> call SQLEXAMPLE.CTE_CustCDT( 0 )
Return Code = 0
Output Parameter #1 = 12
Statement ran successfully (570 ms)
The QIWS/QCUSTCDT table on that system has 12 rows, and that matches the value returned.
It's not exactly the same as your desired CTE, but it should demonstrate that a dynamic CTE can be used. It also shows how FETCHes might pull rows from the CTE for whatever purpose is needed.