I have a procedure that has an in parameter and an out cursor. The results given by that cursor look like:
0100 | 0
0130 | 1
0200 | 2
0230 | 0
...
The first column is a static time code. The second column is an aggregation of how many times something is scheduled in that time slot on a given day.
That procedure is:
PROCEDURE DAILYLOAD (datep IN DATE, results OUT SYS_REFCURSOR)
AS
BEGIN
Open results for
SELECT RUN_TIME_C, COUNT (SCH_RPT_I)
FROM ITS_SCH_RPT_RUN_TIME
LEFT OUTER JOIN
ITS_SCH_RPT
ON ( RUN_TIME_C = RUN_TIME1_C
OR RUN_TIME_C = RUN_TIME2_C
OR RUN_TIME_C = RUN_TIME3_C)
WHERE EXP_DATE_D IS NULL
OR datep < exp_date_d AND datep > start_date_d AND SUSPENDED_USER='N'
AND ( ( (TO_CHAR (datep, 'D') = 1) AND RUN_SUNDAY_C = 'Y')
OR ( (TO_CHAR (datep, 'D') = 2) AND RUN_MONDAY_C = 'Y')
OR ( (TO_CHAR (datep, 'D') = 3) AND RUN_TUESDAY_C = 'Y')
OR ( (TO_CHAR (datep, 'D') = 4) AND RUN_WEDNESDAY_C = 'Y')
OR ( (TO_CHAR (datep, 'D') = 5) AND RUN_THURSDAY_C = 'Y')
OR ( (TO_CHAR (datep, 'D') = 6) AND RUN_FRIDAY_C = 'Y')
OR ( (TO_CHAR (datep, 'D') = 7) AND RUN_SATURDAY_C = 'Y'))
GROUP BY RUN_TIME_C
ORDER BY RUN_TIME_C;
END DAILYLOAD;
I want to call this procedure from a wrapping procedure several times with different parameters so that I can come up with weekly load and monthly load. Conceptually, this would be done by concatenating the individual result sets through something like union all and grouping that by the first column summing the second column for each grouping.
Right now, I have something like
Dailyload(datep, results1);
Dailyload(datep + 1, results2);
...
OPEN results FOR
SELECT run_time_c,
SUM(rpt_option_i)
FROM SELECT *
FROM results1
UNION ALL
SELECT *
FROM results2
UNION ALL ...
GROUP BY run_time_c
ORDER BY run_time_c
Is there a way I can do this in Oracle? Fetch with bulk collect looked promising, but I didn't see a good way to use it for my specific scenario.
You could do this as a union, including a column that identifies the Group. The individual selects would replicate more or less what your DailyLoad SP is doing.
select foo.Mygroup, sum(foo.col1)
from
(
select 'A' as MyGroup, col1 WHERE ...
union all
select 'B' as MyGroup, col1 WHERE ...
union all
select 'C' as MyGroup, col1 WHERE ...
) as Foo
group by MyGroup
If the number of groups is not known in advance, you could build a dynamic sql statement that conforms to this basic structure.
If the number of groups is so large that your dynamic statement would be too large, you could use a stored procedure that pushes the results from each call into a temp table along with a MyGroup column. Then you could issue your group by select statement against the temp table.
If the procedure's out parameter is a ref cursor, and you can't replicate what it's doing internally to make a nice single set-based query as OMG Ponies suggests, this previous answer may help. You can use an intermediate pipelined function to turn the sys_refcursor results into something you can treat as a table:
create package p as
type tmp_rec_type is record (run_time_c varchar2(4),
rpt_option_i number);
type tmp_table_type is table of tmp_rec_type;
procedure dailyload(p_date in date, p_results out sys_refcursor);
function func(p_date in date) return tmp_table_type pipelined;
procedure sumload(p_start_date in date, p_results out sys_refcursor);
end;
/
create package body p as
/* Your existing procedure, which may be elsewhere */
procedure dailyload(p_date in date, p_results out sys_refcursor) is
begin
open p_results for
select to_char(created, 'HH24MI') as run_time_c,
count(*) as rpt_option_i
from all_objects
where trunc(created) = trunc(p_date)
group by to_char(created, 'HH24MI');
end;
/* Intermediate pipelined function */
function func(p_date in date) return tmp_table_type pipelined is
tmp_cursor sys_refcursor;
tmp_rec tmp_rec_type;
begin
dailyload(p_date, tmp_cursor);
loop
fetch tmp_cursor into tmp_rec;
exit when tmp_cursor%notfound;
pipe row(tmp_rec);
end loop;
end;
/* Wrapper function to join the result sets together */
procedure sumload(p_start_date in date, p_results out sys_refcursor) is
begin
open p_results for
select run_time_c, sum(rpt_option_i) from (
select * from table(func(p_start_date))
union all
select * from table(func(p_start_date + 1))
union all
select * from table(func(p_start_date + 2))
)
group by run_time_c;
end;
end;
/
Guessing your data types, and picking data from a random table just as an example, of crouse. To call from SQL*Plus or SQL Developer:
var results refcursor;
exec p.sumload(to_date('01-Jun-11','DD-Mon-RR'), :results);
print :results
I haven't the time to test this, but I believe this will work:
Modify your sproc to make the SYS_REFCURSOR an IN OUT parameter, rather than just an OUT.
Set your parameters in for/each loop (whatever language you are working in...)
In the loop pass in the reference to the same SYS_REFCURSOR.
Inside the sproc create a local SYS_REFCURSOR variable to select into as you currently do.
Inside the sproc merge the local and the parameter SYS_REFCURSOR
This should build your result set.
if you don't want to test this, I may build a test case for this in C#/Oracle 10g over the weekend in order to test my hypothesis.
Another option, if you are on 11g, would be a Pipelined Query as discussed How to create Oracle stored procedure which can return specific entities as well all entity (look to #tbone's answer and the link he provides...)
You can use oracle global temporary table to accumulate and further process the data.
It is in-memory structure and has very little overhead.
Related
I have a package which contains a procedure in which I need to query a list of id numbers, which are varchar2. I will have to query this same list multiple times, and I'd rather not have redo the query. Ideally I'd like to make a function in my package that would return the list of id numbers. So I could load the array of id numbers into a variable, and then use that variable as a table throughout my procedure. I've been googling like crazy and it just doesn't seem like it's possible. Is there some way to do this?
Note: I'm not able to create a new type at the schema level, and it won't let me do this with a local collection type.
Also, I would prefer not to use dynamic sql; the main query in my procedure is enormous, and I don't want to deal with a string of that size.
I want to do something like this:
id_number_list array_type := my_function();
select *
from my_table mt
left join table(id_number_list) idl on mt.id_number = idl.column_value;
EDIT: Thanks for your help so far! MTO's answer works for a select statement join, like I described above. However, I also need to delete from the table where the id_number is in the list. This gives me an "invalid data type" error. What could explain this? The data type should always be the same: varchar2(10).
Here I create the type at package level:
type string_list is table of varchar2(10);
Then I create a function that returns the list of id numbers (for our purposes, the "action" is always c_action_refresh, so the if statement is true):
function get_modified_ids(scope in smallint, action in smallint) return string_list is
modified_ids string_list := string_list();
last_refreshed date;
begin
last_refreshed := get_last_refreshed_date(scope,action);
if action = c_action_refresh then
select id_number
bulk collect into modified_ids
from(
select id_number
from adv.hr_giving cg
join adv.pbi_dates d
on d.DATE_FULL = trunc(cg.processed_date)
where d.RELATIVE_DATE >= last_refreshed
and d.RELATIVE_DATE <= trunc(CURRENT_DATE)
and cg.fiscal_year >= adv.current_fiscal_year - 6
union
select gi.gift_donor_id as id_number
from adv.gift gi
where gi.date_added >= last_refreshed
or gi.date_modified >= last_refreshed
union
select p.pledge_donor_id as id_number
from adv.pledge_rev p
where p.date_added >= last_refreshed
or p.date_modified >= last_refreshed
union
select a.id_number
from adv.affiliation a
where a.date_added >= last_refreshed
or a.date_modified >= last_refreshed
);
end if;
return(modified_ids);
end get_modified_ids;
Then, in my procedure, I initialize a variable by calling the function:
modified_ids string_list := get_modified_ids(scope,action);
Then I try to use the list in a delete statement:
delete from advrpt.pbi_gvg_profile_ag p
where p.id_number in
(select column_value from table(modified_ids));
This gives the error ORA-00902: invalid datatype. The type of id_number is varchar2(10). And again, it works fine in a join in a select statement.
So why am I getting this error?
Don't use a variable in the package (as there would only be a single variable and if your procedure is called twice in short succession then the second set of values would overwrite the first and potentially cause issues if that happened mid-way through processing the first invocation).
Instead, create a user-defined collection type:
CREATE TYPE number_list IS TABLE OF NUMBER;
And pass a collection as an argument to the procedure:
CREATE PROCEDURE your_procedure (
i_numbers IN number_list,
o_cursor1 OUT SYS_REFCURSOR,
o_cursor2 OUT SYS_REFCURSOR
)
IS
BEGIN
OPEN o_cursor1 FOR
SELECT *
FROM your_table
WHERE id MEMBER OF i_numbers;
OPEN o_cursor2 FOR
SELECT y.*
FROM your_table y
INNER JOIN TABLE(i_numbers) n
ON (y.id = n.COLUMN_VALUE);
END;
/
Then call it using, for example:
DECLARE
v_cur1 SYS_REFCURSOR;
v_cur2 SYS_REFCURSOR;
v_id your_table.id%TYPE;
v_value your_table.value%TYPE;
BEGIN
your_procedure(number_list(1,5,13), v_cur1, v_cur2);
LOOP
FETCH v_cur1 INTO v_id, v_value;
EXIT WHEN v_cur1%NOTFOUND;
DBMS_OUTPUT.PUT_LINE(v_id || ', ' || v_value);
END LOOP;
END;
/
Or create the type as part of your package:
CREATE PACKAGE your_package AS
TYPE number_list IS TABLE OF NUMBER;
PROCEDURE your_procedure (
i_numbers IN number_list,
o_cursor1 OUT SYS_REFCURSOR,
o_cursor2 OUT SYS_REFCURSOR
);
END;
/
Then create the package body:
CREATE PACKAGE BODY your_package AS
PROCEDURE your_procedure (
i_numbers IN number_list,
o_cursor1 OUT SYS_REFCURSOR,
o_cursor2 OUT SYS_REFCURSOR
)
IS
BEGIN
OPEN o_cursor1 FOR
SELECT *
FROM your_table
WHERE id IN (SELECT COLUMN_VALUE FROM TABLE(i_numbers));
OPEN o_cursor2 FOR
SELECT y.*
FROM your_table y
INNER JOIN TABLE(i_numbers) n
ON (y.id = n.COLUMN_VALUE);
END;
END;
/
Note: The MEMBER OF operator only works with collections defined in the SQL scope and not collections defined locally in a PL/SQL scope.
Then call it using, for example:
DECLARE
v_cur1 SYS_REFCURSOR;
v_cur2 SYS_REFCURSOR;
v_id your_table.id%TYPE;
v_value your_table.value%TYPE;
BEGIN
your_package.your_procedure(your_package.number_list(1,5,13), v_cur1, v_cur2);
LOOP
FETCH v_cur1 INTO v_id, v_value;
EXIT WHEN v_cur1%NOTFOUND;
DBMS_OUTPUT.PUT_LINE(v_id || ', ' || v_value);
END LOOP;
END;
/
fiddle
To me, it looks as if you have it all. If not, here's an example.
Function:
SQL> create or replace function f_test
2 return sys.odcinumberlist
3 is
4 begin
5 return sys.odcinumberlist(10, 20);
6 end;
7 /
Function created.
How to use it?
SQL> set serveroutput on
SQL> declare
2 id_number_list sys.odcinumberlist := f_test;
3 begin
4 for cur_r in (select e.deptno, e.ename
5 from emp e join table(id_number_list) idl on idl.column_value = e.deptno
6 order by 1, 2
7 )
8 loop
9 dbms_output.put_line(cur_r.deptno ||' '|| cur_r.ename);
10 end loop;
11 end;
12 /
10 CLARK
10 KING
10 MILLER
20 ADAMS
20 FORD
20 JONES
20 SCOTT
20 SMITH
PL/SQL procedure successfully completed.
SQL>
Current table1:
col1
-------------
schema.table1
schema.table2
schema.table3
Desired table1:
col1 col2
------------------------------------------------------------
schema.table1 value of (select count(*) from schema.table1)
schema.table2 value of (select count(*) from schema.table1)
schema.table3 value of (select count(*) from schema.table1)
It is not working, I tried using function too, but function doesn't allow to use 'FROM'
select col1, (select count(*) from col1)
from table1
I am trying to create this query in redshift. Can anyone please help me out?
To perform this task you will need a stored procedure AND a defined cursor. The stored procedure allows for looping and the cursor provides the ability to execute a newly created statement (dynamic querying).
For example:
Create the starting materials, 3 tables and a table that references these tables.
create table foo as (select 1 as A);
create table goo as (select 2 as A);
create table hoo as (select 3 as A);
create table tabs as (select 'foo' as tab union all select 'goo' union all select 'hoo');
Next define the stored procedure the will create the dynamic SQL
CREATE OR REPLACE procedure count_tabs(curs1 INOUT refcursor)
AS
$$
DECLARE
row record;
statement varchar := '';
union_needed BOOL := false;
BEGIN
for row in select tab from tabs LOOP
IF union_needed THEN
statement := statement || ' UNION ALL ';
END IF;
statement := statement || 'select \'' || row.tab || '\' as table_name, count(*) as table_count from ' || row.tab ;
union_needed := true;
END LOOP;
RAISE NOTICE 'sql to execute: %',statement;
open curs1 for execute statement;
END;
$$ LANGUAGE plpgsql;
Lastly we need to call the procedure and execute the cursor
call count_tabs('mycursor');
fetch 1000 from mycursor;
A few notes on this:
This assumes you want the results as output on your bench. If you want to create a table with the results this is doable in the same structure
Since the FROM clause value(s) is unknown at compile time this needs to be done in 2 steps - create the query and then execute the query.
I believe you can have the procedure walk this same cursor itself but doing this is exceptionally slow
I have a table "test_calculate" this has a column "CONN_BY" having values
column can have more than 2 number to multiply and this table may contain millions of rows , I need to get the result of the calculation from "CONN_BY" to "MVP".
I have used xmlquery for the calculation and dynamic query but these are quite slow. Is there another way which is much faster .Please suggest.
You can try the dynamic query.
Create a function which returns the calculated value and use it in your insert or select queries.
CREATE OR REPLACE FUNCTION UFN_CALCULATE (CLM_VALUE VARCHAR2)
RETURN NUMBER IS
RES_VAL NUMBER;
BEGIN
EXECUTE IMMEDIATE 'select '||CLM_VALUE||' FROM DUAL' INTO RES_VAL;
RETURN RES_VAL;
END;
You can use that function like below.
SELECT UFN_CALCULATE('.0876543 * .09876') FROM DUAL;
SELECT UFN_CALCULATE(CONN_BY) FROM YOUR_TABLE;
One option is using select ... connect by level <= regexp_count(conn_by,'[^*]+')... query for the implicit cursor within a PL/SQL code block
SQL> set serveroutput on
SQL> declare
mvp owa.nc_arr; -- numeric array to initialize each multiplication to 1 for each id value
begin
dbms_output.put_line('ID MVP');
dbms_output.put_line('--------');
for c in
(
select id,
to_number( regexp_substr(conn_by,'[^*]+',1,level) ) as nr,
level as lvl , max( level ) over ( partition by id ) as mx_lvl
from test_calculate
connect by level <= regexp_count(conn_by,'[^*]+')
and prior sys_guid() is not null
and prior conn_by = conn_by
order by id, lvl
)
loop
if c.lvl = 1 then mvp(c.id) := 1; end if;
mvp(c.id) := c.nr * mvp(c.id);
if c.lvl = c.mx_lvl then
dbms_output.put_line(c.id||' '||mvp(c.id));
end if;
end loop;
end;
/
where test_calculate is assumed to have an identity column(id)
Demo
I'am trying UNION ALL many tables into a new table.The columns of the old tables are the same, but the order of the columns is different, so the below SQL statement will get wrong result:
CREATE TABLE sum_7_2018_xia_weijian
AS
(
SELECT * FROM huiwen
UNION
SELECT * FROM penglai
UNION
SELECT * FROM baoluo
UNION
SELECT * FROM dongge
UNION
SELECT * FROM resultdonglu
UNION
SELECT * FROM resultwencheng
UNION
SELECT * FROM tan_illeg
);
I finally corrected it, but the SQL statements is too redundant:
step 1. get column names of one of the old tables named huiwen
SELECT string_agg(column_name, ',')
FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = 'huiwen';
results:
> string_agg
> ----------------------------------------------------------------------
>
> gid,id,geom,sxm,sxdm,sxxzqdm,xzqhdm,xzmc,sfzgjsyd,sfkfbj,sfjbnt,sfld,sflyhx,sfhyhx
step 2. union tables as a new table. I copy the string_agg of table huiwen to each SELECT-UNION to keep the order of columns, this is clumsy.
CREATE TABLE sum_2018_xia_weijian
AS
(
SELECT gid,id,geom,sxm,sxdm,sxxzqdm,xzqhdm,xzmc,sfzgjsyd,sfkfbj,sfjbnt,sfld,sflyhx,sfhyhx
FROM huiwen
UNION ALL
SELECT gid,id,geom,sxm,sxdm,sxxzqdm,xzqhdm,xzmc,sfzgjsyd,sfkfbj,sfjbnt,sfld,sflyhx,sfhyhx
FROM penglai
UNION ALL
SELECT gid,id,geom,sxm,sxdm,sxxzqdm,xzqhdm,xzmc,sfzgjsyd,sfkfbj,sfjbnt,sfld,sflyhx,sfhyhx
FROM baoluo
);
results:
> Query returned successfully: 2206 rows affected, 133 msec execution time.
I tried to do some optimization by pl/pgSQL using Declarations of variable to handle column names, but failed to find any SQL data type can handle this. Using of RECORD result Pseudo-Types ERROR：
CREATE or replace FUNCTION ct() RETURNS RECORD AS $$
DECLARE
clms RECORD;
BEGIN
SELECT column_name INTO clms
FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = 'huiwen';
RETURN clms;
END;
$$ LANGUAGE plpgsql;
CREATE TABLE sum_2018_xia_weijian
AS
(
SELECT ct() FROM huiwen
UNION ALL
SELECT ct() FROM penglai
UNION ALL
SELECT ct() FROM baoluo
UNION ALL
SELECT ct() FROM dongge
UNION ALL
SELECT ct() FROM resultdonglu
UNION ALL
SELECT ct() FROM resultwencheng
UNION ALL
SELECT ct() FROM tan_illeg
);
You may use STRING_AGG twice for getting the UNION ALL. You can get all the columns in specific order by explicitly ordering it by column_name in the string_agg.
Here's a generic function which takes an array of tables and a final table name.
CREATE or replace FUNCTION fn_create_tab(tname_arr TEXT[], p_tab_name TEXT)
RETURNS VOID AS $$
DECLARE
l_select TEXT;
BEGIN
select STRING_AGG(query,' UNION ALL ' ) INTO l_select
FROM
(
SELECT 'select ' || string_agg( column_name,','
ORDER BY column_name ) || ' from ' || table_name as query
FROM information_schema.columns
WHERE table_schema = 'public' AND table_name = ANY (tname_arr)
GROUP BY table_name
) s;
IF l_select IS NOT NULL
THEN
EXECUTE format ('DROP TABLE IF EXISTS %I',p_tab_name);
EXECUTE format ('create table %I AS %s',p_tab_name,l_select);
END IF;
END;
$$ LANGUAGE plpgsql;
Now, run the function like this:
select fn_create_tab(ARRAY['huiwen','penglai'],'sum_2018_xia_weijian');
Instead of making the programming block complex you can follow some below concepts from the documentation of Union or Union All as it says :
The number of columns in all queries must be the same.
The corresponding columns must have the compatible data type.
The column names of the first query determine the column names of the combined result set.
The GROUP BY and HAVING clauses are applied to each individual query, not the final result set.
The ORDER BY clause is applied to the combined result set, not within the individual result set.
By following the 3rd point make your Union query adjusted to refer to the table whose column order is expected in the result.
I am stuck at a place.
There is a procedure that checks for something and inserts into an table type upon successful determination of that condition.
But i can insert only once in the table type. Is there a way to insert again and again into the table type.
PROCEDURE "hello"."helloWorld.db::sampleException" (OUT TRACE_RECORD "hello"."LogTrace" )
LANGUAGE SQLSCRIPT AS
BEGIN
DECLARE i int;
select count(*) into i from "hello"."REGION";
IF :i > 1 then
TRACE_RECORD = SELECT '1' AS "LogID", '1' AS "TraceID" FROM DUMMY;
end if;
IF :i > 2 then
TRACE_RECORD = SELECT '2' AS "LogID", '2' AS "TraceID" FROM DUMMY;
end if;
END;
What i get on executing the procedure is only the last record "2,2".
How can i insert both the records 1,1 and 2,2.
Note: I do not want to use Temporary Tables.
Any help on this..
Thanks.!
Editing the Question a bit:
-I have to use Table TYPE (till the time there is no optimal way better than it)
-I have to insert more than 20-30 records in the table type.
Do you have to write this as a procedure? A table-valued function seems more suitable:
CREATE FUNCTION f_tables4 (in_id INTEGER)
RETURNS TABLE (
"LogID" VARCHAR(400),
"TraceID" VARCHAR(400)
)
LANGUAGE SQLSCRIPT
AS
BEGIN
RETURN
SELECT t."LogID", t."TraceID"
FROM (
SELECT 1 AS i, '1' AS "LogID", '1' AS "TraceID" FROM DUMMY
UNION ALL
SELECT 2 AS i, '2' AS "LogID", '2' AS "TraceID" FROM DUMMY
) t
JOIN (SELECT count(*) AS cnt FROM "hello"."REGION") c
ON c.cnt > t.i
END