I have to fetch data from a running-time-defined table and get data based on a running-time-defined column, I'm now using dynamic sql with ref cursor as below. Is there any more efficient ways to improve the performance ?
PROCEDURE check_error(p_table_name IN VARCHAR2
,p_keyword IN VARCHAR2
,p_column_name IN VARCHAR2
,p_min_num IN NUMBER
,p_time_range IN NUMBER
,p_file_desc IN VARCHAR2
)
IS
type t_crs is ref cursor;
v_cur t_crs;
v_file_name VARCHAR2(100);
v_date_started DATE;
v_date_completed DATE;
v_counter NUMBER := 0;
v_sql VARCHAR2(500);
v_num NUMBER :=0;
BEGIN
v_sql := 'SELECT '||p_column_name||', DATE_STARTED,DATE_COMPLETED FROM '||p_table_name
|| ' WHERE '||p_column_name||' LIKE '''||p_keyword||'%'' AND DATE_STARTED > :TIME_LIMIT ORDER BY '||p_column_name;
OPEN v_cur FOR v_sql USING (sysdate - (p_time_range/1440));
LOOP
FETCH v_cur INTO v_file_name,v_date_started,v_date_completed;
EXIT WHEN v_cur%NOTFOUND;
IF v_date_started IS NOT NULL AND v_date_completed IS NULL
AND (sysdate - v_date_started)*1440 > p_time_range THEN
insert_record(co_alert_stuck,v_file_name,p_table_name,0,p_file_desc,p_time_range);
END IF;
END LOOP;
END;
BTW, will this make it better ?
v_sql := 'SELECT :COLUMN_NAME1, DATE_STARTED,DATE_COMPLETED FROM :TABLE WHERE :COLUMN_NAME2 LIKE :KEYWORD AND DATE_STARTED > :TIME_LIMIT ORDER BY :COLUMN_NAME3';
OPEN v_cur FOR v_sql USING p_column_name,p_table_name,p_column_name,p_keyword||'%',(sysdate - (p_time_range/1440)),p_column_name;
First, I'm not sure that I understand what the code is doing. In the code you posted (which you may have cut down to simplify things), the IF statement checks whether v_date_started IS NOT NULL which is redundant since there is a WHERE clause on DATE_STARTED. It checks whether (sysdate - v_date_started)*1440 > p_time_range which is just a redundant repetition of the WHERE clause on the DATE_STARTED column. And it checks whether v_date_completed IS NULL which would be more efficient as an additional WHERE clause in the dynamic SQL statement that you built. It would make sense to do all of your checks in exactly one place and the most efficient place to do them would be in the SQL statement.
Second, how many rows should this query return and where is the time being spent? If the cursor potentially returns many rows (for some definition of many), you'll get a bit of efficiency from doing a BULK COLLECT from the cursor into a collection and modifying the insert_record procedure to accept and process a collection. If the time is all spent executing the SQL statement and the query itself returns just a handful of rows, PL/SQL bulk operations would probably not make things appreciably more efficient. If the bottleneck is executing the SQL statement, you'd need to hope that an appropriate index existed on whatever table was passed in. If the bottleneck is the insert_record procedure, we'd need to know what that procedure is doing to comment.
Third, if the insert_record procedure is (at least primarily) just inserting the data that you fetched into a different table, it would be more efficient to get rid of all the looping and just generate a dynamic INSERT statement.
Fourth, with respect to your edit, you cannot use bind variables for table names or column names so the syntax you're proposing is invalid. It won't be more efficient because it will generate a bunch of syntax errors.
Related
I want to printout a Dynamic built query. Yet I am stuck at variable declaration; Error At line 2.
I need the maximum size for these VARCHAR2 variables.
Do I have a good overall structure ?
I use the result of the WITH inside the dynamic query.
DECLARE l_sql_query VARCHAR2(2000);
l_sql_queryFinal VARCHAR2(2000);
with cntp as (select distinct
cnt.code code_container,
*STUFF*
FROM container cnt
WHERE
cnt.status !='DESTROYED'
order by cnt.code)
BEGIN
FOR l_counter IN 2022..2032
LOOP
l_sql_query := l_sql_query || 'SELECT cntp.code_container *STUFF*
FROM cntp
GROUP BY cntp.code_container ,cntp.label_container, cntp.Plan_Classement, Years
HAVING
cntp.Years=' || l_counter ||'
AND
/*stuff*/ TO_DATE(''31/12/' || l_counter ||''',''DD/MM/YYYY'')
AND SUM(cntp.IsA)=0
AND SUM(cntp.IsB)=0
UNION
';
END LOOP;
END;
l_sql_queryFinal := SUBSTR(l_sql_query, 0, LENGTH (l_sql_query) – 5);
l_sql_queryFinal := l_sql_queryFinal||';'
dbms_output.put_line(l_sql_queryFinal);
The code you posted has quite a few issues, among them:
you've got the with (CTE) as a standlone fragment in the declare section, which isn't valid. If you want it to be part of the dynamic string then put it in the string;
your END; is in the wrong place;
you have – instead of -;
you remove the last 5 characters, but you end with a new line, so you need to remove 6 to include the U of the last UNION;
the line that appens a semicolon is itself missing one (though for dynamic SQL you usually don't want a semicolon, so the whole line can probably be removed);
2000 characters is too small for your example, but it's OK with the actual maximum of 32767.
DECLARE
l_sql_query VARCHAR2(32767);
l_sql_queryFinal VARCHAR2(32767);
BEGIN
-- initial SQL which just declares the CTE
l_sql_query := q'^
with cntp as (select distinct
cnt.code code_container,
*STUFF*
FROM container cnt
WHERE
cnt.status !='DESTROYED'
order by cnt.code)
^';
-- loop around each year...
FOR l_counter IN 2022..2032
LOOP
l_sql_query := l_sql_query || 'SELECT cntp.code_container *STUFF*
FROM cntp
GROUP BY cntp.code_container ,cntp.label_container, cntp.Plan_Classement, Years
HAVING
cntp.Years=' || l_counter ||'
AND
MAX(TO_DATE(cntp.DISPOSITION_DATE,''DD/MM/YYYY'')) BETWEEN TO_DATE(''01/01/'|| l_counter ||''',''DD/MM/YYYY'') AND TO_DATE(''31/12/' || l_counter ||''',''DD/MM/YYYY'')
AND SUM(cntp.IsA)=0
AND SUM(cntp.IsB)=0
UNION
';
END LOOP;
l_sql_queryFinal := SUBSTR(l_sql_query, 0, LENGTH (l_sql_query) - 6);
l_sql_queryFinal := l_sql_queryFinal||';';
dbms_output.put_line(l_sql_queryFinal);
END;
/
db<>fiddle
The q[^...^] in the first assignment is the alternative quoting mechanism, which means you don't have to escape (by doubling-up) the quotes within that string, around 'DESTYORED'. Notice the ^ delimiters do not appear in the final generated query.
Whether the generated query actually does what you want is another matter... The cntp.Years= part should probably be in a where clause, not having; and you might be able to simplify this to a single query instead of lots of unions, as you're already aggregating. All of that is a bit beyond the scope of your question though.
Is there a way to put the maximum size "automaticcaly" like "VARCHAR2(MAX_STRING_SIZE) does it work ?
No. And no.
The maximum size of varchar2 in PL/SQL is 32767. If you want to hedge against that changing at some point in the future you can declare a user-defined subtype in a shared package ...
create or replace package my_subtypes as
subtype max_string_size is varchar2(32767);
end my_subtypes;
/
... and reference that in your program...
DECLARE
l_sql_query my_subtypes.max_string_size;
l_sql_queryFinal my_subtypes.max_string_size;
...
So if Oracle subsequently raises the maximum permitted size of a VARCHAR2 in PL/SQL you need only change the definition of my_subtypes.max_string_size for the bounds to be raised wherever you used that subtype.
Alternatively, just use a CLOB. Oracle is pretty clever about treating a CLOB as a VARCHAR2 when its size is <= 32k.
To solve your other problem you need to treat the WITH clause as a string and assign it to your query variable.
l_sql_query my_subtypes.max_string_size := q'[
with cntp as (select distinct
cnt.code code_container,
*STUFF*
FROM container cnt
WHERE cnt.status !='DESTROYED'
order by cnt.code) ]';
Note the use of the special quote syntax q'[ ... ]' to avoid the need to escape the quotation marks in your query snippet.
A dynamic string query do not access a temp table ?
Dynamic SQL is a string containing a DML or DDL statement which we execute with EXECUTE IMMEDIATE or DBMS_SQL commands. Otherwise it is exactly the same as static SQL, it doesn't behave any differently. In fact the best way to write dynamic SQL is to start by writing the static statement in a worksheet, make it correct and then figure out which bits need to be dynamic (variables, placeholders) and which bits remain static (boilerplate). In your case the WITH clause is a static part of the statement.
As has already been pointed out, you seem to have misunderstood what a with clause is: it's a clause of a SQL statement, not a procedural declaration. My definition, it must be followed by select.
But also, as a general rule, I would recommend avoiding dynamic SQL when possible. In this case, if you can simulate a table with the range of years you want, you can join instead of having to run the same query multiple times.
The easy trick to doing that is to use Oracle's connect by syntax to use a recursive query to produce the expected number of rows.
Once you've done that, adding this table as a join pretty trivially:
WITH cntp AS
(
SELECT DISTINCT code code_container,
[additional columns]
FROM container
WHERE status !='DESTROYED') cntc,
(
SELECT to_date('01/01/'
|| (LEVEL+2019), 'dd/mm/yyyy') AS start_date,
to_date('31/12/'
|| (LEVEL+2019), 'dd/mm/yyyy') AS end_date,
(LEVEL+2019) AS year
FROM dual
CONNECT BY LEVEL <= 11) year_table
SELECT cntp.code_container,
[additional columns]
FROM cntp
join year_table
ON cntp.years = year_table.year
GROUP BY [additional columns],
years,
year_table.start_date,
year_table.end_date
HAVING max(to_date(cntp.disposition_date,''dd/mm/yyyy'')) BETWEEN year_table.start_date AND year_table.end_date
AND SUM(cntp.isa)=0
AND SUM(cntp.isb)=0
(This query is totally untested and may not actually fulfill your needs; I am providing my best approximation based on the information available.)
I've been mauling the following over-and-over, trying to maximize code re-usability, and at the same time, keep optimal performance.
I have a SQL query which is very long and complex, that I'm trying to run multiple times (6 times to be exact), each time with a different WHERE CLAUSE. It took a lot of effort to get it to the form in which I can interchange the WHERE CLAUSE... but I'm having lots of difficulty running the query for each case.
ie>
INSERT INTO table_x (SELECT *
FROM mst_q (+300 lines query)
WHERE complex_where_clause_1);
INSERT INTO table_x (SELECT *
FROM mst_q (+300 lines query)
WHERE complex_where_clause_2);
I've tried using UNION ALL, WHERE with CASE, WHERE with OR, and thought of even having 6 different CURSORS that would independently INSERT the results of each of the 6-cases. Having 6 different CURSORS works and performance is great, there is way too much duplicate code, since all that changes is the WHERE clause.
I thought, great, dynamic SQL! But the "long and complex" statement is much larger than the size of a VARCHAR2.
Is there a way to do something like the following? This is the bast option I can think of, the only issue is that v_sql will definitely be longer that the size limit of a VARCHAR2.
v_sql := 'INSERT INTO table_x (col1, col2) SELECT col1, col2 FROM WHERE ';
v_scen1 := 'complex_where_clause_1';
v_scen2 := 'complex_where_clause_2'
EXECUTE IMMEDIATE v_sql || v_scen1;
EXECUTE IMMEDIATE v_sql || v_scen2;
... ah, too simple, just use: v_sql CLOB;
I'll elaborate in a few hours; but I was able to do exactly what I was asking if possible... Using CLOB breaks down the size limitation of a VARCHAR2, and I simply append the WHERE clauses and execute sequentially...
You mentioned "I thought, great, dynamic SQL! But the "long and complex" statement is much larger than the size of a VARCHAR2."
You could do this if you are still interested in using dynamic SQL:
EXECUTE IMMEDIATE v_sql_stmt_part1 || v_sql_stmt_part2 || ....
That is to say, just break up your SQL statment into 2 (or more) VARCHAR2 variables and then concatenate them when you do the dynanmic SQL.
I have been tasked (as part of an assignment) to write a stored procedure in Oracle PL/SQL. There are 3 requirements that have to be met.
There must be 2 parameters, 1 IN and 1 OUT.
I must use an implicit cursor and SQL function to calculate a count of the numbers of fields of the same type (in this case the type is car models, so how many cars of each model are there).
I must use another implicit cursor to display the description of the models.
To be honest, I am at a loss. So far for the stored proc I have:
CREATE OR REPLACE Procedure model_details_sp
(p_model IN VARCHAR2,
p_noofcars OUT NUMBER)
IS
BEGIN
SELECT COUNT(Model_Name) INTO p_noofcars
FROM i_car
GROUP BY Model_Name;
END;
I really have no idea where to go from here. Any advice or direction would be most appreciated.
Many thanks.
Hi guys I appreciate all the comments. I wasn't very clear with the end requirements. I want to be able to call this procedure via an anonymous block so that the user will enter a model type (&vairalbe) and the procedure will display how many of that model types are in the database.
When dealing with this type of problems, first think about the data you're trying to capture.
Dealing with implicit cursors in PL/SQL require 1-row, so you need to make sure you understand the data.
In this case, you pass in a variable that you don't use in any of your queries, so I suggest you re-evaluate.
I don't have a database at hand to run this, but you should be able to work this out and hopefully get you a bit closer. I put it in an anonymous block so that I can write it really quick.
DECLARE
PROCEDURE model_details_sp (p_model IN VARCHAR2, p_noofcars OUT NUMBER)
IS
p_description VARCHAR2 (200);
BEGIN
--2
SELECT COUNT (model_name)
INTO p_noofcars
FROM i_car
WHERE model_name = p_model;
DBMS_OUTPUT.put_line ('No of Cars for model: ' || p_noofcars);
--3
SELECT model_description
INTO p_description
FROM i_car --the table should be the car_model table so that only one record is returned
WHERE model_name = p_model;
DBMS_OUTPUT.put_line ('Model Desc' || p_description);
END model_details_sp;
BEGIN
dbms_output.put_line('');
END;
To #David Aldridge comment:
Try running this--the result should be a failure--as you cannont select multiple rows using the into CLAUS, unless you aggregate the data:
DECLARE
p_num NUMBER;
BEGIN
SELECT LEVEL INTO p_num FROM DUAL CONNECT BY LEVEL <= 10;
dbms_output.put(p_num);
END;
The error you should see is this:
Error report:
ORA-01422: exact fetch returns more than requested number of rows
ORA-06512: at line 4
01422. 00000 - "exact fetch returns more than requested number of rows"
*Cause: The number specified in exact fetch is less than the rows returned.
*Action: Rewrite the query or change number of rows requested
From the description and subsequent comments, this is the solution I would provide:
DECLARE
PROCEDURE model_details_sp
(p_model IN VARCHAR2,
p_noofcars OUT NUMBER)
IS
BEGIN
SELECT COUNT(*)
INTO p_noofcars
FROM i_car
WHERE model_name = p_model;
END;
no_of_cars NUMBER := 0;
BEGIN
model_details_sp(:model_name, no_of_cars);
dbms_output.put_line('no of cars for ' || :model_name || ' = ' || no_of_cars);
END;
I've created the PROCEDURE in-line but you can just as easily extract this to the database by removing it from the declare section and executing it with CREATE OR REPLACE.
This example assumes use of an IDE that supports bind variable replacement (:model_name) on execute of the anonymous block. In TOAD, for example, the "user" will be prompted to provide a value for :model_name.
I have 2 procedures inside a package. I am calling one procedure to get a comma separated list of user ids.
I am storing the result in a VARCHAR variable. Now when I am using this comma separated list to put inside an IN clause in it is throwing "ORA-01722:INVALID NUMBER" exception.
This is how my variable looks like
l_userIds VARCHAR2(4000) := null;
This is where i am assigning the value
l_userIds := getUserIds(deptId); -- this returns a comma separated list
And my second query is like -
select * from users_Table where user_id in (l_userIds);
If I run this query I get INVALID NUMBER error.
Can someone help here.
Do you really need to return a comma-separated list? It would generally be much better to declare a collection type
CREATE TYPE num_table
AS TABLE OF NUMBER;
Declare a function that returns an instance of this collection
CREATE OR REPLACE FUNCTION get_nums
RETURN num_table
IS
l_nums num_table := num_table();
BEGIN
for i in 1 .. 10
loop
l_nums.extend;
l_nums(i) := i*2;
end loop;
END;
and then use that collection in your query
SELECT *
FROM users_table
WHERE user_id IN (SELECT * FROM TABLE( l_nums ));
It is possible to use dynamic SQL as well (which #Sebas demonstrates). The downside to that, however, is that every call to the procedure will generate a new SQL statement that needs to be parsed again before it is executed. It also puts pressure on the library cache which can cause Oracle to purge lots of other reusable SQL statements which can create lots of other performance problems.
You can search the list using like instead of in:
select *
from users_Table
where ','||l_userIds||',' like '%,'||cast(user_id as varchar2(255))||',%';
This has the virtue of simplicity (no additional functions or dynamic SQL). However, it does preclude the use of indexes on user_id. For a smallish table this shouldn't be a problem.
The problem is that oracle does not interprete the VARCHAR2 string you're passing as a sequence of numbers, it is just a string.
A solution is to make the whole query a string (VARCHAR2) and then execute it so the engine knows he has to translate the content:
DECLARE
TYPE T_UT IS TABLE OF users_Table%ROWTYPE;
aVar T_UT;
BEGIN
EXECUTE IMMEDIATE 'select * from users_Table where user_id in (' || l_userIds || ')' INTO aVar;
...
END;
A more complex but also elegant solution would be to split the string into a table TYPE and use it casted directly into the query. See what Tom thinks about it.
DO NOT USE THIS SOLUTION!
Firstly, I wanted to delete it, but I think, it might be informative for someone to see such a bad solution. Using dynamic SQL like this causes multiple execution plans creation - 1 execution plan per 1 set of data in IN clause, because there is no binding used and for the DB, every query is a different one (SGA gets filled with lots of very similar execution plans, every time the query is run with a different parameter, more memory is needlessly used in SGA).
Wanted to write another answer using Dynamic SQL more properly (with binding variables), but Justin Cave's answer is the best, anyway.
You might also wanna try REF CURSOR (haven't tried that exact code myself, might need some little tweaks):
DECLARE
deptId NUMBER := 2;
l_userIds VARCHAR2(2000) := getUserIds(deptId);
TYPE t_my_ref_cursor IS REF CURSOR;
c_cursor t_my_ref_cursor;
l_row users_Table%ROWTYPE;
l_query VARCHAR2(5000);
BEGIN
l_query := 'SELECT * FROM users_Table WHERE user_id IN ('|| l_userIds ||')';
OPEN c_cursor FOR l_query;
FETCH c_cursor INTO l_row;
WHILE c_cursor%FOUND
LOOP
-- do something with your row
FETCH c_cursor INTO l_row;
END LOOP;
END;
/
I'd like to create a Oracle package where I have a procedure that executes some dynamic SQL. This is no problem if I'm doing it all dynamic with EXECUTE IMMEDIATE but it would be better if the static parts of the query could be coded static (to have compile time checking).
Example of fully dynamic query:
-- v_stmt is built dynamically.
v_stmt := 'SELECT count(*) FROM <here some joins> WHERE <here some conditions>';
EXECUTE IMMEDIATE v_stmt
USING v_param1, v_param2
RETURNING INTO v_count;
Example of what I tried to make the FROM-part static:
-- v_stmt is built dynamically.
v_stmt := 'SELECT count(*) FROM my_package.my_function(:param1, :param2) WHERE <here some conditions>';
EXECUTE IMMEDIATE v_stmt
USING v_param1, v_param2
RETURNING INTO v_count;
FUNCTION my_function(
i_param1 IN VARCHAR2,
i_param2 IN NUMBER
)
RETURN SYS_REFCURSOR
AS
v_cursor SYS_REFCURSOR;
BEGIN
-- Open a cursor for different queries depending on params.
IF i_param2 = 1 THEN
OPEN v_cursor FOR <some static query>;
ELSE
OPEN v_cursor FOR <some other static query>;
END IF;
RETURN v_cursor;
END;
This doesn't work because it's not possible to select from a SYS_REFCURSOR (at least that's what I found with Google).
Is there any way to reach this goal?
edit: As requested, here are some examples:
Static queries:
SELECT a.*, ca.CUS_ID FROM adresses a INNER JOIN customer_adresses ca ON (ca.adr_id = a.adr_id);
SELECT p.*, cp.CUS_ID FROM persons p INNER JOIN customer_persons cp ON (cp.per_id = p.per_id);
Then they are extended dynamically like the following examples:
-- Checks if there is an adress in the customer where the zip is null.
SELECT count(*) FROM <static adresses query> q WHERE q.cus_id = :param1 AND a.zip IS NULL;
-- Checks if there is at least one person in the customer.
SELECT count(*) FROM <static persons query> q WHERE q.cus_id = :param1;
Sorry, but why the need to do this? Seems you're over complicating things by introducing a function that will return different types of data/tables depending on the parameter list. Very confusing imo. Besides, you have to do the work somewhere, you're just trying to hide it in this function (inside if param1=this then x if param1=that then y...)
Besides, even if you did implement a cursor function (even pipelined), it would be a bad idea in this case because you'd be forcing Oracle into doing work that it wouldn't necessarily need to do (ignore all the context switching for now). To just get a count, you'd have Oracle grab each an every row result and then count. Many times Oracle can just do a fast full index scan to get the count (depending on the query of course). And often same query run multiple times will not need to do all the work each time if blocks are found in buffer cache. I'd challenge you to run the count multiple times using straight SQL vs using a function returning a cursor. You might be surprised. And to my knowledge (check me on this) the new 11g function result cache won't work on a pipelined functions or a function returning a ref cursor (along with other issues like invalidations due to relies on tables).
So, what I'm saying is why not just do: select count(1) into v_variable from ...;
If you want to hide and modularize, then just know what you're potentially losing.
You may want to open a query in function1 and then pipeline the results of it as a table to function2 which then will add a where clause to this "table"
In this case you'll want to rewrite your function1 as a pipelined table function
v_stmt := 'SELECT count(*) FROM table(my_package.my_function(:param1, :param2)) WHERE <here some conditions>';
EXECUTE IMMEDIATE v_stmt
USING v_param1, v_param2
RETURNING INTO v_count;
CREATE TYPE object_row_type AS OBJECT (
OWNER VARCHAR2(30),
OBJECT_TYPE VARCHAR2(18),
OBJECT_NAME VARCHAR2(30),
STATUS VARCHAR2(7)
);
CREATE TYPE object_table_type AS TABLE OF object_row_type;
FUNCTION my_function(
i_param1 IN VARCHAR2,
i_param2 IN NUMBER
)
RETURN object_table_type PIPELINED AS
BEGIN
You can have compile time checking of expressions with Oracle expression filter.
It's probably more complicated than the other solutions, but if you really need to verify your conditions it can be helpful.