I am trying to create a function that dynamically runs a query based on inputs. The first input for the function, input_id, is the argument for the dynamic query. The second input, IN_QUERY_ID, specifies which query to use.
create or replace
FUNCTION getResultID(
INPUT_ID NUMBER,
IN_QUERY_ID NUMBER
)
RETURN VARCHAR2
AS
RESULT_ID VARCHAR2(256);
query_str VARCHAR2(256);
BEGIN
select CONSTRUCTOR INTO query_str from query_str_ref
where QUERY_ID=IN_QUERY_ID;
EXECUTE IMMEDIATE query_str INTO RESULT_ID USING INPUT_ID;
RETURN Result_ID;
END getResultID;
I'm getting an error that I'm not properly ending the statement after "RESULT_ID=IN_QUERY_ID;" I'm wondering if I'm missing some other step.
You haven't declared Result_ID as a variable in the function.
The good news is that it's not your function that's wrong. According to the dbms_output that #sebas encouraged you to produce, the string you're trying to execute dynamically is:
select FIRST_NAME||LAST_NAME||to_char(BIRTH_DATE,'yyyy/mm/dd') as HOST_ID FROM INPUT_DATA_TABLE WHERE INPUT_ID=NEW:INPUT_ID;
There are two thing wrong with that. The NEW:INPUT_ID is causing the ORA-00933, because the NEW looks spurious; if you remove that it will recognise the :INPUT_ID as a bind variable. (NEW looks like it's come from a trigger but is probably a coincidence). And you should not have a trailing ; on the string, execute doesn't need it and it will break with an invalid character error.
So it should work if the query_str_ref entry is changed to:
select FIRST_NAME||LAST_NAME||to_char(BIRTH_DATE,'yyyy/mm/dd') as HOST_ID FROM INPUT_DATA_TABLE WHERE INPUT_ID=:INPUT_ID
Related
I have tables that contain the same type of data for every year, but the data gathered varies slightly in that they may not have the same fields.
d_abc_2016
d_def_2016
d_ghi_2016
d_jkl_2016
There are certain constants for each table: company_id, employee_id, salary.
However, each one might or might not have these fields that are used to calculate total incentives: bonus, commission, cash_incentives. There are a lot more, but just using these as a examples. All numeric
I should note at this point, users only have the ability to run SELECT statements.
What I would like to be able to do is this:
Give the user the ability to call in SELECT and specify their own fields in addition to the call
Pass the table name being used into the function to use in conditional logic to determine how the query string should be constructed for the eventual total_incentives calculation in addition to passing the whole table so a ton of arguments don't have to be passed into the function
Basically this:
SELECT employee_id, salary, total_incentives(t, 'd_abc_2016')
FROM d_abc_2016 t;
So the function being called will calculate total_incentives which is numeric for that employee_id and also show their salary. But the user might choose to add other fields to look at.
For the function, because the fields used in the total_incentives function will vary from table to table, I need to create logic to construct the query string dynamically.
CREATE OR REPLACE FUNCTION total_incentives(ANYELEMENT, t text)
RETURNS numeric AS
$$
DECLARE
-- table name lower case in case user typed wrong
tbl varchar(255) := lower($2;
-- parse out the table code to use in conditional logic
tbl_code varchar(255) := split_part(survey, '_', 2);
-- the starting point if the query string
base_calc varchar(255) := 'salary + '
-- query string
query_string varchar(255);
-- have to declare this to put computation INTO
total_incentives_calc numeric;
BEGIN
IF tbl_code = 'abc' THEN
query_string := base_calc || 'bonus';
ELSIF tbl_code = 'def' THEN
query_string := base_calc || 'bonus + commission';
ELSIF tbl_code = 'ghi' THEN
-- etc...
END IF;
EXECUTE format('SELECT $1 FROM %I', tbl)
INTO total_incentives_calc
USING query_string;
RETURN total_incentives_calc;
END;
$$
LANGUAGE plpgsql;
This results in an:
ERROR: invalid input syntax for type numeric: "salary + bonus"
CONTEXT: PL/pgSQL function total_incentives(anyelement,text) line 16 at EXECUTE
Since it should be returning a set of numeric values. Change it to the following:
CREATE OR REPLACE FUNCTION total_incentives(ANYELEMENT, t text)
RETURNS SETOF numeric AS
$$
...
RETURN;
Get the same error.
Figure well, maybe it is a table it is trying to return.
CREATE OR REPLACE FUNCTION total_incentives(ANYELEMENT, t text)
RETURNS TABLE(tot_inc numeric) AS
$$
...
Get the same error.
Really, any variation produces that result. So really not sure how to get this to work.
Look at RESULT QUERY, RESULT NEXT, or RESULT QUERY EXECUTE.
https://www.postgresql.org/docs/9.6/static/plpgsql-control-structures.html
RESULT QUERY won't work because it takes a hard coded query from what I can tell, which won't take in variables.
RESULT NEXT iterates through each record, which I don't think will be suitable for my needs and seems like it will be really slow... and it takes a hard coded query from what I can tell.
RESULT QUERY EXECUTE sounds promising.
-- EXECUTE format('SELECT $1 FROM %I', tbl)
-- INTO total_incentives_calc
-- USING query_string;
RETURN QUERY
EXECUTE format('SELECT $1 FROM %I', tbl)
USING query_string;
And get:
ERROR: structure of query does not match function result type
DETAIL: Returned type character varying does not match expected type numeric in column 1.
CONTEXT: PL/pgSQL function total_incentives(anyelement,text) line 20 at RETURN QUERY
It should be returning numeric.
Lastly, I can get this to work, but it won't be DRY. I'd rather not make a bunch of separate functions for each table with duplicative code. Most of the working examples I have seen have the whole query in the function and are called like such:
SELECT total_incentives(d_abc_2016, 'd_abc_2016');
So any additional columns would have to be specified in the function as:
EXECUTE format('SELECT employee_id...)
Given the users will only be able to run SELECT in query this really isn't an option. They need to specify any additional columns they want to see inside a query.
I've posted a similar question but was told it was unclear, so hopefully this lengthier version will more clearly explain what I am trying to do.
The column names and tables names should not be used as query parameters passed by USING clause.
Probably lines:
RETURN QUERY
EXECUTE format('SELECT $1 FROM %I', tbl)
USING query_string;
should be:
RETURN QUERY
EXECUTE format('SELECT %s FROM %I', query_string, tbl);
This case is example why too DRY principle is sometimes problematic. If you write it directly, then your code will be simpler, cleaner and probably shorter.
Dynamic SQL is one from last solution - not first. Use dynamic SQL only when your code will be significantly shorter with dynamic sql than without dynamic SQL.
I've just started with PL/SQL. My concern is as follows:
I want to create a string dynamically.
I tried the following, but it always results in ORA-00900 & ORA-06512 at the line of "Execute Immediate...".
Here we go:
Declare
l_pre_sql varchar2(4000) := 'schema.';
l_sql varchar2(4000) := 'concat(concat(:a, :b), :c)';
l_after_sql := '.procedure(a,b,c)';
begin
execute immediate l_sql using l_pre_sql, l_sql, l_after_sql;
end;
Is the syntax of execute immediate wrong? Are there any other working possibilities? As you may see, i am working around the problem that one cannot use schema name as a dynamic variable.
For clarification I basically want to do this:
execute immediate ':a'||'.'||':b'||'.procedure(a,b,c)' using schema, name;
Thanks in advance!
In prepared statements (in Oracle and other languages), you can replace constant values in the query string using parameters. However, you cannot replace column names, table names, users (schemas), procedure names, and so on.
In other words, the substitution is not just by replacing values with string representations. It is plugging parameters into a compiled statement.
So, you need to construct the string with the procedure names first and then call it.
I think what you want is something like:
execute immediate l_pre_sql || l_after_sql || '(:a, :b, :c)' using . . .
I just tripped over an answer to a problem I was having with a PL/SQL variable not being recognized by a function and I was hoping someone could explain to me why my solution worked and what is happening "underneath the hood".
Background
As part of an optimization project, I am trying to collect metrics on individual SQL scripts within a Stored Procedure. The Stored Proc that I am dissecting has an In-type date parameter that I need to define in order to run each individual SQL Script:
CREATE OR REPLACE myStoredProc (DATE_IN DATE, ERROR_OUT OUT VARCHAR2)
IS
BEGIN
--Truncate Temp Tables
--6 Individual SQL Scripts
EXCEPTION
--Error Handling
END;
To run each script individually, I decided to just drop each SQL statement into a PL/SQL block and feed the DATE_IN parameter in as a variable:
DECLARE
DATE_IN DATE := TO_DATE('16-JUL-2014','DD-MON-RR');
BEGIN
--Place individual script here
END;
The Problem
This approach worked fine for a couple of the queries that referred to this DATE_IN variable but one query with a reference to an outside function which takes DATE_IN as a parameter started throwing an ORA-00904 error:
DECLARE
DATE_IN DATE := TO_DATE('16-JUL-2014','DD-MON-RR');
BEGIN
insert into temp_table
SELECT table1.field1,
table1.field2,
table2.fieldA,
MyFunction(table1.field1, DATE_IN) --This was the problem line
FROM
table1,
table2
WHERE EXISTS (inner query)
AND table1.keys = table2.keys
AND table2.date <= DATE_IN
END;
Solution
At the advice of another Developer, I was able to get around this error by adding a colon (:) in front of the DATE_IN variable that I was passing into the function so that the problem line read MyFunction(table1.field1, :DATE_IN). As soon as I did that, my error disappeared and I was able to run the query without issue.
I was happy for the result but the other Developer wasn't able to explain why it was needed, only that it was necessary to call any functions or other stored procs from a PL/SQL statement. I assume this has something to do with scope but I would like to get a better explanation as to why this colon was necessary for the function to see the variable.
Questions
I've tried to do a little research looking over Oracle documentation on parameters, variables, binding/declaring and constants but my research has only given me more questions:
After reading up on variables, I now question if that is the correct term for what I have been using (since I didn't actually use the VARIABLE command and I'm passing in a date - which is not an allowable data type). If my DATE_IN DATE := statement is not a variable, then what is it?
Why were the rest of my references to DATE_IN recognized by the compiler but passing the value to the function was out of scope?
What exactly is the colon (:) doing here? Is this turning that into a bind variable?
Thanks in advance. I appreciate any guidance you can provide!
----------------------------------EDIT--------------------------------------
I was asked to provide additional information. My Db version is 11G, 11.2.0.2.0. The query that I was able to reproduce this error is below.
DECLARE
EXTRACT_DT_IN DATE := TO_DATE('16-JUL-2014','DD-MON-RR');
BEGIN
--This begins the pre-optimized query that I'm testing
insert into AELI_COV_TMP_2_OPT
SELECT /*+ ordered use_nl(CM MAMT) INDEX (CM CSMB_CSMB2_UK) INDEX (MAMT (MBAM_CSMB_FK_I) */
CM.CASE_MBR_KEY
,CM.pyrl_no
,MAMT.AMT
,MAMT.FREQ_CD
,MAMT.HOURS
,aeli$cov_pdtodt(CM.CASE_MBR_KEY, EXTRACT_DT_IN)
FROM
CASE_MEMBERS CM
,MEMBER_AMOUNTS MAMT
WHERE EXISTS (select /*+ INDEX(SDEF SLRY_BCAT_FK_I) */
'x'
from SALARY_DEF SDEF
where SDEF.CASE_KEY = CM.CASE_KEY
AND SDEF.TYP_CD = '04'
AND SDEF.SLRY_KEY = MAMT.SLRY_KEY)
AND CM.CASE_MBR_KEY = MAMT.CASE_MBR_KEY
AND MAMT.STAT_CD = '00'
AND (MAMT.xpir_dt is null or MAMT.xpir_dt > EXTRACT_DT_IN)
AND MAMT.eff_dt <= EXTRACT_DT_IN;
--This ends the pre-optimized query that I'm testing
END;
Here is the error I'm encountering when trying to run an Explain Plan on this statement. I am able to get past this error if I remove reference to line 13 or I add a colon (:) to the EXTRACT_DT_IN on that line.
----------------------EDIT 2-------------------
Here is the function signature of aeli$.cov_pdtodt. (I've replaced the owner for security reasons).
CREATE OR REPLACE function __owner__.aeli$cov_pdtodt
(CASE_MBR_KEY_IN IN NUMBER, EXTRACT_EFF_DT_IN DATE)
RETURN DATE IS
PDTODT DATE;
Your anonymous block is fine as it is, as long as you execute the whole block. If you try to execute just the insert, or its select, as a standalone command then it will indeed fail with ORA-00904.
That isn't quite a scope problem, it's a context problem. You're trying to refer to a PL/SQL variable in an SQL context, and that is never going to work.
In a PL/SQL context this would work:
declare
some_var dual.dummy%type := 'X';
begin
insert into some_table
select dummy from dual where dummy = some_var;
end;
/
... because the insert has access to the PL/SQL some_var.
In an SQL context this will error:
select * from dual where dummy = some_var;
... because it's looking for a column called SOME_VAR, and there isn't one.
If you do this instead:
select * from dual where dummy = :some_var;
... the some_var is now a client-managed bind variable. If you execute that you'll either be prompted for the bind value, or given a not-all-variables-bound error, or bind-variable-not-declared, or similar, depending on your client.
If you only do an explain plan of that though, e.g. with
set auto trace traceonly explain
select * from dual where dummy = :some_var;
... then the bind variable doesn't necessarily have to be populated for the plan to be calculated. Some clients may still complain and want a bind value, but the parser would be OK with it - enough to produce a plan anyway. Though not able to take advantage of bind variable peeking or histograms etc.
For example, SQL Developer happily produces a plan for your original sample query if both references are turned into bind variables, just the insert ... part of the block is selected, and you press Explain Plan (F10).
I'm not sure what you read, but you're mixed up on a few things here.
Your DATE_IN is a variable. You don't need to type 'VARIABLE' anywhere to declare a variable, all you need is the name of the variable and the datatype.
All of the below are legitimate variables in PL/SQL (although poorly named).
variable_1 NUMBER;
variable_2 VARCHAR2(100);
variable_3 DATE;
It's hard to tell what you're doing in your code without seeing it all. Do you have two DATE_IN variables declared within the same block? Is DATE_IN the name of a column in your table?
If you have a column named DATE_IN in table1 or table2, that's likely your problem. Oracle doesn't know if you want to use your variable or your column, and it will always default to the column name. Your function would be expecting a DATE and receiving a column, hence the error.
I have 2 procedures inside a package. I am calling one procedure to get a comma separated list of user ids.
I am storing the result in a VARCHAR variable. Now when I am using this comma separated list to put inside an IN clause in it is throwing "ORA-01722:INVALID NUMBER" exception.
This is how my variable looks like
l_userIds VARCHAR2(4000) := null;
This is where i am assigning the value
l_userIds := getUserIds(deptId); -- this returns a comma separated list
And my second query is like -
select * from users_Table where user_id in (l_userIds);
If I run this query I get INVALID NUMBER error.
Can someone help here.
Do you really need to return a comma-separated list? It would generally be much better to declare a collection type
CREATE TYPE num_table
AS TABLE OF NUMBER;
Declare a function that returns an instance of this collection
CREATE OR REPLACE FUNCTION get_nums
RETURN num_table
IS
l_nums num_table := num_table();
BEGIN
for i in 1 .. 10
loop
l_nums.extend;
l_nums(i) := i*2;
end loop;
END;
and then use that collection in your query
SELECT *
FROM users_table
WHERE user_id IN (SELECT * FROM TABLE( l_nums ));
It is possible to use dynamic SQL as well (which #Sebas demonstrates). The downside to that, however, is that every call to the procedure will generate a new SQL statement that needs to be parsed again before it is executed. It also puts pressure on the library cache which can cause Oracle to purge lots of other reusable SQL statements which can create lots of other performance problems.
You can search the list using like instead of in:
select *
from users_Table
where ','||l_userIds||',' like '%,'||cast(user_id as varchar2(255))||',%';
This has the virtue of simplicity (no additional functions or dynamic SQL). However, it does preclude the use of indexes on user_id. For a smallish table this shouldn't be a problem.
The problem is that oracle does not interprete the VARCHAR2 string you're passing as a sequence of numbers, it is just a string.
A solution is to make the whole query a string (VARCHAR2) and then execute it so the engine knows he has to translate the content:
DECLARE
TYPE T_UT IS TABLE OF users_Table%ROWTYPE;
aVar T_UT;
BEGIN
EXECUTE IMMEDIATE 'select * from users_Table where user_id in (' || l_userIds || ')' INTO aVar;
...
END;
A more complex but also elegant solution would be to split the string into a table TYPE and use it casted directly into the query. See what Tom thinks about it.
DO NOT USE THIS SOLUTION!
Firstly, I wanted to delete it, but I think, it might be informative for someone to see such a bad solution. Using dynamic SQL like this causes multiple execution plans creation - 1 execution plan per 1 set of data in IN clause, because there is no binding used and for the DB, every query is a different one (SGA gets filled with lots of very similar execution plans, every time the query is run with a different parameter, more memory is needlessly used in SGA).
Wanted to write another answer using Dynamic SQL more properly (with binding variables), but Justin Cave's answer is the best, anyway.
You might also wanna try REF CURSOR (haven't tried that exact code myself, might need some little tweaks):
DECLARE
deptId NUMBER := 2;
l_userIds VARCHAR2(2000) := getUserIds(deptId);
TYPE t_my_ref_cursor IS REF CURSOR;
c_cursor t_my_ref_cursor;
l_row users_Table%ROWTYPE;
l_query VARCHAR2(5000);
BEGIN
l_query := 'SELECT * FROM users_Table WHERE user_id IN ('|| l_userIds ||')';
OPEN c_cursor FOR l_query;
FETCH c_cursor INTO l_row;
WHILE c_cursor%FOUND
LOOP
-- do something with your row
FETCH c_cursor INTO l_row;
END LOOP;
END;
/
I'm working on parameterizing some JDBC queries against Oracle 10gR2.
Most of the queries are of the form:
String value = "somevalue";
String query = "select dbms_xmlgen.xmlget('select c1, c2 from t1 where c1 = ''"
+ somevalue + "'' ') xml from dual;";
I can't parameterize that as is, since the actual select is in a quoted string inside xmlget and parameters are not expanded inside a string. JDBC will see that query as having no parameters.
I've been fairly successful in emulating the behavior of dbms_xmlgen.xmlget with:
String query = "SELECT xmltype.getclobval(sys_xmlagg(xmlelement(\"ROW\","
+ "xmlforest(c1, c2)))) xml from t1 where c1 = ?";
The only issue I have not been able to resolve is the case where the query returns no rows.
With dbms_xmlgen.xmlget, no rows returns an empty CLOB. But, with sys_xmlagg, no rows results in a CLOB consisting of:
<?xml version="1.0"?><ROWSET></ROWSET>
I'm looking for a solution that will give me an empty CLOB instead of an empty document.
I don't have access to an Oracle DB at the moment, so please forgive inaccuracies.
The parameterization of the DBMS_XMLGEN call seems to be the goal. This is accomplished by using a little PL/SQL. The Oracle Docs for the DBMS_XMLGEN package describe a few operations which should help. First, create a context from a SYS_REFCURSOR using this form:
DBMS_XMLGEN.NEWCONTEXT (
queryString IN SYS_REFCURSOR)
RETURN ctxHandle;
Then, use the context in another form of GetXML:
DBMS_XMLGEN.GETXML (
ctx IN ctxHandle,
tmpclob IN OUT NCOPY CLOB,
dtdOrSchema IN number := NONE)
RETURN BOOLEAN;
Using this method also gives the benefit of potentially reusing the CLOB (not making a new temporary one), which may help with performance. There is another form which is more like the one you were using in your example, but loses this property.
One more thing... The return of GETXML in this example should tell you whether there were rows returned or not. This should be more reliable than checking the contents of the CLOB when the operation completes. Alternately, you can use the NumRowsProcessed function on the context to get the count of the rows included in the CLOB.
Roughly, your code would look something like this:
DECLARE
srcRefCursor SYS_REFCURSOR;
ctxHandle ctxHandle;
somevalue VARCHAR2(1000);
myClob CLOB;
hasRows boolean;
BEGIN
OPEN srcRefCursor FOR
SELECT c1, c2
FROM t1
WHERE c1 = somevalue; --Note parameterized value
ctxHandle := DBMS_XMLGEN.NEWCONTEXT(srcRefCursor);
hasRows := DBMS_XMLGEN.GETXML(
ctxHandle,
myClob -- XML stored in myCLOB
);
IF (hasRows) THEN
/* Do work on CLOB here */
END IF;
DBMS_XMLGEN.CLOSECONTEXT(ctxHandle);
END;