I am trying to avoid oracle's plain to be calculated at every dynamic SQL use.
Using REPLACE and then a EXECUTE IMMEDIATE without USING leads to overhead, as the string is different at every use, Oracle always seems to search a new plain.
But reading about EXECUTE IMMEDIATE statement a found the following:
You can execute a dynamic SQL statement repeatedly using new values for the bind arguments. You still incur some overhead, because EXECUTE IMMEDIATE re-prepares the dynamic string before every execution.
Is that? Does USING bind forces Oracle to recalculate every dynamic string? If so, how to avoid it?
a) querytoselect = REPLACE( querytoselect, 'pattern', 'var' );
EXECUTE IMMEDIATE querytoselect.
b) EXECUTE IMMEDIATE querytoselect USING var
Thanks.
EXECUTE IMMEDIATE always does a parse, but if the statement is identical to one already in memory, it can simply use the plan already determined (this is called a "soft parse") instead of having to fully parse the statement and determine a new plan (a "hard parse").
There is really more subtlety to it than that, and depends on your configuration as well. For instance, even if a matching statement is found in memory, bind peeking and table statistics could result in a hard parse being performed to determine if there is a potentially better plan given the particular input.
But overall, a soft parse is less overhead than a hard parse, and using bind variables increases the chance of avoiding a hard parse.
To completely avoid re-parsing, you can use DBMS_SQL instead of EXECUTE IMMEDIATE and hold onto the cursor after parsing. I wouldn't bother doing this unless you've identified a significant performance issue caused by soft parsing.
Related
I have a work mate who claims that dynamic SQL performs faster than static SQL in many situations, so I constantly see DSQL all over the place. Aside from the obvious downsides like not being able to detect errors until running it and that it's harder to read, is this accurate or not? When I asked why he was using DSQL all the time, he said:
Static is preferred when it is not going to prevent cache reuse and dynamic is preferred when static will prevent cache reuse and reuse is desirable.
I asked why static SQL would prevent cache reuse and he said:
Apparently, when variables are passed to statement predicates it may prevent cache reuse of that execution plan, where DSQL will allow cache reuse in a stored procedure.
So, for example:
select * from mytable where myvar = #myvar
I'm not an expert in SQL Server execution plans, but this seems irrational to me. Why would the engine keep stats in a DSQL statement in a stored procedure, but not a static SQL statement?
Dynamic SQL has the advantage that a query is recompiled every time it is run. This has the advantage that the execution plan can take advantage of the most recent statistics on the table and the values of any parameters.
In addition to being more readable, static SQL has the advantage that it does not need to be recompiled -- saving a step in running the query (well, actually two if you count parsing-->compiling-->executing). This may or may not be a good thing.
You can force static plans to recompile using the with (recompile) option.
Sometimes, you need to use dynamic SQL. As a general rule, though, I would use compiler hints and other efforts to manage performance before depending on dynamic SQL.
From SQL server perspective, I don't think you should be using dynamic SQL much.
Problems can include :
In memory table will be passed read only
With every value concatenated, query will do recompilation.
Execution plan will anyway use table statistics and other db information, but do you need to recompile execution plan. You can mend your queries to use LOOP JOIN, HASH JOIN or MERGE JOIN yourself if you feel you know better than SQL Server engine. Otherwise, just write procedures the way they are intended to be written, and pass parameters.
In my opinion, do not use dynamic queries...Instead use procedures in which every change-able value is passed as parameter and has no hard-coding. Create proper (and fewer) indexes and triggers to deal with data. You will also be hence be able to easily use in memory table variables and memory tables feature directly otherwise in memory table variables are passed to dynamic query as readonly.
I'm still new to SQL, so I'm having some little issues to solve.
I'm running a Postgres database in Acqua Data Studio, with some queries that get follow the same model.
Some variables into these queries are the same, but may change in the future...
Thinking of an optimized database, it would be faster to change the value of a constant than to enter on 20+ queries and change the same aspect in all of them.
Example:
SELECT *
FROM Table AS Default_Configs
LEFT JOIN Table AS Test_Configs
ON Default_Configs.Column1 = 'BLABLABLA'
Imagining 'BLABLABLA' could be 'XXX', how could I make 'BLABLABLA' a constant to every View that is created following this pattern?
Create a tiny function that serves as "global constant":
CREATE OR REPLACE FUNCTION f_my_constant()
RETURNS text AS
$$SELECT text 'XXX'$$ LANGUAGE sql IMMUTABLE PARALLEL SAFE; -- see below
And use that function instead of 'BLABLABLA' in your queries.
Be sure to declare the data type correctly and make the function IMMUTABLE (because it is) for better performance with big queries.
In Postgres 9.6 or later add PARALLEL SAFE, so it won't block parallel query plans. The setting isn't valid in older versions.
To change the constant, replace the function by running an updated CREATE OR REPLACE FUNCTION statement. Invalidates query plans using it automatically, so queries are re-planned. Should be safe for concurrent use. Transactions starting after the change use the new function. But indexes involving the function have to be rebuilt manually.
Alternatively (especially in pg 9.2 or later), you could set a Customized Option as "global constant" for the whole cluster, a given DB, a given role etc, and retrieve the value with:
current_setting('constant.blabla')
One limitation: the value is always text and may have to be cast to a target type.
Related:
User defined variables in PostgreSQL
Many ways to set it:
How does the search_path influence identifier resolution and the "current schema"
I have a collection of sysid's that I am using to iterate over in several forall loops. This script is meant to be run on a regular basis, so I would like to know whether the collection persists in the database and needs to be cleared out, or if the script is alright as-is.
Also, I am new to PL/SQL, so if you see anything wrong with the script, please do let me know.
This is going to run on Oracle 10g and 11g.
Thanks
DECLARE
TYPE sSysid IS TABLE OF person.sysid%TYPE
INDEX BY PLS_INTEGER;
l_sSysid sSysid;
BEGIN
SELECT sysid
BULK COLLECT INTO l_sSysid
FROM person
where purge_in_process = 1;
FORALL i IN l_sSysid.FIRST .. l_sSysid.LAST
delete from person_attribute where property_pk like concat(l_sSysid(i), '%');
FORALL i IN l_sSysid.FIRST .. l_sSysid.LAST
delete from person_property where person_sysid = l_sSysid(i);
FORALL i IN l_sSysid.FIRST .. l_sSysid.LAST
delete from person where sysid = l_sSysid(i);
END;
/
commit;
The collection is a local variable so it will no longer exist after the block finishes executing. There will be no need to clear it out. Depending on the number of rows in the PERSON table where PURGE_IN_PROCESS will be 1, you may want to use the LIMIT clause in order to avoid consuming too much PGA memory though.
The idea of an anonymous PL/SQL block that is run regularly, however, is a bit foreign to me. If you intend the code to be run regularly, I'd strongly suggest that you create a stored procedure rather than an anonymous block and then schedule the procedure to be run regularly. That opens up the ability to use the database's scheduling facilities (DBMS_JOB and DBMS_SCHEDULER) to run the process and allows other applications to call it as well if the need ever arises. Plus, you'll get the benefits of things like dependency tracking in the database.
Justin is correct; but for the sake of completeness, I'll just add that if you ever decide to convert this into a stored PACKAGE, you need to take some extra care, because anything you declare in a PACKAGE specification DOES retain its value through the entire session.
I have worked on SQL stored procedures and I have noticed that many people use two different approaches -
First, to use select queries i.e. something like
Select * from TableA where colA = 10 order by colA
Second, is to do the same by constructing a query i.e. like
Declare #sqlstring varchar(100)
Declare #sqlwhereclause varchar(100)
Declare #sqlorderby varchar(100)
Set #sqlstring = 'Select * from TableA '
Set #sqlwhereclause = 'where colA = 10 '
Set #sqlorderby = 'order by colA'
Set #sqlstring = #sqlstring + #sqlwhereclause + #sqlorderby
exec #sqlstring
Now, I know both work fine. But, the second method I mentioned is a little annoying to maintain.
I want to know which one is better? Is there any specific reason one would resort to one method over the other? Any benefits of one method over other?
Use the first one. This will allow a query plan to be cached properly, apart from being the way you are supposed to work with SQL.
The second one is open to SQL Injection attacks, apart from the other issues.
With the dynamic SQL you will not get compile time checking, so it may fail only when invoked (the sooner you know about incorrect syntax, the better).
And, you noted yourself, the maintenance burden is also higher.
The second method has the obvious drawback of not being syntax checked at compile time. It does however allow a dynamic order by clause, which the first does not. I recommend that you always use the first example unless you have a very good reason to make the query dynamic. And, as #Oded has already pointed out, be sure to guard yourself against sql injection if you do go for the second approach.
I don't have a full comprehensive answer for you, but I can tell you right now that the latter method is much more difficult to work with when importing the stored procedure as a function in an ORM. Since the SQL is constructed dynamically, you have to manually create any type-classes that are returned from the stored procedure that aren't directly correlated to entities in your model.
With that in mind, there are times where you simply can't avoid constructing a SQL statement, especially when where clauses and joins depend on the parameters passed in. In my experience, I have found that stored procs that are creating large, variably joined/whered statements for EXECs are trying to do too many things. In these situations, I would recommend you keep the Single Responsibility Principle in mind.
Executing dynamic SQL inside a stored procedure reduces the value of using stored procedures to just a saved query container. Stored procedures are mostly beneficial in that the query execution plan (a very costly operation) is compiled and stored in memory the first time the procedure is executed. This means that every subsequent execution of the procedure is bypassing the query plan calculations, and jumping right to the data retrieval portiion of the operation.
Also, allowing a stored procedure to take an executable query string as a parameter is dangerous. Anyone with execute permission on granted on the procedure could potentially cause havoc on the rest of the database.
This is a pretty specific question, albeit possibly subjective, but I've been using this pattern very frequently while not seeing others use it very often. Am I missing out on something or being too paranoid?
I wrap all my UPDATE,DELETE,INSERT operations in stored procedures, and only give EXECUTE on my package and SELECT on my tables, to my application. For the UPDATE and DELETE procedures I have an IF statement at the end in which I do the following:
IF SQL%ROWCOUNT <> 1 THEN
RAISE_APPLICATION_ERROR(-20001, 'Invalid number of rows affected: ' || SQL%ROWCOUNT);
END IF;
One could also do this check in the application code, as the number of rows affected is usually available after a SQL statement is executed.
So am I missing something or is this not the safest way to ensure you're updating or deleting exactly what you want to, nothing more, nothing less?
I think this is a fine way to go. If the pl/sql proc is expected to always update/delete/insert a row and it's considered an error otherwise, then what better place to put this check than in the pl/sql proc itself? That way, no matter what client side code (C#, JAVA, PHP, Rails, etc.) uses this proc, you have this error check centralized in one place.
However, I'm not sure you need the check for an insert. If the insert fails, you should get some sort of DB exception, so no need to check for it explicitly unless you are wrapping the error in some other error message and re-raising it.
In most cases I'd use an ORM like Hibernate, which does a similar thing in order to handle Optimistic locking. Also it will use the PK in the where clause.
So I would consider this kind of stored procedure a waste of time:
- A lot of effort for minimal benefit
- Makes usage of tools like ORMs harder, which solve more and more important problems.