I'm still new to SQL, so I'm having some little issues to solve.
I'm running a Postgres database in Acqua Data Studio, with some queries that get follow the same model.
Some variables into these queries are the same, but may change in the future...
Thinking of an optimized database, it would be faster to change the value of a constant than to enter on 20+ queries and change the same aspect in all of them.
Example:
SELECT *
FROM Table AS Default_Configs
LEFT JOIN Table AS Test_Configs
ON Default_Configs.Column1 = 'BLABLABLA'
Imagining 'BLABLABLA' could be 'XXX', how could I make 'BLABLABLA' a constant to every View that is created following this pattern?
Create a tiny function that serves as "global constant":
CREATE OR REPLACE FUNCTION f_my_constant()
RETURNS text AS
$$SELECT text 'XXX'$$ LANGUAGE sql IMMUTABLE PARALLEL SAFE; -- see below
And use that function instead of 'BLABLABLA' in your queries.
Be sure to declare the data type correctly and make the function IMMUTABLE (because it is) for better performance with big queries.
In Postgres 9.6 or later add PARALLEL SAFE, so it won't block parallel query plans. The setting isn't valid in older versions.
To change the constant, replace the function by running an updated CREATE OR REPLACE FUNCTION statement. Invalidates query plans using it automatically, so queries are re-planned. Should be safe for concurrent use. Transactions starting after the change use the new function. But indexes involving the function have to be rebuilt manually.
Alternatively (especially in pg 9.2 or later), you could set a Customized Option as "global constant" for the whole cluster, a given DB, a given role etc, and retrieve the value with:
current_setting('constant.blabla')
One limitation: the value is always text and may have to be cast to a target type.
Related:
User defined variables in PostgreSQL
Many ways to set it:
How does the search_path influence identifier resolution and the "current schema"
Related
Suppose we have a poorly performing stored procedure with 6 parameters. If one of the six parameters is transferred to a local variable within the stored procedure, is that enough to disable parameter sniffing or is it necessary to transfer all 6 parameters that're passed to the stored procedure into local variables within the stored procedure?
Per Paul White's comment, assigning a variable to a local variable is a workaround from older versions of SQL Server. It won't help with sp_executesql, and Microsoft could write a smarter parser that would invalidate this workaround. The workaround works by confusing the parser about a parameter's value, so in order for it to work for each parameter, you'd have to store each parameter in a local variable.
More recent versions of SQL Server have better solutions. For an expensive query that is not run often, I'd use option (recompile). For example:
SELECT *
FROM YourTable
WHERE col1 = #par1 AND col2 = #par2 AND ...
OPTION (RECOMPILE)
This will cause the query planner to recreate ("recompile") a plan every time the stored procedure is called. Given the low cost of planning (typically below 25ms) that is sensible behavior for expensive queries. It's worth 25ms to check if you can create a smarter plan for specific parameters to a 250ms query.
If your query is run so often that the cost of planning is nontrivial, you can use option (optimize for unknown). That will cause SQL Server to create a plan that it expects to work well for all values of all parameters. When you specify this option, SQL Server ignores the first values of the parameters, so this literally prevents sniffing.
SELECT *
FROM YourTable
WHERE col1 = #par1 AND col2 = #par2 AND ...
OPTION (OPTIMIZE FOR UNKNOWN)
This variant works for all parameters. You can use optimize for (#par1 unknown) to prevent sniffing for just one parameter.
I got tired off for last two days, and now I find a solution. If anyone get relief, so I posted my experience here.
I have a query, fairly complex, having 5 CTEs and an union already known for parameter sniffing from its design. We opted OPTION RECOMPILE to solve it, and it works fairly good. After 2 years, we create a high available cluster and separate the report server. All works well for 1 year and now cause of Covid19 we need to shut down for 30 days. Server is on but all activity goes quite. Meanwhile we need to truncate the database because of extreme log size and data growth, taking out of available groups and re adding to availability. From last two date this query shows parameter sniffing activity, and no remedy works except one.
The silver bullet saves me is EXEC sp_updatestats.
This works for me, and now I have time to find a proper solution for permanent fix.
I am trying to avoid oracle's plain to be calculated at every dynamic SQL use.
Using REPLACE and then a EXECUTE IMMEDIATE without USING leads to overhead, as the string is different at every use, Oracle always seems to search a new plain.
But reading about EXECUTE IMMEDIATE statement a found the following:
You can execute a dynamic SQL statement repeatedly using new values for the bind arguments. You still incur some overhead, because EXECUTE IMMEDIATE re-prepares the dynamic string before every execution.
Is that? Does USING bind forces Oracle to recalculate every dynamic string? If so, how to avoid it?
a) querytoselect = REPLACE( querytoselect, 'pattern', 'var' );
EXECUTE IMMEDIATE querytoselect.
b) EXECUTE IMMEDIATE querytoselect USING var
Thanks.
EXECUTE IMMEDIATE always does a parse, but if the statement is identical to one already in memory, it can simply use the plan already determined (this is called a "soft parse") instead of having to fully parse the statement and determine a new plan (a "hard parse").
There is really more subtlety to it than that, and depends on your configuration as well. For instance, even if a matching statement is found in memory, bind peeking and table statistics could result in a hard parse being performed to determine if there is a potentially better plan given the particular input.
But overall, a soft parse is less overhead than a hard parse, and using bind variables increases the chance of avoiding a hard parse.
To completely avoid re-parsing, you can use DBMS_SQL instead of EXECUTE IMMEDIATE and hold onto the cursor after parsing. I wouldn't bother doing this unless you've identified a significant performance issue caused by soft parsing.
I didn't undetstood the difference between Stored Functions and Views.
Using Views in SELECT will execute the query and return the result, but Stored Functions do the same thing, don't they? So what is the difference? When I use Views and when Stored Functions?
View:
A view is a virtual table. It does not physically exist. Rather, it is created by a query joining one or more tables. View returns a table.
Stored procedure: A stored procedure is a group of Transact-SQL statements compiled into a single execution plan.
stored procedures returns Output parameters,return codes (which are always an integer value),
a result set for each SELECT statement contained in the stored procedure or any other stored procedures called by the stored procedure,a global cursor that can be referenced outside the stored procedure.
key benefits of stored procedure are Precompiled execution, reduced client/server traffic,efficient reuse of code, programming abstraction and enhanced security controls.
Update:
A stored function is a named PL/SQL Block which is similar to a procedure. The major difference between a procedure and a function is, a function must always return a value, but a procedure may or may not return a value.
1) Return Type: The header section defines the return type of the function. The return datatype can be any of the oracle datatype like varchar, number etc.
2) The execution and exception section both should return a value which is of the datatype defined in the header section
You can have a stored function return the same data a view would in most databases.
The distinction for me is that a function is executed and a view is selected from.
A view will behave as a table.
A view returns a specific pre-defined statement as exactly one result set.
A function returns a single values or a single result set. This however can differ from different types of database.
Several db implementations also have stored procedures where the result can be a single returned value, a result set or several result sets.
Getting simple (PLEASE start reading a book about SQL): A view looks like a table, so you can filter on the results and the filter will efficiently be part of the views execution, or do joins. A SP does not allow this, but a lot more logic. The rest... is in the documentation.
These can never be compares, these have totally different
approach.
A view is a output of a query ,and makes a virtual image of the table,and the input parameters are not accepted.
Main difference is that a Stored Procedure can alter your data, where
as a view only returns it and I believe from a performance point of
view, a stored procedure is better as it caches the execution plan and
will run faster as a result.
storedprocedure/function is a group of sql statements that are pre-executed and it accepts the parameters.it reduces network traffic, gives faster performance, etc.
SQL Functions in programming languages are subroutines used to encapsulate frequently performed logic. these somewhat slow down the performance.
Check these SQL View, SQL Stored Procedures and SQL User-Defined Functions
My Opinion is that SQL Stored Procedure(Stored Functions) are much better to use because it provides custom manipulations on result set also.
From my experiences I'm sharing to you my knowledge:
Don't use views
Better to use a stored procedure(it is compiled sql statement), you can use parametrized procedure as required.
Stored Function is collection of complied sql statement which is faster.
Note: Views is a SELECT statement( with/without JOIN) for a table which select data from table and if we again run a SELECT statement from VIEWS which provide slower result because the internal operation is as ( SELECT * FROM ( SELECT * FROM TargetTable ) )
So, its better to use Stored Function
Update:
Functions are computed values and cannot perform permanent environmental changed to SQL Server (i.e. no INSERT or UPDATE statements allowed).
A Function can be used inline in SQL Statements if it returns a scalar value or can be joined upon if it returns a result set.
Also please see here for performance comparison: SQL-Server Performance: What is faster, a stored procedure or a view?
I have to execute a loop in database. This is only a one time requirement.
After executing the function, I am dropping the function now.
Is there any good approach for creating temporary / disposable functions?
I needed to know how to do a many time use in a script I was writing. Turns out you can create a temporary function using the pg_temp schema. This is a schema that is created on demand for your connection and is where temporary tables are stored. When your connection is closed or expires this schema is dropped. Turns out if you create a function on this schema, the schema will be created automatically. Therefore,
create function pg_temp.testfunc() returns text as
$$ select 'hello'::text $$ language sql;
will be a function that will stick around as long as your connection sticks around. No need to call a drop command.
A couple of additional notes to the smart trick in #crowmagnumb's answer:
The function must be schema-qualified at all times, even if pg_temp is in the search_path (like it is by default), according to Tom Lane to prevent Trojan horses:
CREATE FUNCTION pg_temp.f_inc(int)
RETURNS int AS 'SELECT $1 + 1' LANGUAGE sql IMMUTABLE;
SELECT pg_temp.f_inc(42);
f_inc
-----
43
A function created in the temporary schema is only visible inside the same session (just like temp tables). It's invisible to all other sessions (even for the same role). You could access the function as a different role in the same session after SET ROLE.
You could even create a functional index based on this "temp" function:
CREATE INDEX foo_idx ON tbl (pg_temp.f_inc(id));
Thereby creating a plain index using a temporary function on a non-temp table. Such an index would be visible to all sessions but still only valid for the creating session. The query planner will not use a functional index, where the expression is not repeated in the query. Still a bit of a dirty trick. It will be dropped automatically when the session is closed - as a depending object. Feels like this should not be allowed at all ...
If you just need to execute a function repeatedly and all you need is SQL, consider a prepared statement instead. It acts much like a temporary SQL function that dies at the end of the session. Not the same thing, though, and can only be used by itself with EXECUTE, not nested inside another query. Example:
PREPARE upd_tbl AS
UPDATE tbl t SET set_name = $2 WHERE tbl_id = $1;
Call:
EXECUTE upd_tbl(123, 'foo_name');
Details:
Split given string and prepare case statement
If you are using version 9.0, you can do this with the new DO statement:
http://www.postgresql.org/docs/current/static/sql-do.html
With previous versions, you'll need to create the function, call it, and drop it again.
For ad hock procedures, cursors aren't too bad. They are too inefficient for productino use however.
They will let you easily loop on sql results in the db.
I have some doubt regarding user defined functions. I would like to know why / when to use functions.
What are the advantages of functions over stored procedure?
Researching via google I have seen articles suggesting:
stored procedure are more advantageous than functions.
function have limited error handling
functions cannot use temporary tables
functions cannot call stored procedures.
The only advantage of function is we can use function as inline queries.
I can get the same result with stored procedure by using temporary tables, but i need to know which scenario to use functions compared to stored procedure.
I need to know why we need UDf , when most of the functionalities provided by UDF can be done by Stored procedure.
Can any one guide me over this.
The main difference (advantage) is that you can call functions inline unlike stored procedures
e.g.
SELECT dbo.fxnFormatName(FirstName, LastName) AS FormattedName
FROM MyTable
SELECT *
FROM dbo.fxnTableReturningFunction() x
User defined functions can return TABLE type data and then the function can then be called within a query as demonstrated above. With a sproc, you'd have to execute it and store the results into a temporary table in order to then manipulate/query the resultset further.
On the flip side, yes you are limited as to what you can do in a function. e.g. you can't use dynamic sql, and pre-SQL 2005 you can't use non-deterministic functions like GETDATE() within a function.
An example of when you may want to use functions, is to wrap up common "formatting" functionality as shown in the first example above - rather than repeat the logic to format a first and last name into one in every query, you wrap it in a function and call that everywhere. Typically I'd recommend leaving the formatting up to the UI but it's a simple example of where/why you might use.
Also, it can often be nicer to not have to create temp tables to hold results from a sproc in order to query it further. If the sproc changes and returns more columns, you'd also need to change everywhere that loads the results into a temp table to synch the schema of the table table it uses to hold the results with the new schema returned. You don't have this problem with the function approach as there is no temp table to be maintained.
There are three types of functions: Scalar, Inline Table and Table Valued. Generally speaking, Scalar & Table Values functions can lead to performance problems, seeing as the Query Optimiser doesn't do very well at optimisation of the use of those types of functions. The performance of Inline Table function is just fine, however.
There is a Connect request to create a new type of scalar function here: The Scalar Expression function would speed performance...
I hope that people do vote for that one, because it would improve performance greatly by allowing the query optimiser to inline functional expressions and take advantage of statistics etc just as it would for a normal query.
The main "disadvantage" of user-defined functions is that they are called for each row. So, if you have such a function in the SELECT list and you're operating on larger sets, there are good chances that your performance will suffer.
Advantage of Mysql Stored Procedure
Multiple applications are running in multiple environment and need to use the same database. By using stored procedure you can make your business logic independent of programming language.
When security is main concern use of stored procedure is vital. By doing your operation through the database you can log your all performed action. Banking site is the best example.
If you are using stored procedure then you do not have table access directly which is one more way to secure the data and transaction.
Stored procedure increases performance of your application sometime
If your application is big or your database server on remote system then by using stored procedure you can decrease the traffic between your database server and application server.
Since stored procedure is written in your database server and application call it sepratly then the degree of re-usability.