I have a Stored Proc which performs search on records.
The problem is that some of the search criteria,which are coming from UI, may be empty string.
So, when the criteria not specified, the LIKE statement becomes redundant.
How can I effectively perform that search or Sql Server? Or, Does it optimize LIKE('%%') query since it means there is nothing to compare?
The Stored proc is like this:
ALTER PROC [FRA].[MCC_SEARCH]
#MCC_Code varchar(4),
#MCC_Desc nvarchar(50),
#Detail nvarchar(50)
AS
BEGIN
SELECT
MCC_Code,
MCC_Desc,
CreateDate,
CreatingUser
FROM
FRA.MCC (NOLOCK)
WHERE
MCC_Code LIKE ('%' + #MCC_Code + '%')
AND MCC_Desc LIKE ('%' + #MCC_Desc + '%')
AND Detail LIKE ('%' + #Detail + '%')
ORDER BY MCC_Code
END
With regard to an optimal, index-using execution plan - no. The prefixing wildcard prevents an index from being used, resulting in a scan instead.
If you do not have a wildcard on the end of the search term as well, then that scenario can be optimised - something I blogged out a while back: Optimising wildcard prefixed LIKE conditions
Update
To clarify my point:
LIKE 'Something%' - is able to use an index
LIKE '%Something' - is not able to use an index out-of-the-box. But you can optimise this to allow it to use an index by following the "REVERSE technique" I linked to.
LIKE '%Something%' - is not able to use an index. Nothing you can do to optimise for LIKE.
The short answer is - no
The long answer is - absolutely not
Does it optimize LIKE('%%') query since it means there is nothing to compare?
The statement is untrue, because there is something to compare. The following are equivalent
WHERE column LIKE '%%'
WHERE column IS NOT NULL
IS NOT NULL requires a table scan, unless there are very few non-null values in the column and it is well indexed.
EDIT
Resource on Dynamic Search procedures in SQL Server:
You simply must read this article by Erland Sommarskog, SQL Server MVP http://www.sommarskog.se/dyn-search.html (pick your version, or read both)
Otherwise if you need good performance on CONTAINS style searches, consider using SQL Server Fulltext engine.
If you use a LIKE clausule, and specify a wildcard-character (%) as a prefix of the searchstring, SQL Server (and all other DBMS'es I guess) will not be able to use indexes that might exists on that column.
I don't know if it optimizes the query if you use an empty search-argument ... Perhaps your question may be answered if you look at the execution plan ?
Edit: I've just checked this out, and the execution plan of this statement:
select * from mytable
is exactly the same as this the exec plan of this statement:
select * from mytable where description like '%'
Both SQL statements simply use a clustered index scan.
Related
Is there difference in performance when querying someting like this?
Query 1:
SELECT * FROM Customer WHERE Name=Name
Query 2:
SELECT * FROM Customer
I will use it in conditional select all
SELECT * FROM Customer
WHERE Name = CASE WHEN #QueryAll='true' THEN Name ELSE #SearchValue END
If there's no performance issue in Query 1 and 2, I think it is a short code for this one:
IF #QueryAll='true'
SELECT * FROM Customer
ELSE
SELECT * FROM Customer WHERE Name=#SearchValue
You should read Dynamic Search Conditions in T‑SQL by Erland Sommarskog.
If you use SQL Server 2008 or later, then use OPTION(RECOMPILE) and write the query like this:
SELECT *
FROM Customer
WHERE
(Name = #SearchValue OR #QueryAll='true')
OPTION (RECOMPILE);
I usually pass NULL for #SearchValue to indicate that this parameter should be ignored, rather than using separate parameter #QueryAll. In this convention the query becomes this:
SELECT *
FROM Customer
WHERE
(Name = #SearchValue OR #SearchValue IS NULL)
OPTION (RECOMPILE);
Edit
For details see the link above. In short, OPTION(RECOMPILE) instructs SQL Server to recompile execution plan of the query every time it is run and SQL Server will not cache the generated plan. Recompilation also means that values of any variables are effectively inlined into the query and optimizer knows them.
So, if #SearchValue is NULL, optimizer is smart enough to generate the plan as if the query was this:
SELECT *
FROM Customer
If #SearchValue has a non-NULL value 'abc', optimizer is smart enough to generate the plan as if the query was this:
SELECT *
FROM Customer
WHERE (Name = 'abc')
The obvious drawback of OPTION(RECOMPILE) is added overhead for recompilation (usually around few hundred milliseconds), which can be significant if you run the query very often.
The query 2 would be faster with an index on name column, and you should specify just the fields you'll need, not all of them.
For some guidance in optional parameter queries, take a look here: Sometimes the Simplest Solution Isn't the Best Solution (The Optional Parameter Problem)
I have a few columns where the data will be the same results.
DECLARE #Alias AS nvarchar(15)
SET #Alias = '%MyAlias%'
SELECT TOP 10
*
FROM
ATSStaging..LIR_Amendment
WHERE
(AppliedByUser LIKE #Alias
OR ModifiedByUser LIKE #Alias
OR CreatedByUser LIKE #Alias)
Rather than searching all 3 columns to chekc for the same value is there a way to shorten this?
Something like:
WHERE
(AppliedByUser OR ModifiedbyUser OR CreatedByUser LIKE #Alias
No, but you can trick it in your case with:
WHERE
concat(AppliedByUser, ModifiedbyUser, CreatedByUser) LIKE #Alias
You could concatenate the values together and then use like:
SELECT TOP 10 *
FROM ATSStaging..LIR_Amendment
WHERE (AppliedByUser + '|' + ModifiedByUser + '|' + CreatedByUser) LIKE #Alias;
I wouldn't recommend this as a general solution. But, your like patterns starts with a wildcard so your query would require a full table scan anyway. The concatenation probably incurs a small amount of additional overhead, versus the or (which should stop at the first column that matches). You might find that a full-text index gives you much improved functionality and performance.
In this specific case (because you are using a LIKE '%match%' expression), you can use concatenation:
WHERE AppliedByUser + ModifiedbyUser + CreatedByUser LIKE #Alias
In general, no: butthere's nothing really wrong with the query the way you had it originally.
Depending on the DBMS that you are using you may use FULLTEXTSEARCH.
In SQL Server for example, that would mean creating a indexed view containing the data you want to search in and then a FULLTEXT index on that column.
This will provide the best performance as patter matching doesn't perform very well over normal indexes.
I am using SQL Server 2008 Enterprise with Windows Server 2008 Enterprise. I have a database table called "Book", which has many columns and three columns are important in this question, they are
Author, varchar;
Country, varchar;
Domain, varchar.
I want to write a store procedure with the following logics, but I do not know how to write (because of complex query conditions), appreciate if anyone could write a sample for me?
Input parameter: p_author as varchar, p_country as varchar, and p_domain as varchar
Query conditions:
if p_author is specified from input, then any row whose Author column LIKE %p_author% is satisfied with condition, if p_author is not specified from input every row is satisfied with this condition;
if p_country is specified from input, then any row whose Country column = p_country is satisfied with condition, if p_country is not specified from input every row is satisfied with this condition;
if p_domain is specified from input, then any row whose Domain column LIKE %p_domain% is satisfied, if p_domain is not specified from input every row is satisfied with this condition;
The results I want to return (must met with all following conditions):
records met with either condition 1 or 2;
records must meet with condition 3;
return distinct rows.
For example, records which met with condition 1 and condition 3 are ok to return, and records which met with condition 2 and condition 3 are ok to return.
thanks in advance,
George
Dynamically changing searches based on the given parameters is a complicated subject and doing it one way over another, even with only a very slight difference, can have massive performance implications. The key is to use an index, ignore compact code, ignore worrying about repeating code, you must make a good query execution plan (use an index).
Read this and consider all the methods. Your best method will depend on your parameters, your data, your schema, and your actual usage:
Dynamic Search Conditions in T-SQL by by Erland Sommarskog
The Curse and Blessings of Dynamic SQL by Erland Sommarskog
If you have the proper SQL Server 2008 version (SQL 2008 SP1 CU5 (10.0.2746) and later), you can use this little trick to actually use an index:
There isn't much you can do since you are using LIKE, but if you were using equality, you could add OPTION (RECOMPILE) onto your query, see Erland's article, and SQL Server will resolve the OR from within (Column = #Param+'%' OR #Param='') AND ... before the query plan is created based on the run-time values of the local variables, and an index can be used if you weren't using LIKE.
If I understand correctly, the following should work:
SELECT *
FROM Books
WHERE (
((Author LIKE '%' + #p_author + '%' OR #p_author = '') OR
(Country LIKE '%' + #p_country + '%' OR #p_country = ''))
AND (#p_author <> '' OR #p_country <> '')
) AND
(Domain LIKE '%' + #p_domain + '%' OR '%' #p_domain = '')
I am converting a stored procedure which I had previously written as a string then, using BIT parameters I decided whether to append certain WHERE/ON clauses
This sp is passed a number of comma-separated strings and then some of the dynamic WHERE clauses are like:
IF #pUse_Clause_A THEN SET #WhereClause = #WhereClause + ' AND [FIELD_A] IN (' + #pComma_Separated_List_A + ')'
In this case, #pComma_Separated_List_A is something like '1,3,6,66,22' ... a list of the things I want included.
Now I am changing these from strings into TVP,s so I can just use "real" SQL like
AND [FIELD_A] IN (SELECT [TVP_FIELD] FROM #pTVP_A)
When I do this, I don't like the string-building method
However, I also don't like having to nest the IF statements.
IF A
ENTIRE SQL WHERE A
ELSE
ENTIRE SQL WITHOUT WHERE CLAUSE
The more parameters I add, the more complicated it gets:
IF A
IF B
SQL WHERE A AND B
ELSE
SQL WHERE A
ELSE
IF B
SQL WHERE B
ELSE
SQL
What I would rather do is something like this:
SELECT * FROM TABLE
WHERE 1=1
CASE USE_A WHEN 1 THEN
AND [FIELD_A] IN (SELECT A FROM TBP_A)
END
CASE USE_B WHEN 1 THEN
AND [FIELD_B] IN (SELECT B FROM TVP_B)
END
I know it ignored SQL outside the chosen "IF" result, but having all that duplicated statement seems sloppy
Dynamically changing searches based on the given parameters is a complicated subject and doing it one way over another, even with only a very slight difference, can have massive performance implications. The key is to use an index, ignore compact code, ignore worrying about repeating code, you must make a good query execution plan (use an index).
Read this and consider all the methods. Your best method will depend on your parameters, your data, your schema, and your actual usage:
Dynamic Search Conditions in T-SQL by by Erland Sommarskog
The Curse and Blessings of Dynamic SQL by Erland Sommarskog
I have the following SQL query:
SELECT Phrases.*
FROM Phrases
WHERE (((Phrases.phrase) Like "*ing aids*")
AND ((Phrases.phrase) Not Like "*getting*")
AND ((Phrases.phrase) Not Like "*contracting*"))
AND ((Phrases.phrase) Not Like "*preventing*"); //(etc.)
Now, if I were using RegEx, I might bunch all the Nots into one big (getting|contracting|preventing), but I'm not sure how to do this in SQL.
Is there a way to render this query more legibly/elegantly?
Just by removing redundant stuff and using a consistent naming convention your SQL looks way cooler:
SELECT *
FROM phrases
WHERE phrase LIKE '%ing aids%'
AND phrase NOT LIKE '%getting%'
AND phrase NOT LIKE '%contracting%'
AND phrase NOT LIKE '%preventing%'
You talk about regular expressions. Some DBMS do have it: MySQL, Oracle... However, the choice of either syntax should take into account the execution plan of the query: "how quick it is" rather than "how nice it looks".
With MySQL, you're able to use regular expression where-clause parameters:
SELECT something FROM table WHERE column REGEXP 'regexp'
So if that's what you're using, you could write a regular expression string that is possibly a bit more compact that your 4 like criteria. It may not be as easy to see what the query is doing for other people, however.
It looks like SQL Server offers a similar feature.
Sinec it sounds like you're building this as you go to mine your data, here's something that you could consider:
CREATE TABLE Includes (phrase VARCHAR(50) NOT NULL)
CREATE TABLE Excludes (phrase VARCHAR(50) NOT NULL)
INSERT INTO Includes VALUES ('%ing aids%')
INSERT INTO Excludes VALUES ('%getting%')
INSERT INTO Excludes VALUES ('%contracting%')
INSERT INTO Excludes VALUES ('%preventing%')
SELECT
*
FROM
Phrases P
WHERE
EXISTS (SELECT * FROM Includes I WHERE P.phrase LIKE I.phrase) AND
NOT EXISTS (SELECT * FROM Excludes E WHERE P.phrase LIKE E.phrase)
You are then always just running the same query and you can simply change what's in the Includes and Excludes tables to refine your searches.
Depending on what SQL server you are using, it may support REGEX itself. For example, google searches show that SQL Server, Oracle, and mysql all support regex.
You could push all your negative criteria into a short circuiting CASE expression (works Sql Server, not sure about MSAccess).
SELECT *
FROM phrases
WHERE phrase LIKE '%ing aids%'
AND CASE
WHEN phrase LIKE '%getting%' THEN 2
WHEN phrase LIKE '%contracting%' THEN 2
WHEN phrase LIKE '%preventing%' THEN 2
ELSE 1
END = 1
On the "more efficient" side, you need to find some criteria that allows you to avoid reading the entire Phrases column. Double sided wildcard criteria is bad. Right sided wildcard criteria is good.