So I've looked around to try to find some posts on this and there are many
Like Query 1 and Like Query 2 but none that address my specific question (that I could find).
I have two tables in which I have around 5000000+ records and I am returning Search result from these tables as :
SELECT A.ContactFirstName, A.ContactLastName
FROM Customer.CustomerDetails AS A WITH (nolock)
WHERE (A.ContactFirstName + ' ' + A.ContactLastName LIKE '%' + 'a' + '%')
UNION
SELECT C.ContactFirstName, C.ContactLastName
FROM Customer.Contacts AS C WITH (nolock)
WHERE (C.ContactFirstName + ' ' + C.ContactLastName LIKE '%' + 'a' + '%')
My problem is it is taking around 1 minute to execute.
For above query I am expecting result like :
Please suggest me the best practice to improve performance. Thanks in advance.
NOTE : No missing Indexes.
when you use "LIKE '%xxx%'" index are not used that why your query is slow i think. When you use "LIKE 'xxx%')" index is used (if an index exist on column of course. >Other proble you do a like on concatenante column, i dont knwo if index is used in this case. And why do a 'xxx' + ' ' + 'yyy' like 'z%', just do 'xxx' like 'z%' its the same. You can try to modify your query like this
SELECT A.ContactFirstName, A.ContactLastName
FROM Customer.CustomerDetails AS A WITH (nolock)
WHERE A.ContactFirstName LIKE '%a%' or A.ContactLastName LIKE '%a%'
UNION
SELECT C.ContactFirstName, C.ContactLastName
FROM Customer.Contacts AS C WITH (nolock)
WHERE C.ContactFirstName LIKE 'a%'
Use Charindex which improves performance of the search ,Here it checks the string to match with first charcter of given search charecter and doesn't search for any more matches.
DECLARE #Search VARCHAR(10)='a'
SELECT A.ContactFirstName, A.ContactLastName
FROM Customer.CustomerDetails AS A WITH (NOLOCK)
WHERE CHARINDEX(#Search,(A.ContactFirstName + ' ' + A.ContactLastName),0)>1
Related
I need to try and optimize a query in Impala SQL that does partial string matches on about 60 different strings, against two columns in a database of 50+ billion rows. The values in these two columns are encrypted and have to be decrypted with a user defined function (in Java) to do the partial string match. So query would look something like:
SELECT decrypt_function(column_A), decrypt_function(column_B) FROM myTable WHERE ((decrypt_function(column_A) LIKE '%' + partial_string_1 + '%') OR (decrypt_function(column_B) LIKE '%' + partial_string_1 + '%')) OR ((decrypt_function(column_A) LIKE '%' + partial_string_2 + '%') OR (decrypt_function(column_B) LIKE '%' + partial_string_2 + '%')) OR ... [up to partial_string_60]
What I really want to do is decrypt the two column values I'm comparing with, once for each row and then compare that value with all the partial strings, then go onto the next row etc (for 55 billion rows). Is that possible somehow? Can there be a subquery that assigns the decrypted column value to a variable before using that to do the string comparison to each of the 60 strings? Then go onto the next row...
Or is some other optimization possible? E.g. using 'IN', so ... WHERE (decrypt_function(column_A) IN ('%' + partial_string_1 + '%', '%' + partial_string_2 + '%', ... , '%' + partial_string_60 + '%')) OR (decrypt_function(column_B) IN ('%' + partial_string_1 + '%', '%' + partial_string_2 + '%', ... , '%' + partial_string_60 + '%'))
Thanks
Use subquery and also regexp_like can have many patterns concatenated with OR (|), so you can check all alternatives in single regexp, though you may need to split into several function calls if the pattern string is too long:
select colA, ColB
from
(--decrypt in the subquery
SELECT decrypt_function(column_A) as colA, decrypt_function(column_B) as ColB
FROM myTable
) as s
where
--put most frequent substrings first in the regexp
regexp_like(ColA,'partial_string_1|partial_string_2|partial_string_3') --add more
OR
regexp_like(ColB,'partial_string_1|partial_string_2|partial_string_3')
In Hive use this syntax:
where ColA rlike 'partial_string_1|partial_string_2|partial_string_3'
OR ColB rlike 'partial_string_1|partial_string_2|partial_string_3'
I have a field in UI interface that passes to a stored procedure a null value (when field is unfilled) or a contract number when it is filled. Substrings of the contract number are accepted as input.
Inside the procedure, I need to filter the results by this parameter.
I need something similar to this:
SELECT * FROM tableName tn
WHERE
tn.ContractNumber LIKE ISNULL('%' + #contractNumber + '%', tn.ContractNumber)
What do you think it is the best approach? Problem is that using a condition like this does not return values.
Simply:
SELECT *
FROM tableName tn
WHERE tn.ContractNumber LIKE '%' + #contractNumber + '%'
OR #contractNumber IS NULL
You are really checking multiple condition, so having them separated reads more intuitive (for most people, anyway).
I assume this is just a sample query, and you are not selecting * in reality...
Another one:
SELECT *
FROM tableName tn
WHERE tn.ContractNumber LIKE '%' + ISNULL(#contractNumber, '%') + '%'
I am using SQL query shown below to compare AMCcode. But if I compare the AMCcode '1' using LIKE operator it will compare all the entries with AMCcode 1, 10,11,12,13. .. 19, 21,31.... etc. But I want to match the AMCcode only with 1. Please suggest how can I do it. The code is given below :
ISNULL(CONVERT(VARCHAR(10),PM.PA_AMCCode), '') like
(
CASE
WHEN #AMCCode IS NULL THEN '%'
ELSE '%'+#AMCCode+ '%'
END
)
This is part of the code where I need to replace the LIKE operator with any other operator which will give the AMCcode with 1 when I want to search AMCcode of 1, not all 10,11,12..... Please help
I think you are looking for something like this:
where ',' + cast(PM.PA_AMCCode as varchar(255)) + ',' like '%,' + #AMCCodes + ',%'
This includes the delimiters in the comparison.
Note that a better method is to split the string and use a join, something like this:
select t.*
from t cross apply
(select cast(code as int) as code
from dbo.split(#AMCCodes, ',') s(code)
) s
where t.AMCCode = s.code;
This is better because under some circumstances, this version can make use of an index on AMCCode.
If you want to exactly match the value, you don't need to use '%' in your query. You can just use like as below
ISNULL(CONVERT(VARCHAR(10),PM.PA_AMCCode), '') like
(
CASE
WHEN #AMCCode IS NULL THEN '%'
ELSE #AMCCode
END
)
Possibly you can remove case statement and query like
ISNULL(CONVERT(VARCHAR(10),PM.PA_AMCCode), '') like #AMCCode
Given two strings A and B, what is the fastest way to compare whether A is a substring of B or B is a substring of A?
A LIKE '%' + B + '%' OR B LIKE '%' + A + '%'
or
CHARIDNEX(A,B) <> 0 OR CHARINDEX(B,A) <> 0
I believe its the former because it doesnt calculate the location.
Question 1: is there a faster way to do it because I want to minimize the number of times B has to be used as B is a string I get by processing another column value.
As an additional note,
Basically I want to do something as follows with a column, C
SELECT
CASE WHEN A LIKE Processing(C) THEN 0
WHEN A LIKE '%' + PROCESSING(C) + '%' OR PROCESSING(C) LIKE '%' + A + '%' THEN LEN(A) - LEN(PROCESSING(C))
END AS Score
FROM #table
where A and C are columns in table, #table. As can be seen, the number of times I am calling Processing(C) is huge as it is done for each record.
Question 2: Should I put Processing(C) it in a separate temp table and then run substring check against that column or continue with the same approach.
My guess is that charindex() and like would have similar performance in this case. Don't hesitate to test which is faster (and report back on the results so we can all learn).
However, this particular optimization probably won't make a difference to the overall query. Your question may be an example of premature optimization.
Once upon a time, I thought that like performed worse than the comparable string operation. However, like is optimized in many databases, including SQL Server. As an example of the optimization, like is able to use indexes (when there is no wildcard or the wildcard is at the end). charindex() does not use indexes. If you are looking for matches at the beginning of the respective strings, then your query could possibly take advantage of indexes.
EDIT:
For your concern about PROCESSING(c), you might consider a subquery:
SELECT (CASE WHEN A LIKE Processing_C THEN 0
WHEN A LIKE '%' + Processing_C + '%' OR Processing_C LIKE '%' + A + '%'
THEN LEN(A) - LEN(Processing_C)
END) AS Score
FROM (select t.*, PROCESSING(C) as Processing_C
from #table
) t
I am trying to search inside a sentence using the string from another column in SQL. I have a large list of keywords and their rating on a scale from 1 to 6. I am trying to search through data pulled down from Twitter to see what people thought of certain movies (i.e. Wreck It Ralph) and give the movie an aggregate score. The problem I am having is that the data pulled from Twitter isn't broken up, but still in a full sentence. I can't figure out a way to search through the sentence for the keyword. Here is a query that works:
SELECT SUM(K.Score)
FROM wreck_it_ralph WIR, Keyword K
WHERE t_text LIKE '%fantastic%'
AND K.word = 'fantastic';
Here is a query I tried using concatenation, but was unsuccessful:
SELECT SUM(K.Score)
FROM Wreck_it_Ralph WIR, Keyword k
WHERE t_text LIKE ('%' + k.word + '%');
Different databases have different ways of concatenating strings. In Oracle or Postgres, try this:
SELECT SUM(K.Score)
FROM Wreck_it_Ralph WIR, Keyword k
WHERE t_text LIKE '%' || k.word || '%'
In SQL Server or Sybase:
SELECT SUM(K.Score)
FROM Wreck_it_Ralph WIR, Keyword k
WHERE t_text LIKE '%' + k.word + '%'
And in MySQL:
SELECT SUM(K.Score)
FROM Wreck_it_Ralph WIR, Keyword k
WHERE t_text LIKE concat('%', k.word, '%')
Do note that you have a problem with partial word matches. So, something "dislike" will match "like". You may want to look into the full text capabilities of whichever database you are using, because that takes into account punctuation, full words, and stop-lists.
MSSQL-:
SELECT SUM(K.Score)
FROM Wreck_it_Ralph WIR, Keyword k
WHERE t_text LIKE '%' + k.word + '%';