I have this stored procedure which I am using to populate a user table. It seems slow because it is taking around avg. 6s to return the records. Is there anything else I can do to tweak this sproc to make it faster?
CREATE PROCEDURE dbo.usmGetPendingAuthorizations
(
#empCorpId char(8) = null,
#empFirstName char(30) = null,
#empLastName char(30) = null,
#accessCompletionStatus char(20) = null,
#ownerCorpId char(8) = null,
#reqCorpId char(8) = null,
#reqDate datetime = null,
#rowCount int = 100
)
AS BEGIN
SET ROWCOUNT #rowCount
SELECT
UPPER(LTRIM(RTRIM(pa.RequestorCorpId))) AS ReqCorpId,
UPPER(LTRIM(RTRIM(pa.AccessCompletionStatus))) AS AccessCompletionStatus,
UPPER(LTRIM(RTRIM(pa.Comment))) AS ReqComment,
UPPER(LTRIM(RTRIM(pa.ValidLoginInd))) AS ValidLoginInd,
UPPER(LTRIM(RTRIM(pa.OwnerCorpId))) AS OwnerCorpId,
UPPER(LTRIM(RTRIM(pa.UserTypeCode))) AS UserTypeCode,
UPPER(LTRIM(RTRIM(pa.SelectMethod))) AS SelectMethod,
pa.ExpirationDate AS ExpirationDate,
pa.RequestorDate AS ReqDate,
pa.BeginDate AS BeginDate,
pa.EndDate AS EndDate,
UPPER(LTRIM(RTRIM(pa.UserGroupTypeCode))) AS UserGroupTypeCode,
pa.SubsidiaryId AS SubsidiaryId,
UPPER(LTRIM(RTRIM(pa.EmployeeCorpId))) AS EmpCorpId,
emp.empKeyId AS EmpKeyId,
LTRIM(RTRIM(emp.firstName)) AS EmpFirstName,
LTRIM(RTRIM(emp.lastName)) AS EmpLastName
FROM
dbo.PendingAuthorization AS pa JOIN capmark..EmployeeDataExtract AS emp
ON
UPPER(LTRIM(RTRIM(pa.EmployeeCorpId))) = UPPER(LTRIM(RTRIM(emp.corporateId)))
WHERE
UPPER(LTRIM(RTRIM(pa.EmployeeCorpId))) LIKE ISNULL(UPPER(LTRIM(RTRIM(#empCorpId))), '%')
AND UPPER(LTRIM(RTRIM(emp.firstName))) LIKE ISNULL('%' + UPPER(LTRIM(RTRIM(#empFirstName))) + '%', '%')
AND UPPER(LTRIM(RTRIM(emp.lastName))) LIKE ISNULL('%' + UPPER(LTRIM(RTRIM(#empLastName))) + '%', '%')
AND pa.AccessCompletionStatus LIKE ISNULL(UPPER(LTRIM(RTRIM(#accessCompletionStatus))), '%')
AND pa.OwnerCorpId LIKE ISNULL(UPPER(LTRIM(RTRIM(#ownerCorpId))), '%')
AND pa.RequestorCorpId LIKE ISNULL(UPPER(LTRIM(RTRIM(#reqCorpId))), '%')
AND DATEDIFF(dd, pa.RequestorDate, CONVERT(VARCHAR(10), ISNULL(#reqDate, pa.RequestorDate), 101)) = 0
SET ROWCOUNT 0
END
The main problem is the liberal use of functions, especially in the join. Where functions are used in this way Sybase cannot take advantage of indexes on those fields. Take for example the join
ON
UPPER(LTRIM(RTRIM(pa.EmployeeCorpId))) = UPPER(LTRIM(RTRIM(emp.corporateId)))
Are all those trims and uppers really needed?
If you have dirty data stored - mixed case, with some leading and some trailing space, I suggest that you try to tighten up the way the data are stored and/or updated - don't allow such data to get in. Carry out a one-time scrub of the data to make all corporate Ids uppercase with no trailing or leading spaces.
Once you've got clean data you can add an index on corporateId column in the EmployeeDataExtract table (or rebuild it if one already exists) and change the join to
ON
pa.EmployeeCorpId = emp.corporateId
If you really can't ensure clean data in the PendingAuthorization table then you'd have to leave the functions wrapping on that side of the join, but at least the index on the emp table will be available for the optimiser to consider.
The use of LIKE with leading edge wildcards makes indexes unusable, but that may be unavoidable in your case.
It looks like the PendingAuthorization.RequestorDate field is used to select data only for one date - the one supplied in #reqDate. You could transform that part of the WHERE clause to a range query, then an index on the date field could be used.
To do that you would use just the date part of #reqDate (ignoring time of day) and then derive from that 'date+1'. These would be the values used. Whether this would help much depends on how many RequestorDate days are present in the PendingAuthorization table.
Although you seem never accepted any answer, I will try to help you :).
First, I would revise the WHERE clause. Instead of your LIKEs:
UPPER(LTRIM(RTRIM(pa.EmployeeCorpId))) LIKE ISNULL(UPPER(LTRIM(RTRIM(#empCorpId))), '%')
I would use this:
(UPPER(LTRIM(RTRIM(pa.EmployeeCorpId))) LIKE UPPER(LTRIM(RTRIM(#empCorpId)))) OR (#empCorpId IS NULL)
So, can you try the following WHERE clause instead of yours to see if there is any difference in performance?
WHERE
(UPPER(LTRIM(RTRIM(pa.EmployeeCorpId))) LIKE UPPER(LTRIM(RTRIM(#empCorpId))) OR #empCorpId IS NULL)
AND (UPPER(LTRIM(RTRIM(emp.firstName))) LIKE '%' + UPPER(LTRIM(RTRIM(#empFirstName))) + '%' OR #empFirstName IS NULL)
AND (UPPER(LTRIM(RTRIM(emp.lastName))) LIKE '%' + UPPER(LTRIM(RTRIM(#empLastName))) + '%' OR #empLastName IS NULL)
AND (pa.AccessCompletionStatus LIKE UPPER(LTRIM(RTRIM(#accessCompletionStatus))) OR #accessCompletionStatus IS NULL)
AND (pa.OwnerCorpId LIKE UPPER(LTRIM(RTRIM(#ownerCorpId))) OR #ownerCorpId IS NULL)
AND (pa.RequestorCorpId LIKE UPPER(LTRIM(RTRIM(#reqCorpId))) OR #reqCorpId IS NULL)
AND (DATEDIFF(dd, pa.RequestorDate, CONVERT(VARCHAR(10), ISNULL(#reqDate, pa.RequestorDate), 101)) = 0)
Second, in general, SELECT works faster if columns referenced in WHERE clause are indexed appropriately.
Then, DATEDIFF along with CONVERT in WHERE clause are not going to speed up your query either.
But, the main question is: how many rows are in the joined tables? Because 6 seconds could be not that bad. You can check/play with Query plan to find out any potential bottle-necks.
Related
In my database I am selecting from a field containing Dillon Brookfield. I tried searching for brookfield with the following statement thinking LIKE would select the data, but it didn't work.
SELECT *
FROM [Reports]
WHERE ([VisitorName] LIKE '%' + #VisitorName + '%'
OR [PlateNumber] = #PlateNumber)
I record people who have been banned from a property along with their plate (if known), but if their name is not put in exactly it doesn't return the value. What would be the best way to return similar results?
This is more like a comment, but there is code inside and the comment only allows one #.
What data type is your #PlateNumber? Is it CHAR or VARCHAR? it can make a difference. Like these tests.
DECLARE #char NCHAR(20) = 'space';
SELECT '%'+#char+ '%',IIF('space' Like '%'+#char+ '%', 1,0);
DECLARE #varchar NVARCHAR(20) = 'space';
SELECT '%'+#varchar+ '%',IIF('space' Like '%'+#varchar+ '%', 1,0);
The first SELECT will give you
'%space %',0. -- there will be 15 blank character between space and %
The second SELECT will give you
'%space%', 1.
Create fulltext index. Search names/addresses and other similar fields with fulltext search - it'll be faster and you can find all names - Dillon Brookfield vs Brookfield Dillon.
I am trying to optimize a Stored procedure which is slow at the moment. It takes few parameters to search on which can be null. The query inside the SP looks like this.
SELECT
*some fields from the table*
FROM
[hrmCase]
WHERE
BrkId = #BrkId
AND
(ChannelId IN ('TO','TD'))
AND
case
when #PsId is null then 1
when (#PsId is not null) and ((SELECT SUBSTRING(UPPER(DATENAME(MONTH, Created)),1,3) + CAST(PSId AS varchar) PSId) like ('%' + #PsId + '%')) then 1 else 0
end = 1
AND
case
when #ACaseId is null then 1
when (#ACaseId is not null) and (AId like ('%' + #ACaseId + '%')) then 1 else 0
end = 1
AND
case
when #DateCreated is null then 1
when (#DateCreated is not null) and (dbo.StripTime(Created) = dbo.StripTime(#DateCreated)) then 1 else 0
end = 1
AND
case
when #Clients is null then 1
when (#Clients is not null) and (Client like ('%' + #Clients + '%')) then 1 else 0
end = 1
Is this the best way to do it or should I build a dynamic query based on the input parameters like below
Declare #SQLQuery AS NVarchar(4000)
Declare #ParamDefinition AS NVarchar(2000)
Set #SQLQuery = 'SELECT *some fields from the table* FROM [hrmCase] WHERE (1=1)'
If #PsId Is Not Null
Set #SQLQuery = #SQLQuery + ' And (SELECT SUBSTRING(UPPER(DATENAME(MONTH, Created)),1,3) + CAST(PSId AS varchar) PSId) like (''%''' + #PsId + '''%'')'
etc..
Which of the above two queries deemed more professional and will be quicker. Please suggest if there is a better way of doing the same thing.
Cheers,
DS
SQL Server is very bad at dealing with parameters like this--we've had many cases where we get it working acceptably and then 4 months later, something changes and it turns into table scans (or worse--I've seen performance FAR worse than simple table scans) all over again. (We have a 1.5TB database on SSD on mirrored 40 core servers). The best solution we've found is to use separate stored procedures or to use dynamic SQL. Many "experts" frown on dynamic SQL, but the reality is that in any decent environment these days, recompilation of dynamic SQL is insignificant in terms of CPU use and performance delay, and it eliminates the types of issues you're seeing because you eliminate the conditionals in the WHERE clauses that confuse the query optimizer.
The leading wildcards also will result in table scans in every case. Fulltext indexing can do this type of search much more efficiently than regular SQL. Some flavors of SQL Server support fulltext indexing and queries and some do not.
Given two strings A and B, what is the fastest way to compare whether A is a substring of B or B is a substring of A?
A LIKE '%' + B + '%' OR B LIKE '%' + A + '%'
or
CHARIDNEX(A,B) <> 0 OR CHARINDEX(B,A) <> 0
I believe its the former because it doesnt calculate the location.
Question 1: is there a faster way to do it because I want to minimize the number of times B has to be used as B is a string I get by processing another column value.
As an additional note,
Basically I want to do something as follows with a column, C
SELECT
CASE WHEN A LIKE Processing(C) THEN 0
WHEN A LIKE '%' + PROCESSING(C) + '%' OR PROCESSING(C) LIKE '%' + A + '%' THEN LEN(A) - LEN(PROCESSING(C))
END AS Score
FROM #table
where A and C are columns in table, #table. As can be seen, the number of times I am calling Processing(C) is huge as it is done for each record.
Question 2: Should I put Processing(C) it in a separate temp table and then run substring check against that column or continue with the same approach.
My guess is that charindex() and like would have similar performance in this case. Don't hesitate to test which is faster (and report back on the results so we can all learn).
However, this particular optimization probably won't make a difference to the overall query. Your question may be an example of premature optimization.
Once upon a time, I thought that like performed worse than the comparable string operation. However, like is optimized in many databases, including SQL Server. As an example of the optimization, like is able to use indexes (when there is no wildcard or the wildcard is at the end). charindex() does not use indexes. If you are looking for matches at the beginning of the respective strings, then your query could possibly take advantage of indexes.
EDIT:
For your concern about PROCESSING(c), you might consider a subquery:
SELECT (CASE WHEN A LIKE Processing_C THEN 0
WHEN A LIKE '%' + Processing_C + '%' OR Processing_C LIKE '%' + A + '%'
THEN LEN(A) - LEN(Processing_C)
END) AS Score
FROM (select t.*, PROCESSING(C) as Processing_C
from #table
) t
I'm trying to understand if the above question is possible. I've been conceptually thinking about it, and basically what I'm looking to do is:
Specify keywords that may appear in a title. Lets use the two terms "Portfolio" and "Mike"
I'm hoping to generate a query that will allow for me to search for when Portfolio is contained within a title, or Mike. These two titles need not to be together.
For instance, if I have a title dubbed: "Portfolio A" and another title "Mike's favorite" I'd like both of these titles to be returned.
The issue I've encountered with using a LIKE statement is the following:
WHERE 1=1
and rpt_title LIKE ''%'+#report_title+'%'''
If I were to input: 'Portfolio,Mike' it would search for the occurrence of just that within a title.
EDIT: I should have been a bit more clear. I believe it's necessary for me to input my variable as 'Portfolio, Mike' in order for it to find the multiple values. Is this possible?
I'm assuming you could maybe use a charindex with a substring and a replace?
Yep, multiple Like statements with OR will work just fine -- just make sure you use the correct parentheses:
SELECT ...
FROM ...
WHERE 1=1
and (rpt_title LIKE '%Portfolio%'
or rpt_title LIKE '%Mike%')
However, I might suggest you look into using a full-text search.
http://msdn.microsoft.com/en-us/library/ms142571.aspx
I can propose a solution where you could specify any number of masks, without using multiple LIKE -
DECLARE #temp TABLE (st VARCHAR(100))
INSERT INTO #temp (st)
VALUES ('Portfolio photo'),('- Mike'),('blank'),('else'),('est')
DECLARE #delims VARCHAR(30)
SELECT #delims = '|Portfolio|Mike|' -- %Portfolio% OR %Mike% OR etc.
SELECT t.st
FROM #temp t
CROSS JOIN (
SELECT substr =
SUBSTRING(
#delims,
number + 1,
CHARINDEX('|', #delims, number + 1) - number - 1)
FROM [master].dbo.spt_values n
WHERE [type] = N'P'
AND number <= LEN(#delims) - 1
AND SUBSTRING(#delims, number, 1) = '|'
) s
WHERE t.st LIKE '%' + s.substr + '%'
I have this SQL Query that is not comparing properly so I commented it out the WHERE clause. When returning the AF.ActivityNote it always has 2 spaces after it no matter if I do RTRIM on it or not. I think those spaces are the issue that wont let the commented WHERE clause to properly match the string against userfield33.
SELECT CAST(UF.UserField33 AS NVARCHAR) , RTRIM(CAST(AF.ActivityNote AS NVARCHAR))
FROM [BCMTEST01].[dbo].[ActivityContacts] as AC INNER JOIN [BCMTEST01].[dbo].[ActivityFullView] as AF
ON AC.ActivityID = AF.ActivityID INNER JOIN [BCMTEST01].[dbo].[OpportunityExportView] as OP
ON AC.ContactID = OP.ContactServiceID INNER JOIN [BCMTEST01].[dbo].[UserFields] as UF
ON OP.ContactServiceID = UF.ContactServiceID
WHERE ContactID = 8376
--WHERE RTRIM(CAST(UF.UserField33 AS NVARCHAR) = RTRIM(CAST(AF.ActivityNote AS NVARCHAR))
ORDER BY ContactID ASC
First, you should always include the length when converting to nvarchar, varchar, char, and nchar. So use something like:
cast(uf.userfield33 as nvarchar(255)) -- or whatever length you like, so long as you have something
The last two characters are not spaces. Do something like:
select ascii(right( AF.ActivityNote, 1))
To see what the character value is. You can then use replace to get rid of it. Or, you can just chop off the last two characters (if that works for your application).
By the way, I am assuming you are using SQL Server based on the syntax of the query.
Here is an alternative where clause:
where (case when right(AF.ActivityNote, 2) = char(10)+CHar(13)
then LEFT(AF.ActivityNote, LEN(AF.ActivityNote) - 2)
else AF.ActivityNote
end) = CAST(UF.UserField33 AS NVARCHAR(255)
I'm not a big fan of case statements in where clauses. I would actually put the logic in a subquery. Also, I might have the order of the 10/13 backwards.