So I'm creating a query for a report that could have several optional filters. I've only included client and station here to keep it simple. Each of these options could be an include or an exclude and could contain NULL, 1, or multiple values. So I split the varchar into a table before joining it to the query.
This test takes about 15 minutes to execute, which... just won't do :p Is there a better way? We have similar queries written with dynamic sql, and I was trying to avoid that, but maybe there's no way around it for this?
DECLARE
#ClientsInc VARCHAR(10) = 'ABCD, EFGH',
#ClientsExc VARCHAR(10) = NULL,
#StationsInc VARCHAR(10) = NULL,
#StationsExc VARCHAR(10) = 'SomeStation'
SELECT *
INTO #ClientsInc
FROM dbo.StringSplit(#ClientsInc, ',')
SELECT *
INTO #ClientsExc
FROM dbo.StringSplit(#ClientsExc, ',')
SELECT *
INTO #StationsInc
FROM dbo.StringSplit(#StationsInc, ',')
SELECT *
INTO #StationsExc
FROM dbo.StringSplit(#StationsExc, ',')
SELECT [some stuff]
FROM media_order mo
LEFT JOIN #ClientsInc cInc WITH(NOLOCK) ON cInc.Value = mo.client_code
LEFT JOIN #ClientsExc cExc WITH(NOLOCK) ON cExc.Value = mo.client_code
LEFT JOIN #StationsInc sInc WITH(NOLOCK) ON sInc.Value = mo.station_name
LEFT JOIN #StationsExc sExc WITH(NOLOCK) ON sExc.Value = mo.station_name
WHERE ((#ClientsInc IS NOT NULL AND cInc.Value IS NOT NULL)
OR (#ClientsExc IS NOT NULL AND cExc.Value IS NULL)
)
AND ((#StationsInc IS NOT NULL AND sInc.Value IS NOT NULL)
OR (#StationsExc IS NOT NULL AND sExc.Value IS NULL)
)
First of all, I always tend to mention Erland Sommarskog's Dynamic Search Conditions in such cases.
However, you already seem to be aware of the two options: one is dynamic SQL. The other is usually the old trick and (#var is null or #var=respective_column). This trick, however, works only for one value per variable.
Your solution indeed seems to work for multiple values. But in my opinion, you are trying too hard to avoid dynamic sql. Your requirements are complex enough to guarantee it. And remember, usually, dynamic sql is harder for you to code, but easier for the server in complex cases - and this one certainly is. Making a performance guess is always risky, but I would guess an improvement in this case.
I would use exists and not exists:
select ...
from media_order mo
where
(
#ClientsInc is null
or exists (
select 1
from string_split(#ClientsInc, ',')
where value = mo.client_code
)
)
and not exist (
select 1
from string_split(#ClientsExc, ',')
where value = mo.client_code
)
and (
#StationsInc is null
or exists (
select 1
from string_split(#StationsInc, ',')
where value = mo.station_name
)
)
and not exist (
select 1
from string_split(#StationsExc, ',')
where value = mo.station_name
)
Notes:
I used buil-in function string_split() rather than the custom splitter that you seem to be using. It is available in SQL Server 2016 and higher, and returns a single column called value. You can change that back to your customer function if you are running an earlier version
as I understand the logic you want, "include" parameters need to be checked for nullness before using exists, while it is unnecessary for "exclude" variables
Related
This question has been asked many times on SO but I never quite found the answer to it, they are mostly solutions to avoid the problem altogether.
I'm working with SQL MS and I'm trying to build a query inside a function (for security reasons) that will either return a table or it's unnested version by country.
meaning that the function should either be
SELECT * FROM SALES AS S
or
SELECT
S.*,
C.Country,
C.CountryPercentage * S.AmountWithouthVAT as CountryValue
FROM SALES AS S
INNER JOIN CountryAllocation AS C ON S.CountryAllocationID = C.CountryAllocationID
(the fact that this join will make a single row into many rows is why I don't simply use the above one. And the reason why I don't make the join outside the function is because the person running the function will not have access to either of the tables. Also note that because of the way permissions in SQL Server work a dynamic query will require permission evaluation, meaning that is not a feasible option unless I'm to develop a structure around certificates)
So, now I got 2 problems:
The output table might or might not have the columns Country and CountryValue causing problems when defining the output type of the function
The actual way to have a function parameter to switch between the 2 versions of the table.
I've got a solution, but this code pains my eyes to look upon:
CREATE FUNCTION [dbo].[fn_I_view] (#Type int)
RETURNS #OutTable TABLE
(
SaleID int,
AmountWithouthVAT decimal(18, 2),
Country varchar(50),
AlocationPercentage decimal(18, 2)
)
AS
BEGIN
WITH
Out1 AS
(
SELECT
S.*,
NULL as Country,
NULL as AlocationPercentage
FROM Sales AS S
WHERE #Type = 1
),
Out2 AS
(
SELECT
S.*,
C.Country,
C.CountryPercentage * S.AmountWithouthVAT as CountryValue
FROM SALES AS S
INNER JOIN CountryAllocation AS C ON S.CountryAllocationID = C.CountryAllocationID
WHERE #Type = 2
)
INSERT INTO #OutTable
SELECT * FROM Out1
UNION ALL
SELECT * FROM Out2
RETURN
END
GO
so, I can't exactly fix the first problem, only worked around it by making SELECT * from [INV].[fn_I_ViewAllMyInvoices](1) still return those 2 extra columns with NULL and I didn't fix the second problem either, as I'm calculating both queries when I only needed 1 of them (and as you can expect this is a demo code, the real deal is way more complex)
Is there any way to improve this code?/solve the problem in a different way? performance, readability as well as maintenance improvements are all welcome
You don't need to calculate both. Just do:
BEGIN
IF #type = 1
BEGIN
INSERT INTO #OutTable
SELECT S.*, NULL as Country, NULL as AlocationPercentage
FROM Sales s;
END;
ELSE
BEGIN
INSERT INTO #OutTable
SELECT S.*, C.Country, C.CountryPercentage * S.AmountWithouthVAT as CountryValue
FROM SALES S JOIN
CountryAllocation C
ON S.CountryAllocationID = C.CountryAllocationID;
END;
RETURN;
END;
I have a stored procedure that needs to filter by a list of ids that are passed as a comma-delimited list (ie '1,2,3').
I want to apply a WHERE IN clause that will match those ids but ONLY if the variable contains anything (IS NOT NULL AND <> '').
Here's a simplified fiddle of the problem: http://sqlfiddle.com/#!18/5f6be/1
It's currently working for single and multiple ids. But when passing '' or NULL it should return everything but it's not returning anything.
The CTEs and pagination stuff is there for a reason, please provide a solution that doesn't change that.
Using DelimitedSplit8k_lead you could achive this by doing:
CREATE PROC YourProc #List varchar(8000) = NULL AS
BEGIN
SELECT {YourColumns}
FROM YourTable YT
OUTER APPLY dbo.DelimitedSplit8k_Lead(#List,',') DS
WHERE DS.item = YT.YourColumn
OR NULLIF(#List,'') IS NULL
OPTION (RECOMPILE);
END
The OUTER APPLY is used, as if NULL is passed the dataset won't be eliminated. the RECOMPILE is there, as it turns into a "Catch-all query" with the addition of handling the NULL.
Why not use JOIN instead of a subquery in the WHERE clause.
set #userIds = nullif(ltrim(#userIds),'')
select u.*
from Users u
left join string_split(#userIds,',') s on u.Id=s.value
where s.value is not null or #userIds is null
The old school method:
WHERE
(#userIds IS NULL OR #userIds = '' OR U.Id IN (SELECT * FROM STRING_SPLIT(#userIds, ',')))
Add OPTION (RECOMPILE) at the end for this to work and trim the plan.
Edit:
Based on comments, this one generates two table scans. It didn't make a difference in my LocalDb setup but don't rely on it regardless.
WHERE U.Id IN
(
SELECT * FROM STRING_SPLIT(#userIds, ',')
UNION ALL
SELECT U.Id WHERE NULLIF(#userIds, '') IS NULL
)
To start, take this snippet as an example:
SELECT *
FROM StatsVehicle
WHERE ((ReferenceMakeId = #referenceMakeId)
OR #referenceMakeId IS NULL)
This will fetch and filter the records if the variable #referenceMakeId is not null, and if it is null, will fetch all the records. In other words, it is taking the first one into consideration if #referenceMakeId is not null.
I would like to add a further restriction to this, how can I achieve this?
For instance
(ReferenceModelId = #referenceModeleId) OR
(
(ReferenceMakeId = #referenceMakeId) OR
(#referenceMakeId IS NULL)
)
If #referenceModelId is not null, it will only need to filter by ReferenceModelId, and ignore the other statements inside it. If I actually do this as such, it returns all the records. Is there anything that can be done to achieve such a thing?
Maybe something like this?
SELECT * FROM StatsVehicle WHERE
(
-- Removed the following, as it's not clear if this is beneficial
-- (#referenceModeleId IS NOT NULL) AND
(ReferenceModelId = #referenceModeleId)
) OR
(#referenceModeleId IS NULL AND
(
(ReferenceMakeId = #referenceMakeId) OR
(#referenceMakeId IS NULL)
)
)
This should do the trick.
SELECT * FROM StatsVehicle
WHERE ReferenceModelId = #referenceModeleId OR
(
#referenceModeleId IS NULL AND
(
#referenceMakeId IS NULL OR
ReferenceMakeId = #referenceMakeId
)
)
However, you should note that this types of queries (known as catch-all queries) tend to be less efficient then writing a single query for every case.
This is due to the fact that SQL Server will cache the first query plan that might not be optimal for other parameters.
You might want to consider using the OPTION (RECOMPILE) query hint, or braking down the stored procedure to pieces that will each handle the specific conditions (i.e one select for null variables, one select for non-null).
For more information, read this article.
If #referenceModelId is not null, it will only need to filter by
ReferenceModelId, and ignore the other statements inside it. If I
actually do this as such, it returns all the records. Is there
anything that can be done to achieve such a thing?
You can think of using a CASE for good short circuit mechanism
WHERE
CASE
WHEN #referenceModelId is not null AND ReferenceModelId = #referenceModeleId THEN 1
WHEN #referenceMakeId is not null AND ReferenceMakeId = #referenceMakeId THEN 1
WHEN #referenceModelId is null AND #referenceMakeId is null THEN 1
ELSE 0
END = 1
I've got table with few numbers inside (or even empty): #states table (value int)
And I need to make SELECT from another table with WHERE clause by definite column.
This column's values must match one of #states numbers or if #states is empty then accept all values (like there is no WHERE condition for this column).
So I tried something like this:
select *
from dbo.tbl_docs docs
where
docs.doc_state in(iif(exists(select 1 from #states), (select value from #states), docs.doc_state))
Unfortunately iif() can't return subquery resulting dataset. I tried different variations with iif() and CASE but it wasn't successful. How to make this condition?
select *
from dbo.tbl_docs docs
where
(
(select count(*) from #states) > 0
AND
docs.doc_state in(select value from #states)
)
OR
(
(select count(*) from #states)=0
AND 1=1
)
Wouldn't a left join do?
declare #statesCount int;
select #statesCount = count(1) from #states;
select
docs.*
from dbo.tbl_docs docs
left join #states s on docs.doc_state = s.value
where s.value is not null or #statesCount = 0;
In general, whenever your query contains sub-queries, you should stop for five minutes, and think hard about whether you really need a sub-query at all.
And if you've got a server capable of doing that, in many cases it might be better to preprocess the input parameters first, or perhaps use constructs such as MS SQL's with.
select *
from dbo.tbl_docs docs
where exists (select 1 from #states where value = doc_state)
or not exists (select 1 from #state)
Writing a stored procedure that will have multiple input parameters. The parameters may not always have values and could be empty. But since the possibility exists that each parameter may contain values I have to include the criterion that utilizing those parameters in the query.
My query looks something like this:
SELECT DISTINCT COUNT(*) AS SRM
FROM table p
WHERE p.gender IN (SELECT * FROM Fn_SplitParms(#gender)) AND
p.ethnicity IN (SELECT * FROM Fn_SplitParms(#race)) AND
p.marital_status IN (SELECT * FROM Fn_SplitParms(#maritalstatus))
So my problem is if #gender is empty(' ') the query will return data where gender field is empty when I really want to just ignore p.gender all together. I don't want to have to accomplish this task using IF/ELSE conditional statements because they would be too numerous.
Is there any way to use CASE with IN for this scenario? OR
Is there other logic that I'm just not comprehending that will solve this?
Having trouble finding something that works well...
Thanks!
Use or:
SELECT DISTINCT COUNT(*) AS SRM
FROM table p
WHERE
(p.gender IN (SELECT * FROM Fn_SplitParms(#gender)) OR #gender = '')
AND (p.ethnicity IN (SELECT * FROM Fn_SplitParms(#race)) OR #race = '')
AND (p.marital_status IN (SELECT * FROM Fn_SplitParms(#maritalstatus)) OR #maritalstatus = '')
You might also want to consider table-valued parameters (if using SQL Server 2008 and up) - these can sometimes make the code simpler, since they are treated as tables (which in your case, may be empty) and you can join - plus no awkward split function required.