Re-use Query Parts in T-SQL - sql

I have a very complicated stored procedure that repeats a very complicated query with different where clauses based on certain values passed in. The stored procedure takes up over 500 lines of code, with the common part of the query taking up just over 100 lines. That common part is repeated 3 times.
I originally thought to use CTE (Common Table Expressions) except in T-SQL you can't define the common part, do your IF statement and then apply the WHERE clause. That's essentially what I need.
As a workaround I created a view for the common code, but it's only used in one stored procedure.
Is there any way to do this without creating a full view or temp tables?
Ideally I would like to do something like this:
WITH SummaryCTE (col1, col2, col3...)
AS
(
SELECT concat("Pending Attachments - ", ifnull(countCol1, 0)) col1
-- all the rest of the stuff
FROM x as y
LEFT JOIN attachments z on z.attachmentId = x.attachmentId
-- and more complicated stuff
)
IF (#originatorId = #userId)
BEGIN
SELECT * FROM SummaryCTE
WHERE
-- handle this security case
END
ELSE IF (#anotherCondition = 1)
BEGIN
SELECT * FROM SummaryCTE
WHERE
-- a different where clause
END
ELSE
BEGIN
SELECT * FROM SummaryCTE
WHERE
-- the generic case
END
Hopefully the pseudo code gives you an idea of what I would like. Right now my workaround is to create a view for the contents of what I defined SummaryCTE as, and then handle the IF/ELSE IF/ELSE clause. Executing this structure will throw an error at the first IF statement because the next command is supposed to be a SELECT instead. At least in T-SQL.
Maybe this doesn't exist in any other way, but I wanted to know for sure.

Well, aside from the temp tables and views that you've identified, you could go with dynamic SQL to build the code then execute it. This keeps you from having to repeat code, but makes it a bit hard to just deal with. Like this:
declare #sql varchar(max) = 'with myCTE (col1, col2) as ( select * from myTable) select * from myCTE'
if (#myVar = 1)
begin
#sql = #sql + ' where col1 = 2'
end
else if (#myVar = 2)
begin
#sql = #sql + ' where col2 = 4'
end
-- ...
exec #sql
Another option would be to incorporate your different where clauses into the original query.
WITH SummaryCTE (col1, col2, col3...)
AS
(
SELECT concat("Pending Attachments - ", ifnull(countCol1, 0)) col1
-- all the rest of the stuff
FROM x as y
LEFT JOIN attachments z on z.attachmentId = x.attachmentId
-- and more complicated stuff
)
select *
from SummaryCTE
where
(
-- this was your first if
#originatorId = #userId
and ( whatever you do for your security case )
)
or
(
-- this was your second branch
#anotherCondition = 1
and ( handle another case here )
)
or
-- etc. etc.
This eliminates the if/else chain but makes the query more complicated. It also can cause some bad cached query plans because of parameter sniffing, but that may not matter much depending on your data. Test that before making a decision. (You can also add optimizer hints to not cache the query plan. You won't get a bad one, but you also take a hit on every execution to create the query plan again. Test to find out, don't guess. Also, a solution with a view and the if/else chain will suffer from the same parameter sniffing/cached query plan problem.)

Related

UNION with a stored procedure

This is what I am trying to do
EXEC sp1 1
SELECT * FROM x
UNION
if(#num <= 1)
EXEC sp1(2)
else
null //want to return null to stop
I could do this is with a programming language but I don't have an idea what is that I am doing wrong with programming in SQL?
This, honestly, makes no sense, and I still suggest that you use an inline Table-value function here, instead of a procedure, but you can do something like this using OPENROWSET to return the dataset from a stored procedure within a SELECT statement. It can't be parametrised though (not in the traditional sense), and if you don't understand this, don't use it.
This is pseudo SQL as well as there's a lack of enough information to provide a complete solution, such as the columns needed in the SELECTs, but it might get you there if you can comprehend it:
EXEC dbo.sp1 1;
SELECT {Columns}
FROM dbo.x
UNION ALL
SELECT {Same Columns again} --This dataset's definition must be IDENTICAL to the above against your table dbo.x
FROM OPENROWSET('SQLNCLI', 'Server=localhost;Trusted_Connection=Yes;Database={YourDatabase}','EXEC dbo.sp1(2);') ORS; --Assumes you are using Windows Authentication
WHERE #Num <= 1
UNION ALL
SELECT NULL,NULL,NULL{,NULL...} --Until you have you enough NULL columns
Note that in this example I am using the deprecated SQLNCLI connection manager. You should really be using MSOLEDBSQL, however, the only instance I currently have access to with a trusted connection is a 2012 instance which doesn't have that driver installed; so I didn't want to post code that I hadn't minimally tested.
You can achieve this in SQL this way:
SELECT * into #temp FROM x
if(#num <= 1)
begin
insert into #temp
EXEC sp1(2)
select * from #temp
end
else
begin
select null
end
First you create a temp table and insert x table records into it and after that you check your condition and then insert records from procedure and then select * from #temp and other case it will return null.

Select all record from all the tables, every derived table must have its own alias

I'm working on a e-learning project in which there is a table named chapter in which there is a column named question_table this is table in which the specific chapter's questions are added.
Now the problem is I want to display all the question from all the chapter for this I used following sql query
SELECT * FROM (SELECT `question_table` FROM `chapter`)
but it doesn't work and gives the error:
"Every derived table must have its own alias".
Note: I want to do it using SQL not PHP.
Firstly, I think you would be better redesigning your database. Multiple tables of the same structure holding the same data are generally not a good idea.
However what you require is possible using a MySQL procedure to build up some dynamic SQL and then execute it, returning the resulting data.
A procedure as follows could be used to do this:-
DROP PROCEDURE IF EXISTS dynamic;
delimiter //
CREATE PROCEDURE dynamic()
BEGIN
DECLARE question_table_value VARCHAR(25);
DECLARE b INT DEFAULT 0;
DECLARE c TEXT DEFAULT '';
DECLARE cur1 CURSOR FOR SELECT `question_table` FROM `chapter`;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET b = 1;
OPEN cur1;
SET b = 0;
WHILE b = 0 DO
FETCH cur1 INTO question_table_value;
IF b = 0 THEN
IF c = '' THEN
SET c = CONCAT('SELECT * FROM `',question_table_value, '`');
ELSE
SET c = CONCAT(c, ' UNION SELECT * FROM `',question_table_value, '`');
END IF;
END IF;
END WHILE;
CLOSE cur1;
SET #stmt1 := c;
PREPARE stmt FROM #stmt1;
EXECUTE stmt;
END
This is creating a procedure called dynamic. This takes no parameters. It sets up a cursor to read the question_table column values from the chapter table. It looks around the results from that, building up a string which contains the SQL, which is a SELECT from each table with the results UNIONed together. This is then PREPAREd and executed. The procedure will return the result set from the SQL executed by default.
You can call this to return the results using:-
CALL dynamic()
Down side is that this isn't going to give nice results if there are no rows to return and they are not that easy to maintain or debug with the normal tools developers have. Added to which very few people have any real stored procedure skills to maintain it in future.
In MySQL you must give every subquery ("derived table") an alias:
SELECT * FROM (SELECT question_table FROM chapter) t --notice the alias "t"
The derived table here is the result of the (SELECT ...). You need to give it an alias, like so:
SELECT * FROM (SELECT question_table FROM chapter) X;
Edit, re dynamic tables
If you know all the tables in advance, you can union them, i.e.:
SELECT * FROM
(
SELECT Col1, Col2, ...
FROM Chapter1
UNION
SELECT Col1, Col2, ...
FROM Chapter2
UNION
...
) X;
SqlFiddle here
To do this solution generically, you'll need to use dynamic sql to achieve your goal.
In general however, this is indicative of a smell in your table design - your chapter data should really be in one table, and e.g. classified by the chapter id.
If you do need to shard data for scale or performance reasons, the typical mechanism for doing this is to span multiple databases, not tables in the same database. MySql can handle large numbers of rows per table, and performance won't be an issue if the table is indexed appropriately.

Return result of select query only if not empty

Working with Sql Server. Writing a stored procedure. Here is the pseudocode for what I want to achieve:
IF EXISTS ( SELECT field1
FROM t1
WHERE field1 = ... AND field2 = ...)
BEGIN
SELECT field1
FROM t1
WHERE field1 = ... AND field2 = ...
END
any better way of doing this? Any help appreciated.
Chirayu
Update: The problem is that the same query is executed twice. I cannot also just the run query once and return null (if the result is null i would like to return an alternative result).
I have done this before using a CTE and table variable, it requires more lines of code but the query is only written once, therefore your logic exists in a single place.
DECLARE #Results TABLE (Result INT);
WITH ResultsCTE AS
(
--Your query goes here
SELECT 1 as Result
WHERE 1 = 1
)
INSERT INTO #Results
SELECT Result
FROM ResultsCTE
IF (SELECT COUNT(*) FROM #Results) > 0
BEGIN
SELECT * FROM #Results
END
ELSE BEGIN
SELECT 'Do Something Else or Do Nothing!'
END
You could check ##ROWCOUNT after running the query once to determine whether or not to return the value:
http://msdn.microsoft.com/en-us/library/ms187316.aspx
If the select doesn't yield any results, no results will be returned. I don't see any reason to use a condition here, unless I'm missing something...
A stored procedure that sometimes returns a result while sometimes it doesn't would be a nightmare to use from any API. The client side API has different entry points depending on whether you return a result set (SqlCommand.ExecuteReader) or it does not return a result set (SqlCommand.ExecuteNonQuery). It would be impossible for the application to know ahead of time which API to use! Modeling tools use the SET FMTONLY option to analyze the metadata of returned result sets and the modeling tools are very confused when your returned result set start changing shape at random. In other words, you are down the wrong path, stop and turn around.
Just run the query, it no rows match your criteria it will simply return an empty result set. Which is exactly what every client API and modeling tool expects from your procedure.

Nested if statements in SQL Server stored procedure SELECT statement

I'm new here, and relatively new to stored procedures, so please bear with me! I've checked related questions on here and can't find anything that works in this instance.
I am trying to build a stored procedure (MS SQL Server 2005) that takes a number of passed in values and in effect dynamically builds up the SQL as you would with inline SQL.
This is where I've come unstuck.
We have (somewhat simplified for clarity):
#searchf1 varchar(100), -- search filter 1
#searchr1 varchar(100), -- search result 1
#searchf2 varchar(100), -- search filter 2
#searchr2 varchar(100), -- search result 2
#direction char(1), -- direction to order results in
AS
set nocount on
set dateformat dmy
SELECT *
FROM database.dbo.table T
WHERE T.deleted = 'n'
ORDER BY CASE #direction
WHEN 'A' THEN T.id
WHEN 'D' THEN T.id DESC
END
END
set nocount off
I have also tried the lines from ORDER BY as:
IF #direction = 'N' THEN
ORDER BY
T.id
ELSE
ORDER BY
T.id DESC
Both approaches give me an error along the lines:
"Incorrect syntax near the keyword 'DESC'." (which references the line id DESC following the final ORDER BY
As part of this stored procedure I also want to try to feed in matched pairs of values which reference a field to look up and a field to match it to, these could either be present or ''. To do that I need to add into the SELECT section code similar to:
WHERE
deleted = 'n'
IF #searchf1 <> '' THEN
AND fieldf1 = #searchf1 AND fieldr1 = #searchr1
This however generates errors like:
Incorrect syntax near the keyword 'IF'.
I know dynamic SQL of this type isn't the most elegant. And I know that I could do it with glocal IF ELSE statements, but if I did the SP would be thousands of lines long; there are going to up to 15 pairs of these search fields, together with the direction and field to order that direction on.
(the current version of this SP uses a passed in list of IDs to return generated by some inline dynamic SQL, through doing this I'm trying to reduce it to one hit to generate the recordset)
Any help greatly appreciated. I've hugely simplified the code in the above example for clarity, since it's the general concept of a nested IF statement with SELECT and ORDER BY that I'm inquiring about.
For this I would try to go with a more formal Dynamic SQL solution, something like the following, given your defined input parameters
DECLARE #SQL VARCHAR(MAX)
SET #SQL = '
SELECT
FROM
database.dbo.table T
WHERE
T.deleted = ''n'' '
--Do your conditional stuff here
IF #searchf1 <> '' THEN
SET #SQL = #SQL + ' AND fieldf1 = ' + #searchf1 + ' AND fieldr1 = ' + #searchr1 + ''' '
--Finish the query
SET #SQL = #SQL + ' ORDER BY xxx'
EXEC(#SQL)
DISCLAIMER: The use of Dynamic SQL is NOT something that should be taken lightly, and proper consideration should be taken in ALL circumstances to ensure that you are not open to SQL injection attacks, however, for some dynamic search type operations it is one of the most elegant route.
Try it this way:
SELECT * FROM database.dbo.table T WHERE T.deleted = 'n'
ORDER BY
CASE WHEN #direction='A' THEN T.id END ASC,
CASE WHEN #direction='D' THEN T.id END DESC
Source Article:
http://blog.sqlauthority.com/2007/07/17/sql-server-case-statement-in-order-by-clause-order-by-using-variable/
Another option that you might have, depending on the data type of your field, if nulls are NOT allowed, would be to do something like this.
SELECT *
FROM database.dbo.table T
WHERE T.deleted = 'n'
AND fieldf1 = COALESCE(#searchf1, fieldf1)
AND fieldr1 = COALESCE(#searchr1, fieldr1)
--ETC
ORDER BY fieldf1
This way you are not using dynamic SQL and it is fairly readable, just have the variable be null when you are looking to omit the data.
NOTE: As I mentioned this route will NOT work if any of the COALESCE columns contain null values.

Proper way to handle 'optional' where clause filters in SQL?

Let's say you have a stored procedure, and it takes an optional parameter. You want to use this optional parameter in the SQL query. Typically this is how I've seen it done:
SELECT * FROM dbo.MyTableName t1
WHERE t1.ThisField = 'test'
AND (#MyOptionalParam IS NULL OR t1.MyField = #MyOptionalParam)
This seems to work well, however it causes a high amount of logical reads if you run the query with STATISTICS IO ON. I've also tried the following variant:
SELECT * FROM dbo.MyTableName t1
WHERE t1.ThisField = 'test'
AND t1.MyField = CASE WHEN #MyOptionalParam IS NULL THEN t1.MyField ELSE #MyOptionalParam END
And it yields the same number of high reads. If we convert the SQL to a string, then call sp_ExecuteSQL on it, the reads are almost nil:
DECLARE #sql nvarchar(max)
SELECT #sql = 'SELECT * FROM dbo.MyTableName t1
WHERE t1.ThisField = ''test'''
IF #MyOptionalParam IS NOT NULL
BEGIN
SELECT #sql = #sql + ' AND t1.MyField = #MyOptionalParam '
END
EXECUTE sp_ExecuteSQL #sql, N'#MyOptionalParam', #MyOptionalParam
Am I crazy? Why are optional where clauses so hard to get right?
Update: I'm basically asking if there's a way to keep the standard syntax inside of a stored procedure and get low logical reads, like the sp_ExecuteSql method does. It seems completely crazy to me to build up a string... not to mention it makes it harder to maintain, debug, visualize..
If we convert the SQL to a string, then call sp_ExecuteSQL on it, the reads are almost nil...
Because your query is no longer evaluating an OR, which as you can see kills sargability
The query plan is cached when using sp_executesql; SQL Server doesn't have to do a hard parse...
Excellent resource: The Curse & Blessing of Dynamic SQL
As long as you are using parameterized queries, you should safe from SQL Injection attacks.
This is another variation on the optional parameter technique:
SELECT * FROM dbo.MyTableName t1
WHERE t1.ThisField = 'test'
AND t1.MyField = COALESCE(#MyOptionalParam, t1.MyField)
I'm pretty sure it will have the same performance problem though. If performance is #1 then you'll probably be stuck with forking logic and near duplicate queries or building strings which is equally painful in TSQL.
You're using "OR" clause (implicitly and explicitly) on the first two SQL statements. Last one is an "AND" criteria. "OR" is always more expensive than "AND" criteria. No you're not crazy, should be expected.
EDIT: Adding link to similar question/answer with context as to why the union / if...else approach works better than OR logic (FYI, Remus, the answerer in this link, used to work on the SQL Server team developing service broker and other technologies)
Change from using the "or" syntax to a union approach, you'll see 2 seeks that should keep your logical read count as low as possible:
SELECT * FROM dbo.MyTableName t1
WHERE t1.ThisField = 'test'
AND #MyOptionalParam IS NULL
union all
SELECT * FROM dbo.MyTableName t1
WHERE t1.ThisField = 'test'
AND t1.MyField = #MyOptionalParam
If you want to de-duplicate the results, use a "union" instead of "union all".
EDIT: Demo showing that the optimizer is smart enough to rule out scan with a null variable value in UNION:
if object_id('tempdb..#data') > 0
drop table #data
go
-- Put in some data
select top 1000000
cast(a.name as varchar(100)) as thisField, cast(newid() as varchar(50)) as myField
into #data
from sys.columns a
cross join sys.columns b
cross join sys.columns c;
go
-- Shwo count
select count(*) from #data;
go
-- Index on thisField
create clustered index ixc__blah__temp on #data (thisField);
go
set statistics io on;
go
-- Query with a null parameter value
declare #MyOptionalParam varchar(50);
select *
from #data d
where d.thisField = 'test'
and #MyOptionalParam is null;
go
-- Union query
declare #MyOptionalParam varchar(50);
select *
from #data d
where d.thisField = 'test'
and #MyOptionalParam is null
union all
select *
from #data d
where d.thisField = 'test'
and d.myField = '5D25E9F8-EA23-47EE-A954-9D290908EE3E';
go
-- Union query with value
declare #MyOptionalParam varchar(50);
select #MyOptionalParam = '5D25E9F8-EA23-47EE-A954-9D290908EE3E'
select *
from #data d
where d.thisField = 'test'
and #MyOptionalParam is null
union all
select *
from #data d
where d.thisField = 'test'
and d.myField = '5D25E9F8-EA23-47EE-A954-9D290908EE3E';
go
if object_id('tempdb..#data') > 0
drop table #data
go
Change from using the "or" syntax to a two query approach, you'll see 2 different plans that should keep your logical read count as low as possible:
IF #MyOptionalParam is null
BEGIN
SELECT *
FROM dbo.MyTableName t1
END
ELSE
BEGIN
SELECT *
FROM dbo.MyTableName t1
WHERE t1.MyField = #MyOptionalParam
END
You need to fight your programmer's urge to reduce duplication here. Realize you are asking for two fundamentally different execution plans and require two queries to produce two plans.