Is it possible to test for a column before selecting it within a select statement?
This may be rough for me to explain, I have actually had to teach myself dynamic SQL over the past 4 months. I am using a dynamically generated parameter (#TableName) to store individual tables within a loop (apologize for the vagueness, but the details aren't relevant).
I then want to be able to be able to conditionally select a column from the table (I will not know if each table has certain columns). I have figured out how to check for a column outside of a select statement...
SET #SQLQuery2 = 'Select #OPFolderIDColumnCheck = Column_Name From INFORMATION_SCHEMA.COLUMNS Where Table_Name = #TABLENAME And Column_Name = ''OP__FolderID'''
SET #ParameterDefinition2 = N'#TABLENAME VARCHAR(100), #OPFolderIDColumnCheck VARCHAR(100) OUTPUT'
EXECUTE SP_EXECUTESQL #SQLQuery2, #ParameterDefinition2, #TABLENAME, #OPFolderIDColumnCheck OUTPUT
IF #OPFolderIDColumnCheck IS NULL
BEGIN
SET #OP__FOLDERID = NULL
END
ELSE
IF #OPFolderIDColumnCheck IS NOT NULL
BEGIN
...etc
but id like to be able to do it inside of a select statement. Is there a way to check and see if OP__FOLDERID exists in the table?
Id like to be able to do something like this:
SELECT IF 'OP__FOLDERID' EXISTS IN [TABLE] THEN 'OP__FOLDERID' FROM [TABLE]
Thank you for any help or direction you can offer.
I'm afraid there isn't any direct way to do this within a SELECT statement at all. You can determine if a column exists in a table, however, and construct your dynamic SQL accordingly. To do this, use something like this:
IF COL_LENGTH('schemaName.tableName', 'columnName') IS NOT NULL
BEGIN
-- Column Exists
END
You could then set a variable as a flag, and the code to construct the dynamic SQL would construct the expression with/without the column, as desired. Another approach would be to use a string value, and set it to the column name if it is present (perhaps with a prefix or suffix comma, as appropriate to the expression). This would allow you to save writing conditionals in the expression building, and would be particularly helpful where you have more than one or two of these maybe-columns in a dynamic expression.
Related
It looks like #temptables created using dynamic SQL via the EXECUTE string method have a different scope and can't be referenced by "fixed" SQLs in the same stored procedure.
However, I can reference a temp table created by a dynamic SQL statement in a subsequence dynamic SQL but it seems that a stored procedure does not return a query result to a calling client unless the SQL is fixed.
A simple 2 table scenario:
I have 2 tables. Let's call them Orders and Items. Order has a Primary key of OrderId and Items has a Primary Key of ItemId. Items.OrderId is the foreign key to identify the parent Order. An Order can have 1 to n Items.
I want to be able to provide a very flexible "query builder" type interface to the user to allow the user to select what Items he want to see. The filter criteria can be based on fields from the Items table and/or from the parent Order table. If an Item meets the filter condition including and condition on the parent Order if one exists, the Item should be return in the query as well as the parent Order.
Usually, I suppose, most people would construct a join between the Item table and the parent Order tables. I would like to perform 2 separate queries instead. One to return all of the qualifying Items and the other to return all of the distinct parent Orders. The reason is two fold and you may or may not agree.
The first reason is that I need to query all of the columns in the parent Order table and if I did a single query to join the Orders table to the Items table, I would be repoeating the Order information multiple times. Since there are typically a large number of items per Order, I'd like to avoid this because it would result in much more data being transfered to a fat client. Instead, as mentioned, I would like to return the two tables individually in a dataset and use the two tables within to populate a custom Order and child Items client objects. (I don't know enough about LINQ or Entity Framework yet. I build my objects by hand). The second reason I would like to return two tables instead of one is because I already have another procedure that returns all of the Items for a given OrderId along with the parent Order and I would like to use the same 2-table approach so that I could reuse the client code to populate my custom Order and Client objects from the 2 datatables returned.
What I was hoping to do was this:
Construct a dynamic SQL string on the Client which joins the orders table to the Items table and filters appropriate on each table as specified by the custom filter created on the Winform fat-client app. The SQL build on the client would have looked something like this:
TempSQL = "
INSERT INTO #ItemsToQuery
OrderId, ItemsId
FROM
Orders, Items
WHERE
Orders.OrderID = Items.OrderId AND
/* Some unpredictable Order filters go here */
AND
/* Some unpredictable Items filters go here */
"
Then, I would call a stored procedure,
CREATE PROCEDURE GetItemsAndOrders(#tempSql as text)
Execute (#tempSQL) --to create the #ItemsToQuery table
SELECT * FROM Items WHERE Items.ItemId IN (SELECT ItemId FROM #ItemsToQuery)
SELECT * FROM Orders WHERE Orders.OrderId IN (SELECT DISTINCT OrderId FROM #ItemsToQuery)
The problem with this approach is that #ItemsToQuery table, since it was created by dynamic SQL, is inaccessible from the following 2 static SQLs and if I change the static SQLs to dynamic, no results are passed back to the fat client.
3 around come to mind but I'm look for a better one:
1) The first SQL could be performed by executing the dynamically constructed SQL from the client. The results could then be passed as a table to a modified version of the above stored procedure. I am familiar with passing table data as XML. If I did this, the stored proc could then insert the data into a temporary table using a static SQL that, because it was created by dynamic SQL, could then be queried without issue. (I could also investigate into passing the new Table type param instead of XML.) However, I would like to avoid passing up potentially large lists to a stored procedure.
2) I could perform all the queries from the client.
The first would be something like this:
SELECT Items.* FROM Orders, Items WHERE Order.OrderId = Items.OrderId AND (dynamic filter)
SELECT Orders.* FROM Orders, Items WHERE Order.OrderId = Items.OrderId AND (dynamic filter)
This still provides me with the ability to reuse my client sided object-population code because the Orders and Items continue to be returned in two different tables.
I have a feeling to, that I might have some options using a Table data type within my stored proc, but that is also new to me and I would appreciate a little bit of spoon feeding on that one.
If you even scanned this far in what I wrote, I am surprised, but if so, I woul dappreciate any of your thoughts on how to accomplish this best.
You first need to create your table first then it will be available in the dynamic SQL.
This works:
CREATE TABLE #temp3 (id INT)
EXEC ('insert #temp3 values(1)')
SELECT *
FROM #temp3
This will not work:
EXEC (
'create table #temp2 (id int)
insert #temp2 values(1)'
)
SELECT *
FROM #temp2
In other words:
Create temp table
Execute proc
Select from temp table
Here is complete example:
CREATE PROC prTest2 #var VARCHAR(100)
AS
EXEC (#var)
GO
CREATE TABLE #temp (id INT)
EXEC prTest2 'insert #temp values(1)'
SELECT *
FROM #temp
1st Method - Enclose multiple statements in the same Dynamic SQL Call:
DECLARE #DynamicQuery NVARCHAR(MAX)
SET #DynamicQuery = 'Select * into #temp from (select * from tablename) alias
select * from #temp
drop table #temp'
EXEC sp_executesql #DynamicQuery
2nd Method - Use Global Temp Table:
(Careful, you need to take extra care of global variable.)
IF OBJECT_ID('tempdb..##temp2') IS NULL
BEGIN
EXEC (
'create table ##temp2 (id int)
insert ##temp2 values(1)'
)
SELECT *
FROM ##temp2
END
Don't forget to delete ##temp2 object manually once your done with it:
IF (OBJECT_ID('tempdb..##temp2') IS NOT NULL)
BEGIN
DROP Table ##temp2
END
Note: Don't use this method 2 if you don't know the full structure on database.
I had the same issue that #Muflix mentioned. When you don't know the columns being returned, or they are being generated dynamically, what I've done is create a global table with a unique id, then delete it when I'm done with it, this looks something like what's shown below:
DECLARE #DynamicSQL NVARCHAR(MAX)
DECLARE #DynamicTable VARCHAR(255) = 'DynamicTempTable_' + CONVERT(VARCHAR(36), NEWID())
DECLARE #DynamicColumns NVARCHAR(MAX)
--Get "#DynamicColumns", example: SET #DynamicColumns = '[Column1], [Column2]'
SET #DynamicSQL = 'SELECT ' + #DynamicColumns + ' INTO [##' + #DynamicTable + ']' +
' FROM [dbo].[TableXYZ]'
EXEC sp_executesql #DynamicSQL
SET #DynamicSQL = 'IF OBJECT_ID(''tempdb..##' + #DynamicTable + ''' , ''U'') IS NOT NULL ' +
' BEGIN DROP TABLE [##' + #DynamicTable + '] END'
EXEC sp_executesql #DynamicSQL
Certainly not the best solution, but this seems to work for me.
I would strongly suggest you have a read through http://www.sommarskog.se/arrays-in-sql-2005.html
Personally I like the approach of passing a comma delimited text list, then parsing it with text to table function and joining to it. The temp table approach can work if you create it first in the connection. But it feel a bit messier.
Result sets from dynamic SQL are returned to the client. I have done this quite a lot.
You're right about issues with sharing data through temp tables and variables and things like that between the SQL and the dynamic SQL it generates.
I think in trying to get your temp table working, you have probably got some things confused, because you can definitely get data from a SP which executes dynamic SQL:
USE SandBox
GO
CREATE PROCEDURE usp_DynTest(#table_type AS VARCHAR(255))
AS
BEGIN
DECLARE #sql AS VARCHAR(MAX) = 'SELECT * FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = ''' + #table_type + ''''
EXEC (#sql)
END
GO
EXEC usp_DynTest 'BASE TABLE'
GO
EXEC usp_DynTest 'VIEW'
GO
DROP PROCEDURE usp_DynTest
GO
Also:
USE SandBox
GO
CREATE PROCEDURE usp_DynTest(#table_type AS VARCHAR(255))
AS
BEGIN
DECLARE #sql AS VARCHAR(MAX) = 'SELECT * INTO #temp FROM INFORMATION_SCHEMA.TABLES WHERE TABLE_TYPE = ''' + #table_type + '''; SELECT * FROM #temp;'
EXEC (#sql)
END
GO
EXEC usp_DynTest 'BASE TABLE'
GO
EXEC usp_DynTest 'VIEW'
GO
DROP PROCEDURE usp_DynTest
GO
DECLARE #a varchar(max);
set #a ='''a'' OR Name like ''%a'';';
--Why the below query not working
Select TOP 10 * FROM Member where Name = #a
-- The query below was executed to make sure that the query above
being constructed properly
print 'SQL: Select TOP 10 * FROM Member where Name ='+ #a
--SQL: Select TOP 10 * FROM Member where Name ='a' OR Name like '%a';
Correct me if im wrong, SQL injection wont work in Stored Procedure is due to some precompiled factor but the above scenario was tested in query statement instead of Stored Procedure. Why still not working?
I'm not sure why you think that would work. #a is a varchar variable, so Select TOP 10 * FROM Member where Name = #a finds rows where Name is equal to the value of that variable.
If you want SQL-Server to take the value of #a and insert it into the query as code, then you need to use sp_executesql (analogous to eval in languages like Bash and Python and JavaScript):
EXECUTE sp_executesql 'Select TOP 10 * FROM Member where Name = ' + #a
SQL Injection occurs when data is confused for and interpreted as code.
This does not happen in your scenario since parameter or variable values are not directly interpreted as code - they're only at risk of being interpreted as code if you construct new code by combining strings and these parameter/variable values and then pass the entire constructed string to the system and ask it to interpret the entire string as code - via exec, sp_executesql or other such means.
Look there is no name ending with 'a'. Try like
Select TOP 10 * FROM Member where Name ='a' OR Name like '%a%'
Updated
Microsoft handle SQL injection for SQL parameters.
I have a statement like this below in one of my big stored proc, and I am wondering if this is valid to write this way.
SELECT #PVDate = pv.Date,
#PVdMBeginDate = dbo.fPVoidDate(pv.myID)
FROM PVMeter pv (NOLOCK)
WHERE pv.PR_ID = #PR_ID
AND #VCommen BETWEEN pv.PVDMDate AND dbo.fPVoidDate(pv.myID)
Now, here, my question is, #VCommen is a declared date variable with a value set on it. It is not at all a column of PVMeter, while PVDMDate is a column in PVMeter and fpVoidDate returns datetime
While I debug SP, I do not see the value on #PVDate and #PVDMBeginDate
The original query is equivalent to:
SELECT
#PVDate = pv.Date,
#PVdMBeginDate = dbo.fPVoidDate(pv.myID)
FROM PVMeter pv (NOLOCK)
WHERE
pv.PR_ID = #PR_ID
AND pv.PVDMDate <= #VCommen
AND #VCommen <= dbo.fPVoidDate(pv.myID)
This style should be more familiar. In general, you can put any expression in the WHERE clause, it can be made of variables or constants without referring to table columns at all.
Examples
Classic example: WHERE 1=1 ... when the query text is generated dynamically. It is easy to add as many expressions as needed in no particular order and prepend all of them with AND.
DECLARE #VarSQL nvarchar(max);
SET #VarSQL = 'SELECT ... FROM ... WHERE 1=1 ';
IF ... THEN SET #VarSQL = #VarSQL + ' AND expression1';
IF ... THEN SET #VarSQL = #VarSQL + ' AND expression2';
IF ... THEN SET #VarSQL = #VarSQL + ' AND expression3';
EXEC #VarSQL;
Thus you don't need to have complex logic determining whether you need to add an AND before each expression or not.
Another example.
You have a stored procedure with parameter #ParamID int.
You have a complex query in the procedure that usually returns many rows and one column of the result set is some unique ID.
SELECT ID, ...
FROM ...
WHERE
expression1
AND expression2
AND expression3
...
You want to return all rows if #ParamID is NULL and only one row with the given ID if #ParamID is not NULL. I personally use this approach. When I open the screen with the results of a query for the first time I want to show all rows to the user, so I pass NULL as a parameter. Then user makes changes to a selected row, which is done through a separate UPDATE statement. Then I want to refresh results that user sees on the screen. I know ID of the row that was just changed, so I need to requery just this row, so I pass this ID to procedure and fetch only one row instead of the whole table again.
The final query would look like this:
SELECT ID, ...
FROM ...
WHERE
(#ParamID IS NULL OR ID = #ParamID)
AND expression1
AND expression2
AND expression3
...
OPTION (RECOMPILE);
Thus I don't have to repeat the complex code of the query twice.
I have a stored procedure with default values, which I set to NULL. In the WHERE clause, for each value specified, rather than NULL or not given (where it would also be NULL), I would like the column to equal this value, otherwise, I would like to search without this column being in the WHERE clause. The basic structure looks something like this:
// Set up the stored procedure
USE Table;
GO
CREATE PROCEDURE dbo.SearchTable
// Specify value(s)
#Name varchar(50) = NULL
AS
// Set up the query
IF(#Name IS NOT NULL
SELECT * FROM Table WHERE Name=#Name;
ELSE
SELECT * FROM Table
BEGIN
END
I have more than 1 parameter, and so, with this logic, I would need 2 IF's for every parameter. As you can imagine, this gets boring, time-consuming, and error-prone fast.
I would like to move this logic into the query, preferably into the WHERE clause, but any way I have found will cause errors (besides exceptions, which would require just as many IF's). SQL Server doesn't like IF's in the WHERE clause as far as I know, and with CASE I would have to specify the column, which I do not want to do.
What should I do?
Edit:
I have SQL Server version 2012, so please concentrate on this or any recent versions in your answer.
If you don't care about performance, you can do:
SELECT *
FROM Table
WHERE #Name is null or Name = #Name;
Often, having an or condition gets in the way of efficient use of indexes. Perhaps this isn't a problem in your case, though.
You could do something like this. The downside to this is that indexes may not be used properly and thus the performance may not be great.
SELECT * FROM Table
WHERE (#Name Is Null Or Name = #Name)
And (#Col2 Is Null Or Col2 = #Col2)
And (#Col3 Is Null Or Col3 = #Col3)
Each column condition is "anded". Or is used to apply that column condition only if #var is not null. So for example, if this is called with just #Name populated, it is equivalent to Where Name = #Name. If both #Name and #Col2 are populated, it is equivalent to Where Name = #Name And Col2 = #Col2.
I have imported a table to my database, and to get the number of columns and assign it to a variable I do
SELECT #HowManyColumns = COUNT(COLUMN_NAME)
FROM INFORMATION_SCHEMA.COLUMNS
WHERE (TABLE_NAME = #table_name )
But it keeps telling that the count is 0!
If I do the same for other tables it works!
I have found that the column count of the table that is not working is more than 40 columns, Why is it not working...
The data is this
Since tables w/o columns don't exists, that can only mean that the WHERE clause is not satisfied. In other words, the table named as the value of #table_name does not exists. Since you say 'sometimes it work, and some does not' that would immediately point toward case sensitive deployments. Make sure you always use the correct name for the table, with the proper case, so your code work correctly on servers which are deployed with a case sensitive collation.
Try issuing
SELECT DISTINCT TABLE_NAME
FROM INFORMATION_SCHEMA
and check table name for case.
Try having the table name as a text string instead:
DECLARE
#table_name varchar(50),
#noOfColumns int
SET
#table_name = 'table_name'
SET
#noOfColumns =
(SELECT count(*)
FROM INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = #table_name)
PRINT #noOfColumns
The answer you get is the number of columns in the #noOfColumns variable