I have a stored procedure that can get the number of records in a table, in which the #tableName is the parameter of the stored procedure. Let's call it FastCount:
SELECT OBJECT_NAME(object_id), SUM(row_count) AS rows
FROM sys.dm_db_partition_stats
WHERE object_id = OBJECT_ID(#tableName)
AND index_id < 2
GROUP BY OBJECT_NAME(object_id);
Now, let's say I have 50 tables, like data_1950, data_1951, .....data_2000. I wrote a batch, query each table's records count, and put them into a temporary table. It works like a charm
CREATE TABLE #Temp
(
TableName varchar(30),
RecordsCount int
)
DECLARE #sql as varchar(max)
DECLARE #yearN as int = 1950
DECLARE #tbName as sysname
WHILE #yearN <= 2000
BEGIN
SET #tbName = QUOTEName(N'[dbo].data_' + Convert(varchar,#yearN))
SET #sql = N'Exec [dbo].FastCount #tableName=' + #tbName
INSERT INTO #Temp
EXEC (#sql)
SET #yearN = #yearN + 1
END
SELECT * FROM #Temp
DROP TABLE #Temp
However, if I replace the dynamic SQL string part
SET #sql = N'Exec [dbo].FastCount #tableName=' + #tbName
INSERT INTO #Temp
EXEC (#sql)
with a straightforward call
INSERT INTO #Temp
EXEC [dbo].FastCount #tableName = #tbName
Then the whole batch just not work...
So I don't understand why... Should I always use dynamic SQL string and exec(#sql) when programmatically using the stored procedure. A big thanks for taking the time to look.
OK, here is what is happening in the two scenarios that you posed in your original question. (Yes, the reality is that there are probably better ways to achieve your end result, but let's look at the actual problem that you posed .... why is the behaviour of your INSERT / EXEC different, depending on how you made the call).
First, you have your variable declared, that will contain your table name:
DECLARE #tbName as sysname
Then you have your looping block that incrementally increases the year number, to generate the different table names. There's nothing inherently wrong with the looping block, so let's just look at an example using one of the table names, to see what's happening within the WHILE block. Take the first table name as the example, which would be [dbo].data_1950.
Your statement:
set #tbName = QUOTEName(N'[dbo].data_' + Convert(varchar,#yearN))
ultimately takes the string "[dbo].data_1950" - which comes from concatenating '[dbo].data_' with the year number (in this case, 1950) converted to a string (varchar) - and passes it to the QUOTENAME() function. The QUOTENAME() function takes its input and a second parameter, which is the character that the input should be quoted with (if the 2nd parameter is not passed, then the default is []). Thus, if we then converted the #tbName variable to a string, it would appear like this:
[[dbo].data_1950]
Now we get to see the funky way that SQL deals with "sysname" data-types. (In fact, as you read further down, maybe the issue is not primarily tied to the "sysname" data-type, but anyhow, take away from this what you will). To be honest, "sysname" is, in itself, a little bit of a funky data-type anyway, which I tend to steer away from, unless absolutely necessary. But anyhow, on to the details of the issue that you were seeing.
Step 1 - I created a version of your stored proc, but I included a statement that would output the value of the #tableName parameter that was passed in. This gives us an opportunity to see what SQL is doing in the two different scenarios, and then explain why the results are different.
CREATE PROC [dbo].FastCount
(
#tableName varchar(100)
)
AS
BEGIN
PRINT #tableName;
SELECT OBJECT_NAME(object_id), SUM(row_count) AS rows
FROM sys.dm_db_partition_stats
WHERE object_id = OBJECT_ID(#tableName)
AND index_id < 2
GROUP BY OBJECT_NAME(object_id);
END
Step 2 - our first scenario is executing the dynamic SQL.
set #tbName = QUOTEName(N'[dbo].data_' + Convert(varchar,#yearN))
set #sql = N'Exec [dbo].FastCount #tableName=' + #tbName
Insert Into #Temp Exec(#sql)
Now, we know that the #tbName variable contains
[[dbo].data_1950]
and therefore we can then infer that the #sql variable contains
Exec [dbo].FastCount #tableName=[[dbo].data_1950]
so that is effectively the statement that is executed by the Exec(#sql) command.
When this runs, and we look at the output of the PRINT command, we see
[dbo].data_1950
and we see a result from our query (the table name and row count). This makes sense, of course, because our table name is "data_1950", and the schema of the table is "dbo", so the SELECT statement to get the row count is going to work as expected.
Step 3 - run the EXEC command directly, without the use of the #sql variable, ie.
Insert Into #Temp Exec [dbo].FastCount #tableName = #tbName
Now, when we look at the output of the PRINT command for this execution of the "FastCount" stored procedure, we see
[[dbo].data_1950]
Of course, this is now NOT going to produce the results that we expect, because we're telling SQL to find the row count for a table named "[dbo].data_1950" (in the absence of the specific schema, SQL will just assume the default schema. In this case, with a schema of [dbo], we'd be telling SQL to get the row count from a table named [dbo].[[dbo].data_1950] - which is clearly NOT the table name).
You should see the obvious difference - in one scenario, the parameter value that is passed into the stored is the "correct" reference to the table name, and in the other scenario it is not.
As a final step, let's look at how the "non-dynamic" SQL would be executed, to achieve the results that we need. In this instance, there's no need for the QUOTENAME() function:
set #tbName = N'[dbo].data_' + Convert(nvarchar,#yearN)
Insert Into #Temp Exec [dbo].FastCount #tableName = #tbName
When we run it in this way, we see the expected output ([dbo].data_1950) from the PRINT command, and we see the expected query results (containing the table name and row count).
Can I explain this behaviour, exactly? Errr, not necessarily ... maybe someone else will be able to explain specifically what is happening, and why. My only interpretation is that when the EXEC() statement is passed the dynamic sql (ie. #sql variable) it is first interpreting the entire string and stripping out identifiers (in the case, the surrounding [] ... on what basis is it making that decision, I don't know). As opposed to the non-dynamic execution, where the #tbName value ([[dbo].data_1950]) is just being passed straight in as the parameter, with no modification (and thus causing the unexpected end result that we saw).
Hopefully this information is useful to you (or, at least, to someone else in the future!).
In general you should avoid dynamic SQL, and you should avoid granting rights to execute dynamic SQL, unless absolutely necessary. This is for performance and security reasons.
The typical way to deal with this situation is to use a partitioned view:
CREATE VIEW DataView
AS
SELECT '1950' TableName, * FROM Data_1950
UNION ALL
SELECT '1951' TableName, * FROM Data_1951
UNION ALL
SELECT '1952' TableName, * FROM Data_1952
UNION ALL
SELECT '1953' TableName, * FROM Data_1953
UNION ALL
SELECT '1954' TableName, * FROM Data_1954
UNION ALL
SELECT '1955' TableName, * FROM Data_1955
(Keep adding select statements until you have covered all of your tables.)
Now to get your table counts all you need to do is execute this:
SELECT TableName, COUNT(*) RecordCount
FROM DataView
GROUP BY TableName
Isn't that much easier?
I am currently using code similar to the following:
SET #Query = 'SELECT * INTO #Temp FROM MyTable'
EXEC sp_executesql #Query
Obviously, this does not work. Until now, I have been getting around this problem by using global temporary tables within my dynamic sql. However, this is not an option once the stored procedure is released to users, as their global temp tables would conflict with one another.
How might I get around this issue? I must specify that the structure of the table MyTable is unknown, so creating the temp table outside of the dynamic sql is (presumably?) not an option.
Edit: Apologies if I was not specific enough with exactly what I am trying to achieve - I thought it would be best to keep this question as relevent to the actual issue as possible.
Obviously, the string that I am trying to execute is not fixed. It is instead constructed so that the table, from which the data is being retrieved, may be specified. Although still not overly complex, the actual query string that I am using is more along the lines of
#Query = CONCAT('SELECT * INTO #Temp FROM ', #Environment, '.[schema].', #Table)
Hopefully this sheds more light on the problem?
Few points:
1. If you want to store the output of dynamic sql into temp table, you need to create the temp table structure first(with CREATE TABLE #TEMP script) and then you can insert data by doing something like below:
INSERT INTO #TEMP EXEC sp_executesql #Query
This will populate data into temp table with all properties of local temp table.
Seeing your query, it does not look like you have a need of dynamic sql unless you are forming some clause like where clause dynamically.
Let me know if this helps.
Couldn't you do something like this:
declare #query nvarchar(4000) = '
IF EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N''[dbo].[temp]'') AND type in (N''U''))
begin
drop table temp
end
Select top 2 *
into temp
from sys.databases
'
exec sp_executesql #query
Select * from temp
With Select into you don't need to know the structure of your query:
https://www.w3schools.com/sql/sql_select_into.asp
Just as an idea.
I need a small help. I want to convert my result of SQL into single row.
Lets say there is a table Students with ID and Name in it.
if I execute query
select * from Students
it returns.
Col1 Col2
1 Rizwan
2 Ahmed
I want result to be like
1 Rizwan 2 Ahmed
Please note that I want every record in a separate column.
Thanks in advance
I can't think of a plausible scenario where this transform serves any useful purpose because relational-algebra is, by design, about sets of data sharing the same attributes (i.e. tables have rows with columns) - by having everything in a single row with meaningless columns you're just effectively serializing data into a blob.
The only way to achieve this is using Dynamic SQL, as this is the only way to achieve a dynamic number of columns without prior knowledge of what columns are desired.
In MS SQL Server you might think of using PIVOT/UNPIVOT but the columns still need to be manually named, thus requiring Dynamic SQL.
MySQL Server (and MariaDB) have GROUP_CONCAT which can be used to combine multiple rows into a single string (text) value but the server lacks any kind of "split" function. So GROUP_CONCAT doesn't work here because it doesn't return discrete columns.
In T-SQL (MS SQL Server, Sybase) you need to iterate over every target row, this is done using a CURSOR. You cannot reliably perform string concatenation inside a SELECT statment:
DECLARE #sql nvarchar(max) = 'SELECT '
DECLARE c CURSOR FOR
SELECT [Id], [Name] FROM Students ORDER BY [Id] ASC
OPEN c
DECLARE #id int
DECLARE #name nvarchar(100)
FETCH NEXT FROM c INTO #id, #name
WHILE ##FETCH_STATUS = 0
BEGIN
SET #sql = #sql + CONVERT( varchar(10), #id ) + ', ' + #name
FETCH NEXT FROM c INTO #id, #name
END
CLOSE c
DEALLOCATE c
sp_executesql #sql -- this will execute the `SELECT` that was generated, where each discrete value will be returned as an anonymous column.
I have this query in SQL Server:
select column
from table_53;
Now, I want to get this 53 from another table, so what I want to do is something like this:
select column
from table_(select id from table2);
Is there any way to do this in SQL Server?
This is definitely not the way SQL thinks and works. Maybe your suggested approach can be mimicked by way of writing stored procedures in which you create SQL-statements which are then evaluated. However, this will not be very efficient.
A better approach would be to store the values of all your individual separate tables into one master table and mark them in a separate column tblid with their number (e.g. 53). Then you can always filter them from this master table by looking for this tblid.
You need dynamic sql query here.
declare #sqlQuery = 'select column
from table_(';
set #sqlQuery = #sqlQuery + 'select id from table2)';
EXEC (#sqlQuery)
Note :- One of cons of using dynamic sql query is sql injection. I would suggest to have better table structure or try to used parameterized query.
Yes, you can, but using something like this:
DECLARE #ID INT;
DECLARE #QUERY NVARCHAR(MAX);
SELECT #ID = ID FROM TABLE_2;
--IF #ID EQUALS 53 THEN
SET #QUERY = 'SELECT COLUMN FROM TABLE_' + CAST(#ID AS NVARCHAR(10));
-- #QUERY EQUALS TO 'SELECT COLUMN FROM TABLE_53'
EXEC (#QUERY);
I have been researching this for a couple of days and feel like I am going around in circles. I have basic knowledge of SQL but there are many areas I do not understand.
I have a table that stores the names and fields of all the other tables in my database.
tblFields
===================================================
TableName FieldName BookmarkName
---------------------------------------------------
Customer FirstName CustomerFirstName
Customer LastName CustomerLastName
Customer DOB CustomerDOB
I want to write a SELECT statement like the following but i am unable to get it work:
SELECT (SELECT [FieldName] FROM [TableName]) FROM tblFields
Is this possible? The application I have developed requires this for user customization of reports.
If i understand what you are trying to do, i think this will help you. It is not pretty and it works for SQL Server 2005 and above, but maybe this is what you are looking for:
declare #tableName nvarchar(100)
declare #sqlQuery nvarchar(max)
declare #fields varchar(500)
set #tableName = 'YourTableName'
set #fields = ''
select #fields = #fields + QUOTENAME(t.fieldname) + ',' from (
select distinct fieldname from tblfields where tablename = #tableName)t
set #sqlQuery = 'select ' + left(#fields, LEN(#fields)-1) + ' from ' + QUOTENAME(#tableName)
execute sp_executesql #sqlQuery
Edit: As Martin suggested, i edited so that the columns and tablename are using QUOTENAME
If I understand correctly what you are trying to do, you are probably better off doing this as two separate queries from your program. One which gets the fields you want to select which you then use in your program to build up the second query which actually gets the data.
If it must be done entirely in SQL, then you will need to tell us what database you are using. If it is SQL Server, you might be able to user a cursor over the first query to build up the second query which you then execute with the sp_executesql stored procedure. But doing doing it outside of SQL would be recommended.