I have 2 stored procedures which returns same but unknown columns. I need to write a proc to combine results from both stored procedures. I tried OPENROWSET but problem is to provide the connection string in OPENROWSET function, even if I specify the connection string one time, it will be different for different environments and I think that will be the worst thing to change connection string each time I deploy the application in different environments or if the user is changed on server. Can someone help me to get this done in the best way.
I cannot write them as function since the procs are using temp tables.
Declare #connection nvarchar(200)
Declare #sql nvarchar(max)
Set #connection= 'Server=servername;initial
catalog=dbname;user=abc,password=xyz';
Set #sql='SELECT * INTO #temp1
FROM OPENROWSET(
''SQLNCLI'',
'''+ #connection + ''',
''EXEC sp_name '')'
Exec(#sql)
--- creating a temporary table
CREATE Table #Dynamic_Temp_Table (_field_only_for_create_ INT )
--- Addition of fields from the first recordset from the first procedure
DECLARE #SQL NVARCHAR(MAX)
SELECT #SQL=ISNULL(#SQL+',','ALTER TABLE #Dynamic_Temp_Table ADD ')+name+' '+system_type_name
FROM sys.dm_exec_describe_first_result_set('exec sp_proc_first', NULL, NULL)
order by column_ordinal
exec sp_executesql #SQL
--- Remove of the first unused column
ALTER TABLE #Dynamic_Temp_Table drop column _field_only_for_create_
--- Addition of the result from the first procedure
INSERT INTO #Dynamic_Temp_Table
exec sp_proc_first
--- Addition of the result from the second procedure
INSERT INTO #Dynamic_Temp_Table
exec sp_proc_second
--- result: exec sp_proc_first UNION ALL exec sp_proc_second
select * from #Dynamic_Temp_Table
--- result: exec sp_proc_first UNION exec sp_proc_second
select DISTINCT * from #Dynamic_Temp_Table
It is possible but not easy at all....
You can change your stored procedures to just create and populate a global temp tables (no select), then you can select both with an union.
To use OPENROWSET as you are doing it in your current approach you will need global temp table too, But as you mention that your stored procedures are using Temp Tables, OPENROWSET, OPENQUERY or sys.dm_exec_describe_first_result_set will not be able to determine the metadata to create the temp table.
Another option is to change your stored procedures to use variable tables instead of temp tables, then the metadata could be redetermined. #chrszcpl's answer https://stackoverflow.com/a/55632401/10932521 is a very good solution if you are able to do that.
If this is not possible (I assume that this isn't, otherwise the columns would't be unknown) because you are using dynamic sql in your procedures, or you simply can't touch those procedures for any reason, I think that the cheapest solution is create a third stored procedure which returns the dynamic columns definition that the other procedures will return..
Related
I have a stored procedure, usp_region and it has a select statement with 50 columns as the result set. This procedure is called by multiple other stored procedures in our application.
Most of the stored procedure pass a parameter to this procedure and display the result set that it returns. I have one stored procedure, usp_calculatedDisplay, that gets the columns from this stored procedure and inserts the values into a temp table and does some more calculations on the columns.
Here's a part of the code in usp_calculatedDisplay.
Begin Procedure
/* some sql statements */
Declare #tmptable
(
-- all the 50 columns that are returned from the usp_region procedure
)
Insert Into #tmptable
exec usp_region #regionId = #id
Select t.*, /* a few calculated columns here */
From #tmptable t
End of procedure
Every time I add a column to the usp_region procedure, I'll also have to make sure I have to add it to this procedure. Otherwise it breaks. It has become difficult to maintain it since it is highly possible for someone to miss adding a column to the usp_calculatedDisplay procedure when the column is added to the usp_region.
In order to overcome this problem, I decided to do this:
Select *
Into #tmptable
From OPENROWSET('SQLNCLI',
'Server=localhost;Trusted_Connection=yes;',
'EXEC [dbo].[usp_region]')
The problem is 'Ad Hoc Distributed Queries' component is turned off. So I can't use this approach to overcome this issue. I was wondering if there are any other ways of overcoming this problem. I would really appreciate any help. Thank you!
Every time I add a column to the usp_region procedure
SQL Server is a structured database and it does not meant to solve such cases that you need to change your structure every day.
If you add/remove columns so often then you probably did not choose the right type of database, and you better re-design your system.
It has become difficult to maintain it since it is highly possible for someone to miss adding a column to the usp_calculatedDisplay procedure when the column is added to the usp_region.
There are two simple solutions for this (1) using DDL Triggers - very bad idea but simple to implement and working. (2) Using my trick to select from stored procedure
Option 1: using DDL trigger
You can automate the entire procedure and ALTER the stored procedure usp_calculatedDisplay every time that the stored procedure usp_region is changed
https://learn.microsoft.com/en-us/sql/relational-databases/triggers/ddl-triggers
The basic approach is
CREATE OR ALTER TRIGGER NotGoodSolutionTrig ON DATABASE FOR ALTER_PROCEDURE AS BEGIN
DECLARE #var_xml XML = EVENTDATA();
IF(
#var_xml.value('(EVENT_INSTANCE/DatabaseName)[1]', 'sysname') = 'tempdb'
and
#var_xml.value('(EVENT_INSTANCE/SchemaName)[1]', 'sysname') = 'dbo'
and
#var_xml.value('(EVENT_INSTANCE/ObjectName)[1]', 'sysname') = 'usp_region'
)
BEGIN
-- Here you can parse the text of the stored procedure
-- and execute ALTER on the first SP
-- To make it simpler, you can design the procedure usp_region so the columns names will be in specific row or between to comment which will help us to find it
-- The code of the Stored Procedure which you need to parse is in the value of:
-- #var_xml.value('(EVENT_INSTANCE/TSQLCommand/CommandText)[1]', 'NVARCHAR(MAX)'))
-- For example we can print it
DECLARE #SP_Code NVARCHAR(MAX)
SET #SP_Code = CONVERT(NVARCHAR(MAX), #var_xml.value('(EVENT_INSTANCE/TSQLCommand/CommandText)[1]', 'NVARCHAR(MAX)'))
PRINT #SP_Code
-- In your case, you need to execute ALTER on the usp_calculatedDisplay procedure using the text from usp_region
END
END
Option 2: trick to select from stored procedure using sys.dm_exec_describe_first_result_set
This is simple and direct way to get what you need.
CREATE OR ALTER PROCEDURE usp_calculatedDisplay AS
-- Option: using simple table, so it will exists outsie the scope of the dynamic query
DROP TABLE IF EXISTS MyTable;
DECLARE #sqlCommand NVARCHAR(MAX)
select #sqlCommand = 'CREATE TABLE MyTable(' + STRING_AGG ([name] + ' ' + system_type_name, ',') + ');'
from sys.dm_exec_describe_first_result_set (N'EXEC usp_region', null,0)
PRINT #sqlCommand
EXECUTE sp_executesql #sqlCommand
INSERT MyTable EXECUTE usp_region;
SELECT * FROM MyTable;
GO
Note!!! Both solutions are not recommended in production. My advice is to avoid such needs by redesign your system. If you need to re-write 20 SP so do it and don't be lazy! Your goal should be what best for the database usage.
We are writing a stored procedure responsible for getting a stored procedure name and returning a result containing the stored procedure columns and their data types.
However, we bumped into a problem executing a dynamic query to return the results of stored procedure, but we can't store it in a temp table!
You can see our query below:
DECLARE #ProcName VARCHAR(100)='spGetOraganizationsList',
#ParamName VARCHAR(100),#DataType VARCHAR(20),
#Query NVARCHAR(MAX)='EXEC '+'spGetOraganizationsList '
SELECT PARAMETER_NAME,DATA_TYPE
INTO #Tmp
FROM information_schema.PARAMETERS
WHERE SPECIFIC_NAME=#ProcName
DECLARE ParamCursor CURSOR
FOR SELECT * FROM #Tmp
OPEN ParamCursor
FETCH NEXT FROM ParamCursor
INTO #ParamName,#DataType
WHILE ##FETCH_STATUS = 0
BEGIN
SET #Query=#Query+#ParamName+'=Null,'
FETCH NEXT FROM ParamCursor INTO #ParamName,#DataType
END
CLOSE ParamCursor
DEALLOCATE ParamCursor
DROP TABLE #Tmp
EXEC sp_executesql #Query
The thing is I can't store the results of it in a temp table,
and OPENROWSET does not accept variables.
I think it comes from sql concept that it doesn't trust in result of stored procedures and because of that we cannot select on it or store it in a table by 'making in query table' method.
Unless you create a table and define it's columns and sql trust to you and you insert result of it into this table for example take below situation
Create table test (name varchar(10),family varchar(20))
Insert into test
Exec sp-testResult
Now if you define wrong column for your table you will receive query runtime error .actually sql doesn't predict result of sp and leaves it to you to define result of your stored procedure.
You can certainly INSERT the results of a stored procedure into a TEMP table:
CREATE PROCEDURE PurgeMe
AS
SELECT convert(int, 1) AS DaData
UNION
SELECT convert(int, 2)
GO
CREATE TABLE #Doodles (AnInteger int)
INSERT #Doodles EXECUTE PurgeMe
SELECT * FROM #Doodles
Questions arise about the SCOPE of TEMP tables, however. You might find that in your calling routine you will not be able to see a TEMP table created within your routine.
The solution to the SCOPE problem is to do the following:
Create a minimal TEMP table (say, with one column)
Use ALTER TABLE on the TEMP table within your routine to make its schema match
your needs (this can be tricky, but it can be done)
Put data into the TEMP table
return from your routine - the calling routine will now be able to access the temp
table
If this is of interest I can make a longer post with a stored procedure to do the above. It was written to facilitate dynamic SQL
Write select query as you want in the stored procedure. You will get the result without creating temp table.
Use global temp table and dynamic OPENROWSET
DROP TABLE ##Tmp;
GO
DECLARE #ProcName VARCHAR(100)='spGetOraganizationsList',
#ParamName VARCHAR(100), #DataType VARCHAR(20),
-- Mind to specify database and schema of the SP
#Query NVARCHAR(MAX)=' EXEC [mydb].[dbo].spGetOraganizationsList ';
SELECT PARAMETER_NAME,DATA_TYPE
INTO #Tmp
FROM information_schema.PARAMETERS
WHERE SPECIFIC_NAME=#ProcName;
-- Build SP exec
DECLARE ParamCursor CURSOR
FOR SELECT * FROM #Tmp
OPEN ParamCursor
FETCH NEXT FROM ParamCursor
INTO #ParamName,#DataType
WHILE ##FETCH_STATUS = 0
BEGIN
SET #Query=#Query+#ParamName+'=Null,'
FETCH NEXT FROM ParamCursor INTO #ParamName,#DataType
END
CLOSE ParamCursor
DEALLOCATE ParamCursor
SET #Query = left(#Query, len(#Query) - 1);
-- Build ad hoc distributed query which creates ##Tmp from SP exec.
SET #Query = 'SELECT * INTO ##Tmp FROM OPENROWSET(''SQLNCLI'', ''Server=localhost;Trusted_Connection=yes;'',''' + #Query + ''')';
EXEC (#Query);
-- Created by dynamic sql `##Tmp` is availabe in the current context.
SELECT *
FROM ##Tmp;
Don't forget to enable ad hoc distributed queries first.
sp_configure 'Show Advanced Options', 1
GO
RECONFIGURE
GO
sp_configure 'Ad Hoc Distributed Queries', 1
GO
RECONFIGURE
GO
EDIT
My answer solves only one problem, storing the result of a dynamic proc call in a temp table. And there are more problems.
First, #p=null just will not compile if the type of #p is user-defined table type. You need kind of declare #t myType;
exec mySp ... ,#p=#t ....
Next is the 'cannot retrieve matadata for sp because contain dynamic query' error you commented on. Looks like you need an application, SqlClr or standalone, which would be capable to read and parse Datasets returned by procs.
Finally, if an SP contains conditional sql which can return a result set of different schema depending on parameter values, the result of all those efforts is still questionable.
In C#, you can use an SqlDataReader or a DataTable to get the results from a stored procedure without knowing the schema beforehand. If you then want to write that data to a temporary table, I think you can do that from C# (though I've never tried to do it).
I'm writing a stored procedure. I have a string which contains an sql query. For example:
DECLARE #sql nvarchar(max)
SET #sql = (N'SELECT pkOrderID FROM Orders')
(Just to note: this isn't what the select statement looks like. This is just an example of what I mean) I then want to execute the string and put the result in a temporary table E.g. #tempTable. I know EXEC(#sql) exists but not sure if it will do me any good in this situation. The other twist is that I do not know the names of all the columns in the returned #sql so the temp table #tempTable needs to be created dyanmically based off the return from #sql. Thanks for any help.
I think you could use SELECT INTO to do what you want but it would mean updating your string:
DECLARE #sql nvarchar(max)
SET #sql = (N'SELECT frompkOrderID INTO #tmporders FROM Orders')
then you should be able to run EXEC #sql to create the table
more information about SELECT INTO here : http://msdn.microsoft.com/en-au/library/ms188029.aspx
There is no simple way to do this. The problem with #JanR's solution is that the #tmporders table will be out of scope to the script that calls your stored procedure (ie It will produce an error like "Invalid object name '#rtmporders'"
One alternative is to use a global temp table (eg ##tmporders).
So your SP might look like this:
CREATE PROCEDURE TestSP
AS
BEGIN
SELECT pkOrderID INTO ##tmporders FROM Orders
END
GO
And the calling script might be like:
EXEC TestSP
SELECT * FROM ##temporders
I have a stored procedure that is responsible to import data from database A to database B. I have customers, they all have their own database B (with the same tables etc.) They also have their own database A. The stored procedure will be executed (deployed) on database B.
The problem:
Each customer have another database name for A and B. I found also that database A can be on a different server/instance. I need a generic way to put automatically the server/instance name and the database name from A in my stored procedure to select data from there. Every customer has a connectstring to database A which is already exists in database B. So from the connecstring, I can get the server/instance name and the databasename of A. (I already created a linked server)
1. What is the best way to put the server/instance name and the database name before the table name??
Stored procedure
In my stored procedure I have used a lot of variable (declare) tables to insert the data from database A. There are many articles about using dynamic sql but, I cant find an example with a declare table.
My solution
I am thinking about inserting all the data from database A to variable tables. I am importing data from 7 tables, so I need to declare 7 tables and further in my SP I can select data from my declare tables. Note that the hole stored procedure is very huge.
Questions
2.What do you think about my solution?
3.Do you have another solution?
4.How can I insert into my declare table using dynamic sql?
**note that I am using sql server 2005.
I have a few statements like below:
declare #Temp table (Id int, etc
insert into #Temp (Id, etc)
Select Id, etc
From [databasename].dbo.TableName //hardcoded
Where .......
// doing staff like selecting from the #Temp table etc.
I also have subqueries, but I can change if it is necessary.
You can't use dynamic sql to enter into declare tables as the temporary table is only available in the session. Executing any new sql i.e. through dynamic sql will create a new session.
You can get around it by not using any declared or temp tables but instead using a normal table. The dynamic sql will have access to this and anything you do to it isnt lost.
You can prefix your normal tables with something like Temp_ just to note they are temp tables and then make sure you drop them at the beginning of each query i.e.
DROP TABLE TEMP_Table
You can call multiple local databases by doing
SELECT * FROM [DatabaseName].dbo.[TableName]
Instead of creating #temp, create it as [TempDB].[DBO].[Temp] and it will be accessible outside the dynamic SQL. However, remember to explicitly drop the table, once you are done
DECLARE #sql VARCHAR(200)
SET #sql = 'CREATE TABLE tempdb.dbo.temp(id INT IDENTITY(1,1), DESCRIPTION VARCHAR(100))
INSERT INTO tempdb.dbo.temp SELECT ''1'' SELECT * FROM tempdb.dbo.temp'
PRINT (#sql)
EXEC (#sql)
SELECT * FROM tempdb.dbo.temp
DROP TABLE tempdb.dbo.temp
Raj
Check this for creating temporary table in dynamic query
DECLARE #sql VARCHAR(200)
SET #sql = 'CREATE TABLE #temp(id INT IDENTITY(1,1), DESCRIPTION VARCHAR(100))
INSERT INTO #temp SELECT ''1'' SELECT * FROM #temp'
PRINT (#sql)
EXEC (#sql)
SQLFiddle
to be precise
DECLARE #sql VARCHAR(1000)
SET #sql =
'DECLARE #Temp TABLE (
Id INT,
etc
INSERT INTO #Temp (Id, etc)
SELECT Id,
etc
FROM [databasename].dbo.TableName / / hardcoded
WHERE ....... SELECT * FROM #temp'
PRINT (#sql)
EXEC (#sql)
so I have a sproc in a db.. lets call this db A. This db makes use of tables (t1, t2) in another db. Lets call this db B.
okay, so the way i call it right now is: A.dbo.My_Proc but i get another error:
Invalid object name 'dbo.t1'.
so how i tried supplying a parameter. In my Sproc i do, select * from #dbname.dbo.t1
however that results in an error. I can't put the sproc in db B.
While it is sufficient to hardcode it (if there is a way), db B changes every year, so it would be nice to "supply" a database.
I tried using use B; go but it gives me error saying can't have that in a sproc.
You could create a synonym:
EDIT: I see now that the synonym is needed for the table, not the proc. Got them switched. So you could create a synonym in database A for the table in database B:
USE A;
CREATE SYNONYM dbo.t1 FOR B.dbo.t1;
Then your procedure in A could simply say:
SELECT * FROM dbo.t1;
Without having to manually supply the database name at all, the query knows (based on the synonym) to get the data from the table in database B. When the database B changes to C, you can simply:
DROP SYNONYM dbo.t1;
CREATE SYNONYM dbo.t1 FOR C.dbo.t1;
If you used "real" database names in your narrative as opposed to arbitrary A/B names, it might lead to easier comprehension. Just a suggestion. :-)
/EDIT
The other option is to pass in the database name and construct via dynamic SQL. E.g. instead of select * from #dbname.dbo.t1 (which will never work), you could do:
DECLARE #sql NVARCHAR(MAX) = N'SELECT * FROM ' + QUOTENAME(#dbname) + '.dbo.t1;';
EXEC sp_executesql #sql;
But if this other database name really only changes once a year, I suggest that the synonym route is better overall.
When you execute exec someDB.dbo.SomeProc the execution context switches to someDB. So if the procedure issues a SELECT FROM dbo.t1 then dbo.t1 must be in someDB. IF you want the procedure to select from a 'supplied' database then the procedure must use dynamic-SQL:
create procedure someProc
#dbname sysname
as
begin
...
set #sql = N'SELECT ... FROM ' + quotename(#dbname) +N'.dbo.t1';
exec sp_executesql #sql;
...
end