I have two TSQL EXEC statements
EXECUTE (N'MyDynamicallyGeneratedStoredProcedure') -- return 0 on success
SELECT #errCode = ##ERROR ;
IF (#errCode = 0)
BEGIN
EXEC 'A Sql Statement using ##temptable created from first', #returnValue
END
How do I make the two EXEC's synchronous? ; Right now the second EXEC does not wait for the first EXECUTE to complete. I tried issuing a WaitFor Delay, It waits but the second EXEC statement is never returing back.
Thanks.
Update, Here is more info:
First execute creates a global temp table and populates it from a complex SELECT query.
Second EXEC is a CLR Stored Procedure that generates a dynamic SP, based on the variables from recently created and populated Global Temp table.
Now the second EXEC, complains that the Global Temp table is not found.
Update 2, Found the issue (And its me!!)
GBN (and others) was point blank on the answer. EXEC IS synchronous. The problem? My understanding of the problem itself.. I had mentioned
EXECUTE (N'MyDynamicallyGeneratedStoredProcedure') -- return 0 on success
It should have been:
1(a) EXECUTE (N'CreateMyDynamicStoredProcedure') -- return 0 on success
1(b) EXECUTE (N'MyDynamicStoredProcedure') -- return 0 on success
I missed that 1(b) was actually executed somewhere else and after step (2) .
(I should go get a life!!)
EXECUTE is sychronous. The 2nd one runs after the 1st one. Always.
Do you have multiple connections running the same code? You are using a global temp table that will be visible to all connections so it may look like asyncc execution...
As gbn's answer has pointed out, EXECUTE is synchronous.
The problem might be that your SQL Connection object within CRL stored procedure is not in the same context as your batch script. Your global temporary table should have been dropped after running EXECUTE (N'MyDynamicallyGeneratedStoredProcedure')
Make sure that you create your SQLConnection object by passing "context connection=true"
Here is the post answer where someone had a similar problem accessing temporary table since SQLConnection was not in the same connection context.
Accessing TSQL created #temp tables from CLR stored procedure. Is it possible?
If your second CRL stored procedure runs through a different connection, CRL sproc will not be able to access the global temp table since it should have been dropped.
Refer to this post on Global Temporary life cycle (when the gloal temp is dropped)
Deleting Global Temporary Tables (##tempTable) in SQL Server
Related
Background Information:
In Python, I might write something like this if I want to apply the same logic to different values in a list.
database_list = ["db_1", "db_2", "db_3"]
for x in range(0,len(database_list),1):
print("the database name is " + database_list[x])
What I am trying to do:
What I am trying to do in SSMS, is pull a list of DB objects for each database. I created a stored procedure to pull exactly what I want, but I have to run it against each database, so 10 databases mean running it 10 times.
My goal is to do this with a T-SQL query instead of Python.
I tried doing something like this:
exec sp_MSforeachdb 'USE ?; EXEC [dbo].[my_stored_procedure]';
The problem with this is, [dbo].[my_stored_procedure] has to exist in every database I want to do this in.
How can I create the stored procedure in 1 database, but execute it for all databases or a list of databases that I choose?
I know what you are trying to do and if it's what I think (you seem reluctant to actually say!) you can do the following:
In the master database, create your procedure. Normally you wouldn't do this, but in this case you must prefix it sp_
use master
go
create procedure sp_testproc as
select top 10 * from sys.tables
go
Now if you run this, it will return tables from the master database.
If you switch context to another database and exec master.dbo.sp_testproc, it will still return tables from the master database.
In master, run
sys.sp_MS_marksystemobject sp_testproc
Now switch context to a different database and exec master.dbo.sp_testproc
It will return tables from the database you are using.
Try creating your sproc in master and naming it with an sp_ prefix:
USE master
GO
CREATE PROCEDURE sp_sproc_name
AS
BEGIN
...
END
GO
-- You *may* need to mark it as a system object
EXEC sys.sp_MS_marksystemobject sp_sprocname
See: https://nickstips.wordpress.com/2010/10/18/sql-making-a-stored-procedure-available-to-all-databases/
It should then be available in all dbs
Create the stored procedure in the Master database with the sp_ prefix, and use dynamic SQL in the stored procedure so it resolves object names relative to the current database, rather than the database which contains the stored procedure.
EG
use master
go
CREATE OR ALTER PROCEDURE [dbo].[sp_getobjects]
AS
exec ('
select *
from [sys].[objects]
where is_ms_shipped = 0
order by type, name
')
go
use AdventureWorks2017
exec sp_getobjects
#LunchBox - it's your single stored procedure (that you create in one database) that is actually going to need to contain the "exec sp_MSforeach ...." command, and instead of the command to be executed being "EXEC ", it will need to be the actual SQL that you were going to put into the stored proc.
Eg. (inside your single stored procedure)
EXEC sp_MSforeachdb 'USE ?; SELECT * FROM <table>; UPDATE <another table> SET ...';
Think of the stored procedure (that you put into one database) as being no different than your Python code file - if you had actually wanted to achieve the same thing in Python, you would have either needed to create the stored proc in each database, or build the SQL statement string in Python and execute it against each database.
I understand what you thought you might be able to achieve with SQL, but stored procedures really don't work the way you were expecting. Even when you're in the context of a different database, but you run EXEC <different_db>.stored_proc, that stored proc ends up running in the context of the database in which it exists (not your context database).
Now, the only one issue you may come up against is that the standard sp_MSforeachdb stored proc has a limit of 2000 characters for the command that can be executed (although, it does have multiple "command" parameters, this may not be practical if you were planning on running a very large code block, perhaps with variables that carry all the way through). If this is something that might impact what you're intending to do, you could do a search online for "sp_MSforeachdb alternatives" - there seem to be a handful that people have created where the command parameter can contain a larger string.
I have three stored procedures Sp1, Sp2 and Sp3.
The first one (Sp1) will execute the second one (Sp2) and save returned data into #tempTB1 and the second one will execute the third one (Sp3) and save data into #tempTB2.
If I execute the Sp2 it will work and it will return me all my data from the Sp3, but the problem is in the Sp1, when I execute it it will display this error:
INSERT EXEC statement cannot be nested
I tried to change the place of execute Sp2 and it display me another error:
Cannot use the ROLLBACK statement
within an INSERT-EXEC statement.
This is a common issue when attempting to 'bubble' up data from a chain of stored procedures. A restriction in SQL Server is you can only have one INSERT-EXEC active at a time. I recommend looking at How to Share Data Between Stored Procedures which is a very thorough article on patterns to work around this type of problem.
For example a work around could be to turn Sp3 into a Table-valued function.
This is the only "simple" way to do this in SQL Server without some giant convoluted created function or executed sql string call, both of which are terrible solutions:
create a temp table
openrowset your stored procedure data into it
EXAMPLE:
INSERT INTO #YOUR_TEMP_TABLE
SELECT * FROM OPENROWSET ('SQLOLEDB','Server=(local);TRUSTED_CONNECTION=YES;','set fmtonly off EXEC [ServerName].dbo.[StoredProcedureName] 1,2,3')
Note: You MUST use 'set fmtonly off', AND you CANNOT add dynamic sql to this either inside the openrowset call, either for the string containing your stored procedure parameters or for the table name. Thats why you have to use a temp table rather than table variables, which would have been better, as it out performs temp table in most cases.
OK, encouraged by jimhark here is an example of the old single hash table approach: -
CREATE PROCEDURE SP3 as
BEGIN
SELECT 1, 'Data1'
UNION ALL
SELECT 2, 'Data2'
END
go
CREATE PROCEDURE SP2 as
BEGIN
if exists (select * from tempdb.dbo.sysobjects o where o.xtype in ('U') and o.id = object_id(N'tempdb..#tmp1'))
INSERT INTO #tmp1
EXEC SP3
else
EXEC SP3
END
go
CREATE PROCEDURE SP1 as
BEGIN
EXEC SP2
END
GO
/*
--I want some data back from SP3
-- Just run the SP1
EXEC SP1
*/
/*
--I want some data back from SP3 into a table to do something useful
--Try run this - get an error - can't nest Execs
if exists (select * from tempdb.dbo.sysobjects o where o.xtype in ('U') and o.id = object_id(N'tempdb..#tmp1'))
DROP TABLE #tmp1
CREATE TABLE #tmp1 (ID INT, Data VARCHAR(20))
INSERT INTO #tmp1
EXEC SP1
*/
/*
--I want some data back from SP3 into a table to do something useful
--However, if we run this single hash temp table it is in scope anyway so
--no need for the exec insert
if exists (select * from tempdb.dbo.sysobjects o where o.xtype in ('U') and o.id = object_id(N'tempdb..#tmp1'))
DROP TABLE #tmp1
CREATE TABLE #tmp1 (ID INT, Data VARCHAR(20))
EXEC SP1
SELECT * FROM #tmp1
*/
My work around for this problem has always been to use the principle that single hash temp tables are in scope to any called procs. So, I have an option switch in the proc parameters (default set to off). If this is switched on, the called proc will insert the results into the temp table created in the calling proc. I think in the past I have taken it a step further and put some code in the called proc to check if the single hash table exists in scope, if it does then insert the code, otherwise return the result set. Seems to work well - best way of passing large data sets between procs.
This trick works for me.
You don't have this problem on remote server, because on remote server, the last insert command waits for the result of previous command to execute. It's not the case on same server.
Profit that situation for a workaround.
If you have the right permission to create a Linked Server, do it.
Create the same server as linked server.
in SSMS, log into your server
go to "Server Object
Right Click on "Linked Servers", then "New Linked Server"
on the dialog, give any name of your linked server : eg: THISSERVER
server type is "Other data source"
Provider : Microsoft OLE DB Provider for SQL server
Data source: your IP, it can be also just a dot (.), because it's localhost
Go to the tab "Security" and choose the 3rd one "Be made using the login's current security context"
You can edit the server options (3rd tab) if you want
Press OK, your linked server is created
now your Sql command in the SP1 is
insert into #myTempTable
exec THISSERVER.MY_DATABASE_NAME.MY_SCHEMA.SP2
Believe me, it works even you have dynamic insert in SP2
I found a work around is to convert one of the prods into a table valued function. I realize that is not always possible, and introduces its own limitations. However, I have been able to always find at least one of the procedures a good candidate for this. I like this solution, because it doesn't introduce any "hacks" to the solution.
I encountered this issue when trying to import the results of a Stored Proc into a temp table, and that Stored Proc inserted into a temp table as part of its own operation. The issue being that SQL Server does not allow the same process to write to two different temp tables at the same time.
The accepted OPENROWSET answer works fine, but I needed to avoid using any Dynamic SQL or an external OLE provider in my process, so I went a different route.
One easy workaround I found was to change the temporary table in my stored procedure to a table variable. It works exactly the same as it did with a temp table, but no longer conflicts with my other temp table insert.
Just to head off the comment I know that a few of you are about to write, warning me off Table Variables as performance killers... All I can say to you is that in 2020 it pays dividends not to be afraid of Table Variables. If this was 2008 and my Database was hosted on a server with 16GB RAM and running off 5400RPM HDDs, I might agree with you. But it's 2020 and I have an SSD array as my primary storage and hundreds of gigs of RAM. I could load my entire company's database to a table variable and still have plenty of RAM to spare.
Table Variables are back on the menu!
I recommend to read this entire article. Below is the most relevant section of that article that addresses your question:
Rollback and Error Handling is Difficult
In my articles on Error and Transaction Handling in SQL Server, I suggest that you should always have an error handler like
BEGIN CATCH
IF ##trancount > 0 ROLLBACK TRANSACTION
EXEC error_handler_sp
RETURN 55555
END CATCH
The idea is that even if you do not start a transaction in the procedure, you should always include a ROLLBACK, because if you were not able to fulfil your contract, the transaction is not valid.
Unfortunately, this does not work well with INSERT-EXEC. If the called procedure executes a ROLLBACK statement, this happens:
Msg 3915, Level 16, State 0, Procedure SalesByStore, Line 9 Cannot use the ROLLBACK statement within an INSERT-EXEC statement.
The execution of the stored procedure is aborted. If there is no CATCH handler anywhere, the entire batch is aborted, and the transaction is rolled back. If the INSERT-EXEC is inside TRY-CATCH, that CATCH handler will fire, but the transaction is doomed, that is, you must roll it back. The net effect is that the rollback is achieved as requested, but the original error message that triggered the rollback is lost. That may seem like a small thing, but it makes troubleshooting much more difficult, because when you see this error, all you know is that something went wrong, but you don't know what.
I had the same issue and concern over duplicate code in two or more sprocs. I ended up adding an additional attribute for "mode". This allowed common code to exist inside one sproc and the mode directed flow and result set of the sproc.
what about just store the output to the static table ? Like
-- SubProcedure: subProcedureName
---------------------------------
-- Save the value
DELETE lastValue_subProcedureName
INSERT INTO lastValue_subProcedureName (Value)
SELECT #Value
-- Return the value
SELECT #Value
-- Procedure
--------------------------------------------
-- get last value of subProcedureName
SELECT Value FROM lastValue_subProcedureName
its not ideal, but its so simple and you don't need to rewrite everything.
UPDATE:
the previous solution does not work well with parallel queries (async and multiuser accessing) therefore now Iam using temp tables
-- A local temporary table created in a stored procedure is dropped automatically when the stored procedure is finished.
-- The table can be referenced by any nested stored procedures executed by the stored procedure that created the table.
-- The table cannot be referenced by the process that called the stored procedure that created the table.
IF OBJECT_ID('tempdb..#lastValue_spGetData') IS NULL
CREATE TABLE #lastValue_spGetData (Value INT)
-- trigger stored procedure with special silent parameter
EXEC dbo.spGetData 1 --silent mode parameter
nested spGetData stored procedure content
-- Save the output if temporary table exists.
IF OBJECT_ID('tempdb..#lastValue_spGetData') IS NOT NULL
BEGIN
DELETE #lastValue_spGetData
INSERT INTO #lastValue_spGetData(Value)
SELECT Col1 FROM dbo.Table1
END
-- stored procedure return
IF #silentMode = 0
SELECT Col1 FROM dbo.Table1
Declare an output cursor variable to the inner sp :
#c CURSOR VARYING OUTPUT
Then declare a cursor c to the select you want to return.
Then open the cursor.
Then set the reference:
DECLARE c CURSOR LOCAL FAST_FORWARD READ_ONLY FOR
SELECT ...
OPEN c
SET #c = c
DO NOT close or reallocate.
Now call the inner sp from the outer one supplying a cursor parameter like:
exec sp_abc a,b,c,, #cOUT OUTPUT
Once the inner sp executes, your #cOUT is ready to fetch. Loop and then close and deallocate.
If you are able to use other associated technologies such as C#, I suggest using the built in SQL command with Transaction parameter.
var sqlCommand = new SqlCommand(commandText, null, transaction);
I've created a simple Console App that demonstrates this ability which can be found here:
https://github.com/hecked12/SQL-Transaction-Using-C-Sharp
In short, C# allows you to overcome this limitation where you can inspect the output of each stored procedure and use that output however you like, for example you can feed it to another stored procedure. If the output is ok, you can commit the transaction, otherwise, you can revert the changes using rollback.
On SQL Server 2008 R2, I had a mismatch in table columns that caused the Rollback error. It went away when I fixed my sqlcmd table variable populated by the insert-exec statement to match that returned by the stored proc. It was missing org_code. In a windows cmd file, it loads result of stored procedure and selects it.
set SQLTXT= declare #resets as table (org_id nvarchar(9), org_code char(4), ^
tin(char9), old_strt_dt char(10), strt_dt char(10)); ^
insert #resets exec rsp_reset; ^
select * from #resets;
sqlcmd -U user -P pass -d database -S server -Q "%SQLTXT%" -o "OrgReport.txt"
I'm running a stored procedure on server1 from my application. The stored procedure does a bunch of stuff and populate a table on server2 with the result from the procedure.
I'm using linked server to accomplish this.
When the stored procedure is done running the application continues and tries to do some manipulation of the result from the stored procedure.
My problem is that the results from the stored procedure has not been completely inserted into the tables yet, so the manipulation of the tables fails.
So my question is. Is it possible to ensure the insert into on the linked server is done synchronous? I would like to have the stored procedure not return until the tables on the linked server actually is done.
You can use an output parameter of the first procedure. When the table is create on the second server the output parameter value will be return to your application and indicates the operation is ready.
If the things are difficult then this you can try setting a different isolation level of your store procedure:
http://msdn.microsoft.com/en-us/library/ms173763.aspx
I found the reason for this strange behavior. There was a line of code in my stored procedure added during debug that did a select on a temporary mem table before the data in the same table was written to the linked server.
When the select statement was run, the control was given back to my application and at the same time the stored procedure continued running. I guess the stored procedure was running synchronously from the start.
I have 10 stored procedures.
For example -
stored procedure fetches the rows from table A
then stored procedure runs and then third...
How can I do error handling in this.. for example I have to check with if first stored procedure executed successfully run second else throw error. If first executed successfully run second stored procedure if second runs successfully run third otherwise throw error.
ALTER PROCEDURE [dbo].[MASTER_PROCEDURE] AS
EXEC QRY_STEP3
EXEC QRY_STEP_3_1_1
EXEC OQRY_STEP_3_1_1
I would add logic to each of your subsidiary stored procedures to determine whether they have succeeded or not. eg test for existence of the temporary table. Then use a return value to indicate success of the proc. Typically this would be 0 for succes and non-zero for failure.
You would then call the procs from your master proc like this
DECLARE #ReturnValue INT
EXEC #ReturnValue = QRY_STEP1
IF(#ReturnValue = 0)
BEGIN
EXEC #ReturnValue = QRY_STEP2
END
ELSE
BEGIN
--REPORT ERROR
END
Using this approach, your master proc doesnt need to know about the inner workings of each child proc, and your master proc code will be cleaner and more readable.
use ##error.. Can be done like this
ALTER PROCEDURE [dbo].[MASTER_PROCEDURE] AS
EXEC QRY_STEP3
IF ##error =0
begin
EXEC QRY_STEP_3_1_1
else
begin
print "error in proc name"
return 1
End
if (##error=0)
Begin
EXEC OQRY_STEP_3_1_1
Else
print "error in proc name"
return 1
End
END
END
First to do this correctly you should use TRY CATCH blocks in the child packages. Those should return to the calling proc if there is an error. This way you can also return an error code to the calling proc if results are unexpected, such as a temp table with zero records which is not an error but which might make the subsequent procs fail.
Next, why are you using child procs at all? Honestly this is something that is probably better done in one proc. You say for instance that you are creating temp tables in one proc that you use in subsequent procs. To do this you need global temp tables. The problem is that global temp tables are not specific to the orginal connection that called them and thus two people trying to do this simluatnaeously might have their data mixedup. Whereas if you use one proc and local temp tables that can't happen.
I currently am working on a legacy application and have inherited some shady SQL with it. The project has never been put into production, but now is on it's way. During intial testing I found a bug. The application calls a stored procedure that calls many other stored procedures, creates cursors, loops through cursors, and many other things. FML.
Currently the way the app is designed, it calls the stored procedure, then reloads the UI with a fresh set of data. Of course, the data we want to display is still being processed on the SQL server side, so the UI results are not complete when displayed. To fix this, I just made a thread sleep for 30 seconds, before loading the UI. This is a terrible hack and I would like to fix this properly on the SQL side of things.
My question is...is it worthwhile to convert the branching stored procedures to functions? Would this make the main-line stored procedure wait for a return value, before processing on?
Here is the stored procedure:
ALTER PROCEDURE [dbo].[ALLOCATE_BUDGET]
#budget_scenario_id uniqueidentifier
AS
BEGIN
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
DECLARE #constraint_type varchar(25)
-- get project cache id and constraint type
SELECT #constraint_type = CONSTRAINT_TYPE
FROM BUDGET_SCENARIO WHERE BUDGET_SCENARIO_ID = #budget_scenario_id
-- constraint type is Region by Region
IF (#constraint_type = 'Region by Region')
EXEC BUDGET_ALLOCATE_SCENARIO_REGIONBYREGION #budget_scenario_id
-- constraint type is City Wide
IF (#constraint_type = 'City Wide')
EXEC BUDGET_ALLOCATE_SCENARIO_CITYWIDE #budget_scenario_id
-- constraint type is Do Nothing
IF (#constraint_type = 'Do Nothing')
EXEC BUDGET_ALLOCATE_SCENARIO_DONOTHING #budget_scenario_id
-- constraint type is Unconstrained
IF (#constraint_type = 'Unconstrained')
EXEC BUDGET_ALLOCATE_SCENARIO_UNCONSTRAINED #budget_scenario_id
--set budget scenario status to "Allocated", so reporting tabs in the application are populated
EXEC BUDGET_UPDATE_SCENARIO_STATUS #budget_scenario_id, 'Allocated'
END
To avoid displaying an incomplete resultset in the calling .NET application UI, before the cursors in the branching calls are completed, is it worthwile to convert these stored procedures into functions, with return values? Would this force SQL to wait before completing the main call to the [ALLOCATED_BUDGET] stored procedure?
The last SQL statement call in the stored procedure sets a status to "Allocated". This is happening before the cursors in the previous calls are finished processing. Does making these calls into function calls affect how the stored procedure returns focus to the application?
Any feedback is greatly appreciated. I have a feeling I am correct in going towards SQL functions but not 100% sure.
** additional information:
Executing code uses [async=true] in the connection string
Executing code uses the [SqlCommand].[ExecuteNonQuery] method
How are you calling the procedure? I'm going to guess that you are using ExecuteNonQuery() to call the procedure. Try calling the procedure using ExecuteScalar() and modify the procedure like the following:
ALTER PROCEDURE [dbo].[ALLOCATE_BUDGET]
#budget_scenario_id uniqueidentifier
AS
BEGIN
...
RETURN True
END
This should cause your data execution code in .NET to wait for the procedure to complete before continuing. If you don't want your UI to "hang" during the procedure execution, use a BackgroundWorkerProcess or something similar to run the query on a separate thread and look for the completed callback to update the UI with the results.
You could also try using the RETURN statement in your child stored procedures, which can be used to return a result code back to the parent procedure. You can call the child procedure by something along the lines of "exec #myresultcode = BUDGET_ALLOCATE_SCENARIO_REGIONBYREGION()". I think this should force the parent procedure to wait for the child procedure to finish.
I have never heard that it's possible for a stored procedure to return to the caller while still executing in the background.
In fact, I'll go as far as to say I don't believe that's happening. If you're seeing a difference between the UI and what you believe the SP should have done, then I believe it has a different cause.
Does the connection string have async=true in it? Is the SP being executed by using BeginExecuteReader or Begin-anything else?
At the risk of sounding to simple, I suggest you could create a table which can store the status of the stored proc. Somehow, a flag that can indicate that the entire process & sub-process has finished executing.
You could query this from UI to see if things are done by polling this status code.
Does making these calls into function calls affect how the stored procedure returns focus to the application?
No.
The stored procedure has no idea that its caller is a UI application. There is nothing in the stored procedure that can influence the behavior of the UI application.
Most likely the UI application is calling the stored procedure on one connection, and then refreshing its data on another connection. There's a plethora of ways of getting the UI to delay refreshing, but the one I'll push is that there should be a single database connection.
Personally, I would be far more concerned about replacing those cursors than converting this to functions.
And I would not run the last proc until checking for a valid return code from the previous procs (this thing is in real trouble if one of the preceding procs dies!)
Also consider if this should all be in a transaction (are these procs changing data in a table?)
(Am I the only one who finds it funny you have a proc to run the process for Do Nothing?)