SQL Server (TSQL) - Is it possible to EXEC statements in parallel? - sql

Question

SQL Server 2008 R2
Here is a simplified example:
EXECUTE sp_executesql N'PRINT ''1st '' + convert(varchar, getdate(), 126) WAITFOR DELAY ''000:00:10'''
EXECUTE sp_executesql N'PRINT ''2nd '' + convert(varchar, getdate(), 126)'
The first statement will print the date and delay 10 seconds before proceeding.
The second statement should print immediately.
The way T-SQL works, the 2nd statement won't be evaluated until the first completes. If I copy and paste it to a new query window, it will execute immediately.
The issue is that I have other, more complex things going on, with variables that need to be passed to both procedures.
What I am trying to do is:
Get a record
Lock it for a period of time
while it is locked, execute some other statements against this record and the table itself
Perhaps there is a way to dynamically create a couple of jobs?
Anyway, I am looking for a simple way to do this without having to manually PRINT statements and copy/paste to another session.
Is there a way to EXEC without wait / in parallel?

Answer 1

Yes, there is a way, see Asynchronous procedure execution.
However, chances are this is not what you need. T-SQL is a data access language, and when you take into consideration transactions, locking and commit/rollback semantics is almost impossible to have a parallel job. Parallel T-SQL works for instance with requests queues, where each requests is independent and there is no correlation between jobs.
What you describe doesn't sound at all like something that can, nor should, actually be paralellized.

Answer 2

If you want to lock a record so you can execute statements against it, you may want to execute those statements as a transaction.
To execute SQL in parallel, you need to paralellize SQL calls, by executing your SQL from within separate threads/processes in Java, C++, perl, or any other programming language (hell, launching "isql" in shell script in background will work)

Answer 3

If after reading all above about potential problems and you still want to run things in parallel, you probably can try sql jobs, put your queries in different jobs, then execute by calling the jobs like this
EXEC msdb..sp_start_job 'Job1'
EXEC msdb..sp_start_job 'Job2'

Answer 4

SQL Agent Jobs can run in parallel and be created directly from TSQL. The answer by Remus Rusanu contains a link that mentions this along with some disadvantages.
Another disadvantage is that additional security permissions are required to create the job. Also, for the implementation below, the job must run as a specific user+login with additional job management privileges.
It is possible to run the arbitrary SQL as a different (safer) user however I believe it requires sysadmin privilege to designate the job as such.
The returned #pJobIdHexOut can be used to stop the job if needed.
create function Common.ufn_JobIdFromHex(
#pJobIdBinary binary(16)
)
returns varchar(100) as
/*---------------------------------------------------------------------------------------------------------------------
Purpose: Convert the binary represenation of the job_id into the job_id string that can be used in queries
against msdb.dbo.sysjobs.
http://stackoverflow.com/questions/68677/how-can-i-print-a-binary-value-as-hex-in-tsql
http://stackoverflow.com/questions/3604603
MsgBoards
Modified By Description
---------- -------------- ---------------------------------------------------------------------------------------
2014.08.22 crokusek Initial version, http://stackoverflow.com/questions/3604603 and MsgBoards.
---------------------------------------------------------------------------------------------------------------------*/
begin
-- Convert from binary and strip off the '0x'.
--
declare
#jobIdHex varchar(100) = replace(convert(varchar(300), #pJobIdBinary, 1), '0x', '');
-- The endianness appears to be backwards and there are dashes needed.
--
return
substring(#jobIdHex,7,2) +
substring(#jobIdHex,5,2) +
substring(#jobIdHex,3,2) +
substring(#jobIdHex,1,2) +
'-' +
substring(#jobIdHex,11,2) +
substring(#jobIdHex,9,2) +
'-' +
substring(#jobIdHex,15,2) +
substring(#jobIdHex,13,2) +
'-' +
substring(#jobIdHex,17,4) +
'-' +
substring(#jobIdHex,21,12);
end
go
create proc [Common].[usp_CreateExecuteOneTimeBackgroundJob]
#pJobNameKey varchar(100), -- Caller should ensure uniqueness to avoid a violation
#pJobDescription varchar(1000),
#pSql nvarchar(max),
#pJobIdHexOut varchar(100) = null out, -- JobId as Hex string. For SqlServer 2014 binary(16) = varchar(64)
#pDebug bit = 0 -- True to include print messages
--
with execute as 'TSqlBackgroundJobOwner' -- requires special permissions (See below)
as
/*---------------------------------------------------------------------------------------------------------------------
Purpose: Create a one time background job and launch it immediately. The job is owned by the "execute as" UserName
Caller must ensure the #pSql argument is safe.
Required Permissions for "execute as" user:
-- User must be created with associated login (w/ deny connect).
use [msdb];
create user [$UserName$] for login [$LoginName$];
alter role [SQLAgentUserRole] add member [$UserName$];
alter role [SQLAgentReaderRole] add member [$UserName$];
alter role [SQLAgentOperatorRole] add member [$UserName$];
grant select on dbo.sysjobs to [$UserName$];
grant select on dbo.sysjobactivity to [$UserName$];',
use [Master];
create user [$UserName$] for login [$LoginName$];
grant execute on xp_sqlagent_is_starting to [$UserName$];
grant execute on xp_sqlagent_notify to [$UserName$];';
Modified By Description
---------- ----------- ------------------------------------------------------------------------------------------
2014.08.22 crokusek Initial version
2015.12.22 crokusek Use the SP caller as the job owner (removed the explicit setting of the job owner).
---------------------------------------------------------------------------------------------------------------------*/
begin try
declare
#usp varchar(100) = object_name(##procid),
#currentDatabase nvarchar(100) = db_name(),
#jobId binary(16),
#jobOwnerLogin nvarchar(100);
set xact_abort on; -- ensure transaction is aborted on non-catchables like client timeout, etc.
begin transaction
exec msdb.dbo.sp_add_job
#job_name=#pJobNameKey,
#enabled=1,
#notify_level_eventlog=0,
#notify_level_email=2,
#notify_level_netsend=2,
#notify_level_page=2,
#delete_level=3,
#description=#pJobDescription,
#category_name=N'Database Maintenance',
-- If not overridden then the the current login is the job owner
--#owner_login_name=#jobOwnerLogin, -- Requires sysadmin to set this so avoiding.
#job_id = #jobId output;
-- Get the job_id string of the jobId (special format)
--
set #pJobIdHexOut = Common.ufn_JobIdFromHex(#jobId);
if (#pDebug = 1)
begin
print 'JobId: ' + #pJobIdHexOut;
print 'Sql: ' + #pSql;
end
exec msdb.dbo.sp_add_jobserver #job_id=#jobId; -- default is local server
exec msdb.dbo.sp_add_jobstep
#job_id=#jobId,
#step_name=N'One-Time Job Step 1',
#step_id=1,
#command=#pSql,
#database_name=#currentDatabase,
#cmdexec_success_code=0,
#on_success_action=1,
#on_fail_action=2,
#retry_attempts=0,
#retry_interval=0,
#os_run_priority=0,
#subsystem=N'TSQL',
#flags=0
;
declare
#startResult int;
exec #startResult = msdb.dbo.sp_start_job
#job_id = #jobId;
-- End the transaction
--
if (#startResult != 0)
raiserror('Unable to start the job', 16, 1); -- causes rollback in catch block
else
commit; -- Success
end try
begin catch
declare
#CatchingUsp varchar(100) = object_name(##procid);
if (xact_state() = -1)
rollback;
--exec Common.usp_Log
-- #pMethod = #CatchingUsp;
--exec Common.usp_RethrowError
-- #pCatchingMethod = #CatchingUsp;
end catch
go

Answer 5

It might be worth to check out the article Asynchronous T-SQL Execution Without Service Broker.

Answer 6

You can create an SSIS that has 2 tasks that run in parallel. Then make an unscheduled agent job to call this SSIS. You can finally execute this unscheduled agent job using sp_start_job.

Answer 7

Hi Picking one answer above it is possible to produce something similar to a multi-thread execution using SQL Agent jobs and some auxiliary tables and using SQL Server metadata. I made it already and I was able to call the same procedure 32 times on a server processing 1/32 parts of the processed data each.
Of course one needs to pay high attention to the data partitioning logic so datasets do not overlap. The best way is to use the mod operator over a numeric field.
This logic even allows different partitioning sets between steps on the procedure. On one step you can use field A on the next step field B.
As mentioned above you need to be very carefully with table locks and something I noticed is partitioning tables will also speed data insertion and updates.
I built a master job generator engine using T-SQL that triggered the requested number of procedures using jobs. All of this was called from a SSIS job.
The process was far from being simple to develop but it mimics quite well C# or Java multi-thread logic.
I also had to build some auxiliary tables that hold each job status so the master T-SQL job engine procedure was able to check each job status.
I used SQL Server metadata but each job that was created and started knew how to update its own status -> Job X when it starts updates its status on the main table status monitor, when running the same and when it finishes it closes its status. The main job procedure keeps checking on those auxiliary tables if there are Jobs on running status and will only end when all of them have the status finished.
Microsoft could think on developing something similar on SSIS.