U-SQL procedure call order - azure-data-lake

Let we have script that consists of 3 procedure calls:
mydb.myschema.PopulateDimensions(...)
mydb.myschema.PopulateFactTable(...)
mydb.myschema.UpdateStatistics(...)
Will they be executed in the same order or they will be inlined into the script and the entire mix will be executed in parallel?

They will be inlined and then the compiler, optimizer and scheduler will determine how to best execute the overall script.
What are you trying to achieve/hope for?

Related

Can you run a portion of a script in parallel, based off the results of a select statement?

I have a portion of code which, when simplified, looks something like this:
select #mainlooptableid = min(uid)
from queueofids with (nolock)
while (#mainlooptableid is not null)
begin
-- A large block of code that does several things depending on the nature of #mainlooptableid
-- .
-- .
-- .
-- End of this blocks main logic
delete from queueofids where uid = #mainlooptableid
select #mainlooptableid = min(uid)
from queueofids with (nolock)
end
I would like to be able to run the segment of code that's inside the while loop parallel for all uids inside the queueofids table. Based on what happens inside the loop I can guarantee that they will not interfere with each other in any way if they were to run concurrently, so logically it seems perfectly safe for it to run like this. The real question is if there is any way to get sql to run a portion of code for all values in there?
NOTE: I did think about generating a temp table with a series of created sql statements stored as strings, where each one is identical except for the #mainlooptableid value. But even if I have this table of sql statements ready to execute, I'm not sure how I would get all of these statements to execute concurrently.
I can't think of a way to do this within a single SQL script; scripts are procedural. If you want to explore this idea, you'd probably need to involve some form of multi-threaded application which would handle the looping aspect, and open a thread to hand off the parallelized portion of your current script. Not impossible, but it does introduce some complexity.
If you want to do this all in SQL, then you'll have to rewrite the code to eliminate the loop. As noted in the comments above, SQL Server is set-based, which means that it handles a certain amount of parallelization by doing work "all at once" against a set.
No, there is no way to get SQL statements in the same script to run in parallel.
The closest thing to it is to try to create a set-based way of handling them, instead of running them in a loop.
Be aware that run in parallel will not necessarily make it faster if the threads are competing for the same resources.
I don't think SQL will parallelize statements. But SQL will parallelize execution within a single statement.
Most programming frameworks have parallel. For example in .NET this would rather straight forward. Create a procedure where you pass #mainlooptableid and just call it in parallel.

Execute SQL task with T-SQL query and with stored procedure in SSIS

In SSIS Execute SQL Task currently I am calling a stored procedure and inside the procedure I have a MERGE statement.
Is there any difference if I call that query (T-SQL MERGE) directly in the Execute SQL Task?
(are there any differences like Log will create if we used SP?)
Please reply me...
Thanks in advance
Stored procedure will provide you with a more maintainable solution as you will be able to leverage code reuse and there will be no need to change / re-release the package if your query logic changes
A stored procedure is also likely to provide you with the fastest execution time as it will be compiled and the execution plan will be reused on subsequent runs

can we execute different queries in a stored procedure same time?

In my stored procedure there are 8 queries for 8 tables.Each query has joins and sub queries. I am passing parameters & stored procedure name from
front end(designed in asp.net 3.5)
can we execute that 8 queries at the same time i.e. parallel execution so that I can minimize stored procedure execution time?
Regards,
N.SRIRAM
The stored procedure runs each each query within it sequentially. If you want to run more than one at once, you'll have to create threads in your asp.net application and run each query in it's own thread. However, due to locking and processor constraints, this may not necessarily run faster.

Stored Procedure taking time in execution

I am breaking my head on this issue since long. I have stored procedure in MS SQl and when I try to execute that procedure by providing all the parameters in SQL Query, it takes long time to execute but when I try to directly run the query which is there in SP it executes in no time. This is affecting my application performance also as we are using stored procedures to fetch the data from DB Server.
Please help.
Regards,
Vikram
Looks like parameter sniffing.
Here is a nice explanation: I Smell a Parameter!
Basically, sql server has cached query execution plan for the parameters it was first run with so the plan is not optimal for the new values you are passing. When you run the query directly the plan is generated at that moment so that's why it's fast.
You can mark the procedure for recompilation manually using sp_recompile or use With Recompile option in its definition so it is compiled on every run.

Parallelism in PL/SQL

how can i run one query in pl-sql- in parallel?
i need all the flow...
You can create JOBs in order to run the same query with parallelism.
EXAMPLE
CREATE OR REPLACE PROCEDURE target_deletion
IS
number_of_the_job NUMBER;
BEGIN
DBMS_JOB.submit (number_of_the_job, 'begin stored_procedure_for_deletion; end;', SYSDATE);
END;
/
EXPLAINATION
Please suppose you have, in your Oracle DataBase, a stored procedure called exactly as follows:
stored_procedure_for_deletion
If you wish to execute that stored procedure many times with PARALLELISM, you have to create a stored procedure called for example "TARGET_DELETION" (written above), which creates a new job that invokes, with the PL/SQL block:
begin stored_procedure_for_deletion; end;
... the execution of your procedure called "stored_procedure_for_deletion".
The job starts immediately, so you can run the stored procedure target_deletion many consecutive times, in order to run the same procedure with parallelism.
If enabled at instance level, Oracle itself has parallel query features:
http://www.orafaq.com/wiki/Parallel_Query_FAQ
edit: it's not clear what you are trying to do. Maybe you want asynchronous query execution, so the Oracle job suggestion is correct (see the other answer).
Oracle parallel feature is not asynchronous, it just says the optimizer to use a certain number of CPUs in query execution, to speed up the result. For example:
select /*+ PARALLEL(DOGS,4) */ * from DOGS where A>20
executes your query with parallelism at degree 4.