Is it possible to execute a simple query against a Db but 100 of it at the same time, in parallel via TSQL in Management Studio?
(The idea is to see how it impacts the Db server in terms of performance)
you can execute your batch in a loop like in
select getdate()
go 100
it will not execute in parallel but 100 times one after another. if this is not enough, you can execute it in the same time in another session
You can use sqlcmd and put it in a batch to spawn 100 parallel processes. As an alternative you can use LinqPad and a simple C# script that uses Tasks and, too, spawns 100 processes.
If you want to have a parallel stored procedure execution, then you need to setup service broker and a queue with the length of 100, but you won't be having exactly 100 parallel activations at once because service broker adds workers as workload increases, so it won't happen instantaneously.
Related
My python program is supposed to execute approx 200 SQLs. 200 (inptLstCnt) is based on the number of tables received as the input to the function.
Instead running 200 SQLs one by one, i want use the same database connection instance, run 10 or certain pool of SQLs at a time and iterate until the inptLstCnt is met. I am quite confused what to choose between Multiprocessing pools and Threads.
Are there any code references for similar approach but not necessarily SQL execution or database connection.
Good day.
Need to get records from an Oracle database to a database in SQL Server. The data source type (ODBC) the performed using a SQL command, where I am taking all possible indices according to my requirement. The process runs fine, the problem is that it takes a long time and I need to be something quick. The process can not be performed with lookup, requires merge or merge join, simply load a table from Oracle to SQL under certain conditions.
Thank you for your help
Check what is your limiting factor. Generally there are 3 points to check:
Remote server is slow.
Source DB can run low on memory, read speed or free CPU. Substitute you query with a straight SELECT statement with no WHERE clause or JOINs and see if your SSIS package runs faster.
Target DB.
You may have indexes enabled, high write latency on HDD or not enough CPU.
Run an INSERT for your target table and see how longer it takes.
Problem may be in the middle: transfer between 2 servers. Network usually is main bottleneck. Is SSIS hosted on the same server as SQL server? then you have 2 network connections + possible hardware bottleneck on dedicated SSIS machine.
Depending on the bottleneck there are different solutions.
If you have network capacity and bottleneck is 1 CPU per query on Oracle, then you can partition your data horisontally (IDs 1 to 100, 101 to 200 etc); establish multiple connections to Oracle and load data in several streams. Number of streams is 1 less then number of CPUs on Oracle, SSIS or SQL Server (which ever is smaller).
I have the following code running to catch any SQL statements that might get hung. While trying to test this, I wrote a horibly optimized sql statement that takes over a minute to run. I put a 20 second timeout wrapper around an activerecord execute sql statement, but it doesn't seem to interrupt the sql call for taking to long. This is running against Oracle databases.
start_time = Time.now
Timeout.timeout(20) do #20 timeout for long running sql
#connection.connection.execute(sql_string)
end
total_sql_time = Time.now - start_time
puts "Sql statement took #{total_sql_time} seconds
output
Sql statement took 64 seconds
(Just a guess…)
Ruby's Timeout.timeout() is probably operating in much the same way as the equivalent function in PHP, which is to say that it counts time executed within Ruby, and stops counting when you're waiting for database queries to run, waiting to finish reading or writing files, waiting to for a network request to complete, etc. — i.e. waiting for anything IO-related.
Admittedly, I don't have much experience with Rails backed by Oracle, but there's the timeout option in the database config file.
# config/database.yml
production:
...
timeout: 20000
It's in milliseconds.
If I can assume you are in the development environment, you might want to explicitly set your explain timeout and review the console log:
# config/environments/development.rb
config.active_record.auto_explain_threshold_in_seconds 20
Remember that ActiveRecord is an ORM, which is to say it's a layer of abstraction that sits on top of your database.
This is important in the context of what happens when Timeout.timeout executes. Database adapters are responsible for taking a command in ruby and converting them into commands that are executed in your database, an entirely different process.
When a timeout happens, the call to #connection.connection.execute, as a ruby command, is interrupted in the adapter but not the sql code itself as you've noted.
You should be able to make use of the optional Exception Class parameter klass in Timeout.timeout to specify a signal/command to send to your database in order to cancel the query itself. However this would depend on your database adapter. Here's the kill query command I would try to execute if I were using mysql.
On a legacy SQL 2000 box that is awaiting upgrade
Does anyone have a script to kill / abort a job in SQL Agent 2000 if it over runs a set duration or overlaps with another agent job.
This is causing us a serious problem every few weeks a job overuns then locks the reindex job and that freezes our 24 / 7 server
One solution would be to add a step at the beginning of the job with which the long-running job overlaps which checks whether the long-running job is still running, and if it is issue an sp_stop_job command.
You can get the current execution status of a job using sp_help_job, but its output is not very easy to manipulate in a query.
Various methods of coercing the output of sp_help_job into a temp table in SQL 2000 are discussed here - alternatively you can query msdb.sysjobhistory to get the execution status information.
Is it possible to make Job Schedule, which will occur every less than 10 seconds?
Because Sql server doesn't allow that.
Schedule type is "Recurring" and Occurs "Daily".
Select occurs Daily and run every 10 seconds. Although keep in mind if your job takes longer than the time specified to run, the agent won't invoke it again until the running task completes.
See comments below. The UI tool doesn't allow input for less than 10 seconds. In your case you could schedule two job tasks offset by some number of seconds. The issue then is that the jobs could overlap as they are distinct tasks as far as SQL Agent knows.
I tried do something similar (SQL Server job with prcise timing), but the only reliabile way I found was custom windows service execution SQL statements.