Can someone let me know if its actually possible to pause Azure Synapse Analytics SQL Dedicated Pool after a period of activity?
I know its possible to shut down VMs in Spark Clusters after a period of inactivity.
Microsoft have provided the following link on how to automate the pause / resume of SQL Dedicated Pool. https://learn.microsoft.com/en-us/azure/synapse-analytics/sql/how-to-pause-resume-pipelines
But that link doesn't appear to answer the question pausing after a period of inactivity
This can be achieved by running monitoring queries in the SQL pool using script activity or stored procedure activity.
Use the following query in Script activity :
select count(*) as queriesRunning from sys.dm_pdw_exec_requests WHERE status='Running' and session_id <> session_id();
SELECT min(DATEDIFF (mi, end_compile_time, getutcdate())) as minutesPassed FROM sys.dm_pdw_exec_requests WHERE session_id <> session_id()
Use the below expression in If condition block:
#and(equals(activity('Script1').output.resultSets[0].rows[0].queriesRunning,0),greaterOrEquals(activity('Script1').output.resultSets[3].rows[0].minutesPassed,2))
Case 1: When query is running :
equals(activity('Script1').output.resultSets[0].rows[0].queriesRunning,0 will result in false
Case 2: When there is no active query but minutes passed is < 2:
Case 3: When there is no active query and minutes passed is >=2:
Related
This was an interview question. Let me explain it
Let's say we have an Update query and it will take 30 seconds to complete but during that time machine on which we have SQL server installed, gets powered off.
What will happen to data in the following two cases?
Case 1: Update query without Transaction
Case 2: Update query within Transaction
I have a table with about 4 million rows. What I'd like to do is to add two more columns and then update the values of these two columns based on the third column in this same table. Basically I'm trying to set IsoWeek and IsoYear based on ReportDate.
I've added the columns and all the values are NULL, now I've started with simple update all script like below:
UPDATE Report
SET IsoWeek = DATEPART(ISO_WEEK, ReportDate), IsoYear = dbo.ISO_YEAR(ReportDate)
It took 5sec locally, but it was over 10min on Azure test DB so I cancelled and reimplemented the query with batches. It was around the same 5sec locally, but on Azure test DB it was still super slow. This time I've waited more and it completed in about 45 minutes.
I have to run a similar script on PROD Azure DB, so now I'm trying to find ways to optimize this update.
I've added WHERE Id <= 50000 to update only one chunk:
UPDATE Report
SET IsoWeek = DATEPART(ISO_WEEK, ReportDate), IsoYear = dbo.ISO_YEAR(ReportDate)
WHERE Id <= 50000
This query executed locally in 0sec and about 7sec on Azure TEST db. This seems like a good comparison test and I started comparing execution plans.
Locally:
Azure TEST db:
So I'm not sure why is it different on local and Azure Test DB and how can I make it faster on Azure.
Any ideas?
UPD:
When I removed dbo.ISO_YEAR, execution plan is now better but execution time went down from 7sec to 6sec only.
Looks like you have a scalar UDF in your query, causing a table spool, plus a lot of context switching. Azure will not inline these UDFs.
The table spool might be removed by changing the UDF to use SCHEMABINDING, but you're best off inlining it yourself, either direct in the query or as an iTVF.
Here is a request to add scalar UDF inlining to Azure:
https://feedback.azure.com/forums/217321-sql-database/suggestions/38955436-bring-scalar-udf-inlining-to-azure-sql-database
There are many things that could be different on Azure SQL vs SQL Server on-premises and that may affect performances. For example:
are you using Simple Recovery model on SQL Server? Azure SQL always run in Full Recovery
are you using ADR on SQL Server? Azure SQL always run with ADR on
are you using TDE on SQL Server? Azure SQL has TDE enabled by default
Also, you don't mention with Azure SQL Tier are you using. Standard/GeneralPurpose or Premium/BusinessCritical? Or Hyperscale? How many cores or DTUs?
I am developing a custom system for scheduling task & I am following Microsoft SQL Server Agent approach to schedule task.
I have replicated sysschedules table for capturing users scheduling occurrences and timings.
Now I need to know how SQL Server get the Next_run_date and time from this table.
I have found a solution on the internet which are using sysjobhistory table to get the next run date, but in my case, I have an only sysschedules table. So, what is the formula to calculate the next_run_date and time for any job?
Note: I am looking for entire code, concept/algorithm will work for me.
I am trying to optimize an SQL process using the dmv ([sys].[dm_os_wait_stats]).
Is there any way that we can see the waiting/suspended queries between a time period in the past?. Like want to have records only from 3pm today.
Currently I clean the instance every time before running the process using
DBCC SQLPERF ('sys.dm_os_wait_stats', CLEAR);
GO
I suggest that using monitoring tools such as Idera or Redgate monitor in order to monitor sql server waiting. You can also copy ([sys].[dm_os_wait_stats]) data in other table periodically.
On a legacy SQL 2000 box that is awaiting upgrade
Does anyone have a script to kill / abort a job in SQL Agent 2000 if it over runs a set duration or overlaps with another agent job.
This is causing us a serious problem every few weeks a job overuns then locks the reindex job and that freezes our 24 / 7 server
One solution would be to add a step at the beginning of the job with which the long-running job overlaps which checks whether the long-running job is still running, and if it is issue an sp_stop_job command.
You can get the current execution status of a job using sp_help_job, but its output is not very easy to manipulate in a query.
Various methods of coercing the output of sp_help_job into a temp table in SQL 2000 are discussed here - alternatively you can query msdb.sysjobhistory to get the execution status information.