I am using Azure SQL Database.
In the Regressed Queries section of the Azure SQL Database query store, there are Duration and Wait Time.
I thought that Duration includes Wait Time.
However, in some of the query stores I have seen, the Total Wait Time is larger than the Total Duration.
Does Duration not include Wait Time?
No, the duration time in Query Store regression queries section doesn't include wait time. Wait time has been calculated separately.
Duration time is the time (in ms) for which query was in execution.
Wait time is the time query was in waiting state when resources reach
limits.
Both these regression criteria in query store measured separately to monitor the performance. The wait time has multiple categories like parallelism, CPU, Network IO as shown in below image.
Please find some useful articles: Use the Regressed Queries feature, Wait Statistics and Query Store
Related
I'm trying to make sense of Process Default behavior on SSAS 2017 Enterprise Edition.
My cube is processed daily in this standard sequence:
Loop through 30 dimensions and performing Process Add or Process Update as required.
Process approximately 80 partitions for the previous day.
Exec a Process Default as the final step.
Everything works just fine, and for the amount of data involved, performs really well. However I have observed that after the process default completes, if I re-run the process default step manually (with no other activity having occurred whatsoever), it will take exactly the same time as the first run.
My understanding was that this step basically scans the cube looking for unprocessed objects and will process any objects found to be unprocessed. Given the flow of dimension processing, and subsequent partition processing, I'd certainly expect some objects to be unprocessed on the first run - particularly aggregations and indexes.
The end to end processing time is around 65 mins, but 10 mins of this is the final process default step.
What would explain this is that if the process default isn't actually finding anything to do, and the elapsed time is the cost of scanning the meta data. Firstly it seems an excessive amount of time, but also if I don't run the step, the cube doesn't come online, which suggests it is definitely doing something.
I've had a trawl through Profiler to try to find events to capture what process default is doing, but I'm not able to find anything that would capture the event specifically. I've also monitored the server performance during the step, and nothing is under any real load.
Any suggestions or clarifications..?
Whenever I execute a query in Bigquery, I can see that in the Explanation tab the Waiting time is always average. Is it possible to execute without wait time or to reduce the wait time.
This image shows the query explanation (Bigquery wait time is average here)
Why Big Query always has waiting time?
You might have more work than can be immediately scheduled.
is it possible to execute without waiting time?
Purchase more BigQuery Slots.
Contact your sales representative for more information or support.
Otherwise, just wait. If your job isn’t time-critical, you can schedule it and wait for it to be executed as resources permit.
Bigquery has a complex scheduling system. Besides, usually there are couple stages for a simple query, and for each stage scheduler needs to find the best shard(node) which could execute the computation.
It takes some time for the scheduler to send the job to shards. But usually it should not be significantly long.
I have a redis server running and I wanted to use JMeter to get the benchmarks and to find in how much time it hits 20K transactions per second. I have a hash setup. How should I go about querying it. I have put one of the keys as redis key and have put one of the fields of the hash as variable name.
If I use constant throughput timer, what should I enter in the name field.
Thanks in advance.
If you're planning to use Constant Throughput Timer and your target it to get 20k requests per second load you need to configure it as follows:
Target Throughput: 1200000 (20k per second * 60 seconds in minute)
Calculate Throughput based on: all active threads
See How to use JMeter's Throughput Constant Timer article for more details.
Few more recommendations:
Constant Throughput Timer can only pause the threads so make sure you have enough virtual users on Thread Group level
Constant Throughput Timer is accurate enough on "minute" level, so make sure your test lasts long enough so the timer will be correctly applied. Also consider reasonable ramp-up period.
Some people find Throughput Shaping Timer easier to use
20k+ concurrent threads is normally something you cannot achieve using single machine so it is likely you'll need to consider Distributed Testing when multiple JMeter instances act as a cluster.
The incentive to use batch queries instead of interactive mode queries was pricing, but with newer price changes there is no cost difference anymore - so is there any other incentive (quota, performance, other...) to use batch queries?
With the price change, there are two primary reasons to use batch priority:
it lets you queue up your jobs.
it lets you run low priority queries in a way that doesn't impact high priority ones.
There are a number of rate limits that affect interactive (i.e. non-batch) queries -- you can have at most 20 running concurrently, there are concurrent byte limits and 'large query' limits. If those limits are hit, the query will fail immediately. This is because BigQuery assumes that an interactive query is something you need run immediately.
When you use batch, if you ever hit a rate limit, the query will be queued and retried later. There are still similar rate limits, but they operate separately from interactive rate limits, so your batch queries won't affect your interactive ones.
One example might be that you have periodic queries that you run daily or hourly to build dashboards. Maybe you have 100 queries that you want to run. If you try to run them all at once as interactive, some will fail because of concurrent rate limits. Additionally, you don't necessarily want these queries to interfere with other queries you are running manually from the BigQuery Web UI. So you can run the dashboard queries at batch priority and the other queries will run normally as interactive.
One other point to note is that the scheduling for Batch queries has changed so the average wait times should come down considerably. Instead of waiting a half hour or so, batch queries should start within a minute or two (subject to queueing, etc).
currently as a single user, it takes the 260ms for a certain query to run from start to finish.
what will happen if I have 1000 queries sent at the same time? should I expect the same query to take ~4 minutes? (260ms*1000)
It is not possible to make predictions without any knowledge of the situation. There will be a number of factors which affect this time:
Resources available to the server (if it is able to hold data in memory, things run quicker than if disk is being accessed)
What is involved in the query (e.g. a repeated query will usually execute quicker the second time around, assuming the underlying data has not changed)
What other bottlenecks are in the system (e.g. if the webserver and database server are on the same system, the two processes will be fighting for available resource under heavy load)
The only way to properly answer this question is to perform load testing on your application.