Is there a way to take a Snapshot of CPU and Heap usage over time.
Say every hour for 2 days?
Related
I am using Azure SQL Database.
In the Regressed Queries section of the Azure SQL Database query store, there are Duration and Wait Time.
I thought that Duration includes Wait Time.
However, in some of the query stores I have seen, the Total Wait Time is larger than the Total Duration.
Does Duration not include Wait Time?
No, the duration time in Query Store regression queries section doesn't include wait time. Wait time has been calculated separately.
Duration time is the time (in ms) for which query was in execution.
Wait time is the time query was in waiting state when resources reach
limits.
Both these regression criteria in query store measured separately to monitor the performance. The wait time has multiple categories like parallelism, CPU, Network IO as shown in below image.
Please find some useful articles: Use the Regressed Queries feature, Wait Statistics and Query Store
We have redis cluster which holds more than 2 million and these keys has been updated with the time interval of 1 minute. Now we have a requirement to take the snapshot of the redis db in a particular interval For eg every 10 minute. This snapshot should not pause the redis command execution.
Is there any async way of taking snapshot from redis ?
It would be really helpful if we get any suggestion on open source tools or frameworks.
The Redis BGSAVE is async and takes a snapshot.
It calls the fork() function of the OS. According to the Redis manual,
Fork() can be time consuming if the dataset is big, and may result in Redis to stop serving clients for some millisecond or even for one second if the dataset is very big and the CPU performance not great
Two million updates in one minutes, that is 30K+ QPS.
So you really have to try it out, run the benchmark that similutes your business, then issue BGSAVE, monitor the I/O and CPU usage of your system, and see if there's a spike in your redis calling latency.
Then issue LASTSAVE, which will tell you when your last success snapshot happened. So you can adjust your backup schedule.
I Want to go for stress testing to start with the anticipated number of users (or just from 1 virtual user) and gradually increase the load such as for 10 threads, 20 threads, …. 100 threads until response time starts exceeding the acceptable value or errors start occurring.But For all this test run should i increases the Ramp-up Period(Seconds) or it will remain the same for all test?
Picture is given below:
Apparently, the Ramp-up time shouldn't be the same for all of your tests. You have to set the Ramp-up period accordingly.
Ramp up is the time in which all the users arrive on your tested application server.
You can check this thread also: How should I calculate Ramp-up time in Jmeter
As per JMeter documentation:
The ramp-up period tells JMeter how long to take to "ramp-up" to the full number of threads chosen. If 10 threads are used, and the ramp-up period is 100 seconds, then JMeter will take 100 seconds to get all 10 threads up and running. Each thread will start 10 (100/10) seconds after the previous thread was begun. If there are 30 threads and a ramp-up period of 120 seconds, then each successive thread will be delayed by 4 seconds.
Ramp-up needs to be long enough to avoid too large a work-load at the start of a test, and short enough that the last threads start running before the first ones finish (unless one wants that to happen).
Start with Ramp-up = number of threads and adjust up or down as needed.
So if you don't have a better idea - go for the ramp-up period in seconds equal to the number of users.
The point of ramp-up is increasing the load gradually so will be able to correlate increasing load with other performance metrics for websites like response time, throughput, number of server hits per second, number of errors per second, etc.
See JMeter Glossary for the metrics which are stored by JMeter explained
Currently, we are planning to generate threshold of 20K notifications per minute and add delay to fetch notification status at few intervals like 2 mins, 5 mins, 1 hour and then 1 day if the status isn't final.
I have done POC with
https://www.rabbitmq.com/blog/2015/04/16/scheduling-messages-with-rabbitmq/
it looks good but wanted to know real-time stats or any other suggestion before going live.
How do we empty the cache every n seconds (so that we can run queries on the data that has come in for the n second window - batch window querying)? I could only find FIFO and LRU based eviction policies in the ignite code where the eviction policy is based on the cache entry getting added or modified.
I understand we can have a sliding window using CreatedExpiryPolicy
cfg.setExpiryPolicyFactory(FactoryBuilder.factoryOf(new CreatedExpiryPolicy(new Duration(SECONDS, 5))));
But I don't think this will help me maintain batch windows. Neither will FIFO or LruEvictionPolicy.
I need some eviction policy which is based on some static time window (every 5 seconds for example).
Will I have to write my own implementation?
Well, it's possible to change ExpiryPolicy for each added entry with
IgniteCache.withExpiryPolicy
and calculate remaining time every time, but it will be too big overhead - each entry will have their own EvictionPolicy.
I would recommend to schedule job that will clear the cache using cron based scheduling: