I want to test the resource autoscaling policy for my microservices.
I have workload concurrency traces that record the historical metrics. Are there any tools like (Jmeter, wrk2, hey) that support custom generate patterns according to historical data?
While these tools (Jmeter, wrk2, hey) do not support this feature.
Thanks a lot!
Related
There are documents available for to mention that standard load balancer have monitor metrics: https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-monitor-log
But, I need to understand why do the basic ones not have any monitor metrics. Is it because of pricing? If yes, is there any official document to prove that?
No, Basic Load Balancers don’t support metrics. Azure Monitor multi-dimensional metrics are only available for Standard Load Balancer, which is also the recommended SKU by Microsoft.
Azure offers a Basic SKU and Standard SKU that have different functional, performance, security and health tracking capabilities. These differences are explained in the SKU Comparison article.
Upgrading to the Standard SKU is recommended for any production workloads to take advantage of the robust set of Load Balancer metrics.
We have our applications (Map Reduce Jobs, microservices) completely running out of AWS.
We intend to use a single service for viewing (for debug purposes), monitoring and alarms (notifications based on a threshold) on logs.
Is there any specific benefits of using external service providers like sumo logic over the one provided by AWS itself (cloudwatch in this case)
In full disclosure, I'm an engineer at Sumo Logic. But here is an analysis done by one of my colleagues a few months ago as to why you would want to use Sumo Logic specifically over AWS Cloudwatch itself:
You can’t easily search across multiple Cloudwatch Log Groups in Cloudwatch. In Sumo, you can define metadata to easily query across log groups, and even log sources outside of AWS within the same query.
Cloudwatch does not include any pre-built apps (out of the box queries, dashboards, and alerts). Sumo Logic provides a wide variety of apps and integrations, all included with your subscription: https://www.sumologic.com/applications/
With Cloudwatch, you pay for dashboards, you pay to query your data, and you pay to use its query API. These are all included in your Sumo Logic subscription (depending on the subscription level you choose).
Sumo provides 30 days of log retention out of the box. Data retention is another a la carte cost when using CloudWatch. Sumo Logic also provides you with the ability to forward your logs off to S3 for long-term storage outside of our platform.
Cloudwatch does not include advanced analytics operators. Sumo Logic includes operators like Outlier, Log Reduce, and Log Compare, which are all part of the Sumo Logic platform.
Regarding search time, Sumo Logic vs AWS CloudWatch Insights (AWS CloudWatch log search): Here is a quote from a customer with 100 AWS accounts worth of CloudTrail logs: "We can search all 100 of our accounts in Sumo in the same amount of time it takes us to search 1 account with AWS's CloudWatch.”
Sumo Logic provides Threat Intelligence as part of your subscription as well, to be able to check all of your logs against Crowdstrike’s threat database of known IoC’s.
Sumo training and certification is included with your subscription.
On a personal note, I can also say that Sumo Logic's post-sales support is top-notch, as we put a huge emphasis on customer success.
Please keep in mind that this analysis by my colleague is a few months old, so some of these items may have been addressed by AWS since then.
I'm trying to architect a microservice based system utilizing BigQuery as one of services. We need to preserve eventual consistency between BigQuery and other microservices, so that changes to BigQuery (data uploads, table creates, etc) were eventually propagated to other services.
I'm wondering if BigQuery has mechanisms, supporting this kind of consistency? As I checked, BigQuery does not support publishing its events to pub/sub, which would definitely solve a problem.
I'm thinking of utilizing labels for this. I hope updates of data and labels should be atomic in respect to one API call.
Something like keeping two labels with current version and committed version, and maybe uncommitted operation type. Mutation operation increases current version and queues task, publishing update to pub/sub, which on success updates committed version to match the current one. I though see a number of problems with this solution.
Basically, there is a broader question, of how APIs need to be designed to support eventual consistency with other systems, and if it is possible to use API not specially designed for this, in an eventually consistent distributed system.
I am writing software that creates a large graph database. The software needs to access dozens of different REST APIs with millions of total requests. The data will then be processed by the Hadoop cluster. Each of these APIs have rate limits that vary by requests/second, per window, per day and per user (typically via OAuth).
Does anyone have any suggestions on how I might use either a Map function or other Hadoop-ecosystem tool to manage these queries? The goal would to be to leverage the parallel processing in Hadoop.
Because of the varied rate limits, it often makes sense to switch to a different API query while waiting for the first limit to reset. An example would be one API call that creates nodes in the graph and another that enriches the data for that node. I could have the system go out and enrich the data for the new nodes while waiting for the first API limit to reset.
I have tried using SQS queuing on EC2 to manage the various API limits and states (creating a queue for each API call), but have found it to be ridiculously slow.
Any ideas?
It looks like the best option for my scenario will be using Storm, or specifically the Trident abstraction. It gives me the greatest flexibility for both workload management but process management as well
I've been trying to find out ways to improve our nservicebus code performance. I searched and stumbled on these profiles that you can set upon running/installing the nservicebus host.
Currently we're running the nservicebus host as-is, and I read that by default we are using the "Lite" version of the available profiles. I've also learnt from this link:
http://docs.particular.net/nservicebus/hosting/nservicebus-host/profiles
that there are Integrated and Production profiles. The documentation does not say much - has anyone tried the Production profiles and noticed an improvement in nservicebus performance? Specifically affecting the speed in consuming messages from the queues?
One major difference between the NSB profiles is how they handle storage of subscriptions.
The lite, integration and production profiles allow NSB to configure how reliable it is. For example, the lite profile uses in-memory subscription storage for all pub/sub registrations. This is a concern because in order to register a subscriber in the lite profile, the publisher has to already be running (so the publisher can store the subscriber list in memory). What this means is that if the publisher crashes for any reason (or is taken offline), all the subscription information is lost (until each subscriber is restarted).
So, the lite profile is good if you are running on a developer machine and want to quickly test how your services interact. However, it is just not suitable to other environments.
The integration profile stores subscription information on a local queue. This can be good for simple environments (like QA etc.). However, in a highly distributed environment holding the subscription information in a database is best, hence the production profile.
So, to answer your question, I don't think that by changing profiles you will see a performance gain. If anything, changing from the lite profile to one of the other profiles is likely to decrease performance (because you incur the cost of accessing queue or database storage).
Unless you tuned the logging yourself, we've seen large improvements based on reduced logging. The performance from reading off the queues is same all around. Since the queues are local, you won't gain much from the transport. I would take a look at tuning your handlers and the underlying infrastructure. You may want to check out tuning MSMQ and look at the disk you are using etc. Another spot would be to look at how distributed transactions are working assuming you are using a remote database that requires them.
Another option to increase processing time is to increase the number of threads consuming the queue. This will require a license. If a license is not an option you can have multiple instances of a single threaded endpoint running. This requires you shard your work based on message type or something else.
Continuing up the scale you can then get into using the Distributor to load balance work. Again this will require a license, but you'll be able to add more nodes as necessary. All of the opportunities above also apply to this topology.