I'm using CloudWatch to monitor other metrics but I can't find response time metric. Am I missing something here?
Pls configure the following cloudwatch metrics
Read - SelectLatency
Write - DMLLatency
Related
I have the amazon-cloudwatch-agent configured to retrieve custom metrics via collectd.
That is working in the sense that I can see the metrics in CloudWatch UI, but they have no unit (for example the collectd_GenericJMX_java_heap_used should have Bytes as CloudWatch Unit).
As far as I know collectd does not have the concept of units, and it seems that neither does telegraf.Metric (which I think is the underlying concept that is used in the amazon-cloudwatch-agent to ingest collectd metrics).
Is there any way, to specify the "units" for collectd metrics?
We are using Cloud sleuth to instrument microservices that use PubSub (using spring cloud stream binder). Currently PubSub traces are not getting propagated automatically to GCP cloud trace. So we are writing the traces using google cloud trace service client which internally calls trace v2 API. For each span we also have to set start time and end time.
In order to handle a retry scenario, I need to get start time of span. Is there a provision to get start time and end time of span? Tracer object does not have method to return these values. Any help will be appreciated.
No you can't get them from a Span. You can get them only when the span has been finished. If you're using Brave as the Tracer implementation you can create your own SpanHandler and you can access the information from the MutableSpan instance.
we have multiple kinesis consumer applications(KCL 2.0) are consuming the data from the same kinesis stream. All the consumer is sending the metrics to cloud watch and in the cloud watch those are showing up.
If i wanted to specifically understood and scale to multiple instances of one consumer application. how can we achive that... ?
cloud watch metrics Get records iterator age, Incoming data - sum (Count)
KCL metrics are provided under separate namespace in cloudwatch. The namespace that is used to upload metrics is the applicationName that you provide in KCL configuration. So if you have multiple KCL application with differnt applicationName then, you will find those metrics in cloudwatch metric console under "custom namespaces"
complete list of KCL metrics can be found here
Is there an API point or whatever other queryable source where I can get the total queued data?:
setting up a little dataflow in NiFi to monitor NiFi itself sounds sketchy, but if it's a common practice, let's be it. Anyway, I cannot find the API endpoint to get that total
Note: I have a single NiFi instance: I don't have nor will implement S2S reporting since I am on a single instance, single node NiFi setup
The Site-to-Site Reporting tasks were developed because they work for clustered, standalone, and multiple instances thereof. You'd just need to put an Input Port on your canvas and have the reporting task send to that.
An alternative as of NiFi 1.10.0 (via NIFI-6780) is to get the nifi-sql-reporting-nar and use QueryNiFiReportingTask, you can use a SQL query to get the metrics you want. That uses a RecordSinkService controller service to determine how to send the results, there are various implementations such as Site-to-Site, Kafka, Database, etc. The NAR is not included in the standard NiFi distribution due to size constraints, but you can get the latest version (1.11.4) here, or change the URL to match your NiFi version.
#jonayreyes You can find information about how to get queue data from NiFi API Here:
NiFi Rest API - FlowFile Count Monitoring
I am sending messages to the queue via cron. So my queue receives thousands of messages at a time and then it processes those messages.
Now I want to set a monitor such that it notifies me when there are messages available but not a single message is processed in an hour.
It can be possible by using two metrics, 1)NumberOfMessagesSent and 2)ApproximateNumberOfMessagesVisible. (NumberOfMessagesSent <= 0 AND ApproximateNumberOfMessagesVisible > 0)
But in CloudWatch, I cannot set a combination of two metrics to set an alarm. Can anyone suggest how can I implement this? Or is there any other alternate monitoring system or way to achieve this?