Dependency Details:
InfluxDB version : v0.10.0
I am working on various infrastructure metrics for example CPU and RabbitMQ metrics. These all servers are type of m3.large on AWS. I want to send all these metrics to influxDB. I tried python-diamond for sending metrics to influxdb but InfluxDBHandler is not supported for InfluxDB versions > 0.9.
I am using influxDB v0.10.0
and I want to send RabbitMQ Metrics to InfluxDB server.I can't revert version to latest one (That is a dependency).What should I do ?.
If python-diamond doesn't support versions >0.9, I'd recommend that you use the Graphite plugin in Telegraf which should support diamond output.
Related
We are currently trying to use Prometheus/Grafana in order to monitor several RabbitMQ instances deployed on multiple Docker containers.
My question is quite simple, what's the difference between using the Rabbitmq Prometheus Exporter vs Prometheus Plugin for RabbitMQ ?
Does the exporter scrape different/more information compared to the Plugin ?
Is there an overhead when using the Pluging compared to the exporter ?
Is it just a question of RabbiMQ's version ?
What is the added value from using one of the two options?
So basically what approach is better or can they be used in combination.
I have not trued out the plugin, but as far as I read it exports the same metrics as the exporter. The plugin has the advantage that it does not add complexity:
You need to host the rabbit exporter (which is not much effort, but still you need to make sure it runs, is updated from time to time,...)
You need an account for the rabbit exporter that can query the metrics which is a security issue. Your credentials might get stolen or the exporter might get compromised and and attacker would have access to your rabbit cluster.
Since there might be network between your rabbit cluster and the Rabbit exporter there might be the situation the exporter cannot reach the cluster while the plugin could still produce the metrics.
These are not big issues, we use the exporter for years now and never had an issue with it, but still, if we would start from scratch, we would give the plugin a try.
I have a challenge - build and release monitoring system in consist of RabbitMQ cluster (with 3 nodes) and standalone Grafana server for visualisation metrics.
I have found in official documentation of prometheus plugin for RabbitMQ (documentation) next section:
This plugin is new as of RabbitMQ 3.8.0.
But i have cluster of version 3.6.8 and when i run the next command
rabbitmq-plugins enable rabbitmq_prometheus
The output is:
Error: The following plugins could not be found:
rabbitmq_prometheus
Upgrade the cluster now is not possible and my question is:
How do i may configure monitoring of the cluster without upgrade it and with prometheus (preferred option) and grafana?
Thanks in advance!
The Prometheus plugin is not the only way to monitor a RabbitMQ cluster.
You can also use the rabbitmq exporter in sidecar. If you are not on a docker platform, you can download the exporter from the release assets and install it as a service somewhere.
It would be best to install the exporter on every server hosting the RabbitMQ node because:
you will need to have as many install as there are nodes (Prometheus is service oriented monitoring)
from the settings, the exporter is accessing the management plugin interface of RabbitMQ; it should stay bound to localhost to reduce attack surface
If your hands are really tied, you can deploy them anywhere (let say on the same server) and point each exporter to a different RabbitMQ node. Prometheus configuration can then identify the underlying service.
- job_name: rabbitmq
honor_labels: true
static_configs:
- targets: ['monitoring-server:97001']
labels:
instance: 'rabbitmq_node_A'
- targets: ['monitoring-server:97002']
labels:
instance: 'rabbitmq_node_B'
# or play with relabeling to acchieve the same.
An important drawback is that there are more cases where the exporter may not be able to access RabbitMQ and you end up alert on events not impacting your RabbitMQ cluster.
I want to get flink's metrics information via REST api, my flink is managed by YARN, but after changing web.port configuration in flink-conf.yaml, the change has no affect, and the web.port in the flink dashboard is always 0. So I can not get the flink metrics information via REST api.
Environment:
ubuntu 16.04
openjdk-8
hadoop 2.7.1.2.3.6.0-3796
flink 1.4.0
When running Flink on Yarn, Flink will pick a random port (0) for the web UI in order to avoid port conflicts with other applications running on the same machine.
In order to access the Flink web UI you can query the Yarn web application proxy (YarnResourceManagerURL/proxy/application_/...). But be aware that only GET requests are properly forwarded to the Yarn application.
Alternatively, Flink logs the web UI URL to stdout when starting a Yarn session. Moreover, you can retrieve the chosen port from the log files. In newer versions (>= 1.5) Flink will log Rest endpoint listening at hostname:port on INFO level and in older versions (<= 1.4 or if using the legacy mode) Flink will log Web frontend listening at hostname:port.
The last time I checked the logstash doc said that the official redis input plugin uses "blpop" command to get messages from redis. (Correct me if I'm wrong.)
However, our redis cluster does not allow command "blpop". So I'm wondering is there any redis input plugin that uses "lpop" command? Or do I have to write my own.
Thx!
There wasn't any so I wrote my own version of logstash redis input plugin that using "lpop" instead of "blpop" (in the official logstash redis input plugin) here
I just started evaluating Redis. I am using Redis 2.8.19 which the most latest stable release. Redis 2.9 is still unstable and Redis 3.0 is just available for developer's preview (not recommended for production). I was tryin to setus a cluster of Redis and when I changed my redis.conf and appended
cluster-enabled yes
cluster-config-file nodes.conf
cluster-node-timeout 5000
and started my Redis server by
src/redis-server ./redis.conf
it gave me an error as follows
* FATAL CONFIG FILE ERROR *
Reading the configuration file, at line 2
'cluster-enabled yes'
Bad directive or wrong number of arguments
I googled the error and got to know that my version (2.8.19) does not support cluster. I was still unable to fine any such specification in Redis Docs. My question is simple. Does Redis 2.8.19 supports redis cluster configuration? Or I have to upgrade to Redis 2.9 or Redis 3.0. I am evaluating Redis because I need to deploy it in production. Please guide.
Redis Cluster support is only for versions >= 3.0.0. Redis 3.0.0 will be released as a stable version in a matter of days, it's a good idea to use it if you want to use Cluster. The cluster support is considered to be stable, however for it to be considered mature we want to see adoption. Btw there is already at least a very large site using it in production. Currently the most sane thing to do if you need Redis Cluster is to test it for your use case, and if it looks great, use it.
Redis cluster is supported only in Redis 3.0+ (which is now stable). I have written a simple API called "Simple Redis Cluster Client" which can be used in redis's sub 3.0 versions for running in a cluster like mode (Not precisely a cluster, it just distributes keys among redis nodes based on the key's hashcode, You can have a look # https://github.com/prash-mi/simple-redis-cluster-client
Cluster support for Redis is only from v3 - v2.8.19 doesn't do clustering.