We have an existing Hadoop cluster that is not managed by Ambari. Is it possible to install Apache Ambari on top of an existing Hadoop cluster?
No, Ambari must provision the cluster it's monitoring.
Ambari is designed around a Stack concept where each stack consists of several services. A stack definition is what allows Ambari to install, manage and monitor the services in the cluster.
Related
I have a challenge - build and release monitoring system in consist of RabbitMQ cluster (with 3 nodes) and standalone Grafana server for visualisation metrics.
I have found in official documentation of prometheus plugin for RabbitMQ (documentation) next section:
This plugin is new as of RabbitMQ 3.8.0.
But i have cluster of version 3.6.8 and when i run the next command
rabbitmq-plugins enable rabbitmq_prometheus
The output is:
Error: The following plugins could not be found:
rabbitmq_prometheus
Upgrade the cluster now is not possible and my question is:
How do i may configure monitoring of the cluster without upgrade it and with prometheus (preferred option) and grafana?
Thanks in advance!
The Prometheus plugin is not the only way to monitor a RabbitMQ cluster.
You can also use the rabbitmq exporter in sidecar. If you are not on a docker platform, you can download the exporter from the release assets and install it as a service somewhere.
It would be best to install the exporter on every server hosting the RabbitMQ node because:
you will need to have as many install as there are nodes (Prometheus is service oriented monitoring)
from the settings, the exporter is accessing the management plugin interface of RabbitMQ; it should stay bound to localhost to reduce attack surface
If your hands are really tied, you can deploy them anywhere (let say on the same server) and point each exporter to a different RabbitMQ node. Prometheus configuration can then identify the underlying service.
- job_name: rabbitmq
honor_labels: true
static_configs:
- targets: ['monitoring-server:97001']
labels:
instance: 'rabbitmq_node_A'
- targets: ['monitoring-server:97002']
labels:
instance: 'rabbitmq_node_B'
# or play with relabeling to acchieve the same.
An important drawback is that there are more cases where the exporter may not be able to access RabbitMQ and you end up alert on events not impacting your RabbitMQ cluster.
By this famous guestbook example:
https://github.com/kubernetes/examples/tree/master/guestbook
It will create Redis master/slave deployment and services. It also has a subfolder named redis-slave which used for create a docker image and run Redis replication command.
Dockerfile
run.sh
The question is, if deployed the Redis master and slave to the k8s cluster. Then how to run that command? Deploy a new container? That will not relate to the slave container already deployed.
Is there a better way to do Redis repliaciton between master and slave running in k8s cluster?
One option you have is using helm to deploy the redis-ha app.
Info about helm: https://github.com/kubernetes/helm
The redis-ha helm app page: https://hub.kubeapps.com/charts/stable/redis-ha
Redis Sentinel is often suggested for simple master-slave replication and high availability.
Unfortunately, Sentinel does not fit Kubernetes world well and it also requires a Sentinel-aware client to talk to Redis.
You could try Redis operator which can be considered a Kubernetes-native replacement for Sentinel and allows to create a Redis deployment that would resist without human intervention to most kind of failures.
Here is how you can setup Redis HA Master Slave Cluster in Kubernetes/Openshift OKD
Basically you have to use configMap, StatefulSet in collaboration with VolumeClaims
https://reachmnadeem.wordpress.com/2020/10/01/redis-ha-master-slave-cluster-up-and-running-in-openshift-okd-kubernetes/
I have 3 virtual machines that I want to add to an Ambari cluster.
I'm going through the setup wizard to do this.
My VMs have less than 2GB of space in the drive mounted on /
Ambari complains about this.
Is there any way to tell Ambari that I want it to use a different location?
I would like to tell Ambari to use the /data location for each host.
Any help will be greatly appreciated.
Assuming you're attempting to install an HDP stack. No. You will need more space on the root partition. All of the services (Spark, HBase, HDFS, Yarn, Oozie, etc.) are installed into /usr/hdp/<version>. This is not overridable since each of these services is installed using an rpm and the default location in which those rpms install into is hard coded throughout several of the service descriptors that help provision the cluster.
I have a small hadoop cluster i.e one master and three slave nodes. I have to monitor cluster. I have found that we can use Ambari. CentOS 7 is installed on all machines. Please provide a complete details how I can do that ?. I have found that Ambari can be used for new cluster i.e you have to install new cluster. It does not work with already running cluster?
At the moment Ambari does not support CentOS 7, so that's not going to work.
However, Ambari does not perform cluster monitoring on its own. It uses Nagios for the purpose. Nagios is an independent software project that you can setup independently. That said it's kinda painful to do.
ambari-server for Ambari 2.2+ can be installed and works good on CentOS 7.
You have to installed ambari-server on one of the hosts (master node) and can use the webUI hostname:8080 for installing ambari agents on other hosts. Alternatively, ambari agents can be installed manually on other hosts can can be linked to communicate with the ambari-server.
I have listed few queries related to ambari as follows:
Can I configure more than one hadoop cluster via UI of ambari ?
( Note : I am using Ambari 1.6.1 for my hadoop cluster deployment purpose and I am aware that this can be done via Ambari API, but not able to find via ambari portal)
We can check the status of services on each node by “jps” command, if we have configured hadoop cluster w/o ambari.
Is there any way similar to “jps” to check from back end if the setup for hadoop cluster was successful from the backend ?
( Note : I can see that services are showing UP on the ambari portal )
Help Appreciated !!
Please let me know if any additional information is required.
Thanks
The ambari UI is served up by the ambari server for a specific cluster. To configure another cluster you need to point your browser to the URL for that other cluster's ambari server. So you can't see the configuration for multiple servers on the same web page, but you can set browser bookmarks to jump from configuration to configuration.