Ambari cluster : Host registration failed - ssh

I am setting up an ambari cluster with 3 virtualbox VMs running Ubuntu 16.04LTS.
I followed this hortonworks tutorial.
However when I am going to create a cluster using Ambari Cluster Install Wizard I get the below error during the step 3 - "Confirm Hosts".
26 Jun 2017 16:41:11,553 WARN [Thread-34] BSRunner:292 - Bootstrap process timed out. It will be destroyed.
26 Jun 2017 16:41:11,554 INFO [Thread-34] BSRunner:309 - Script log Mesg
INFO:root:BootStrapping hosts ['thanuja.ambari-agent1.com', 'thanuja.ambari-agent2.com'] using /usr/lib/python2.6/site-packages/ambari_server cluster primary OS: ubuntu16 with user 'thanuja'with ssh Port '22' sshKey File /var/run/ambari-server/bootstrap/5/sshKey password File null using tmp dir /var/run/ambari-server/bootstrap/5 ambari: thanuja.ambari-server.com; server_port: 8080; ambari version: 2.5.0.3; user_run_as: root
INFO:root:Executing parallel bootstrap
Bootstrap process timed out. It was destroyed.
I have read number of posts saying that this is related to not enabling Password-less SSH to the hosts. But I can ssh to the hosts without password from the server.
I am running ambari as non-root user with root privileges.

This post helped me.
I modified the users in host machines so that they can execute sudo commands without password using visudo command.
Please post if you have any alternative answers.

Related

facing the problems with dns-api-go setup in google cloud

I created the Ubuntu virtual machine on that I created the Go environment after that I successfully imported and installed the skx/dns-api-go
I followed this doc: https://github.com/skx/dns-api-go for to set up the environment also I installed the required dependencies like
Dancer,Plack::Middle ware::Throttle::Lite,Net::DNS::Resolver,twiggy,
libdancer-perl ,libnet-cidr-lite-perl ,
libplack-middleware-reverseproxy-perl.
after that, at last, I run the following command:
dns-api-go -redis-server localhost:6379
then am getting the following result :
Launching the server on http://127.0.0.1:9999 redis: 2018/09/25
11:18:56 ring.go:263: ring shard state changed: Redis is down.
when I changed the port number in web preview in cloud shell it is showing as your server is not listing to 6379 port number, but in the instance level when I run the netstat -an command it is showing as the server is listing for 6379 port
please help me in resolving these issue.
6379 is redis server port which is there for rate limiting feature. If you go through the main.go file, you'll see go server port is 9999. Just run "dns-api-go" command and hit http://localhost:9999/

Could not connect to Redis at 127.0.0.1:6379: Connection refused with homebrew

Using homebrew to install Redis but when I try to ping Redis it shows this error:
Could not connect to Redis at 127.0.0.1:6379: Connection refused
Note :
I tried to turn off firewall and edit conf file but still cannot ping.
I am using macOS Sierra and homebrew version 1.1.11
After installing redis, type from terminal:
redis-server
And Redis-Server will be started
I found this question while trying to figure out why I could not connect to redis after starting it via brew services start redis.
tl;dr
Depending on how fresh your machine or install is you're likely missing a config file or a directory for the redis defaults.
You need a config file at /usr/local/etc/redis.conf. Without this file redis-server will not start. You can copy over the default config file and modify it from there with
cp /usr/local/etc/redis.conf.default /usr/local/etc/redis.conf
You need /usr/local/var/db/redis/ to exist. You can do this easily with
mkdir -p /usr/local/var/db/redis
Finally just restart redis with brew services restart redis.
How do you find this out!?
I wasted a lot of time trying to figure out if redis wasn't using the defaults through homebrew and what port it was on. Services was misleading because even though redis-server had not actually started, brew services list would still show redis as "started." The best approach is to use brew services --verbose start redis which will show you that the log file is at /usr/local/var/log/redis.log. Looking in there I found the smoking gun(s)
Fatal error, can't open config file '/usr/local/etc/redis.conf'
or
Can't chdir to '/usr/local/var/db/redis/': No such file or directory
Thankfully the log made the solution above obvious.
Can't I just run redis-server?
You sure can. It'll just take up a terminal or interrupt your terminal occasionally if you run redis-server &. Also it will put dump.rdb in whatever directory you run it in (pwd). I got annoyed having to remove the file or ignore it in git so I figured I'd let brew do the work with services.
If after install you need to run redis on all time, just type in terminal:
redis-server &
Running redis using upstart on Ubuntu
I've been trying to understand how to setup systems from the ground up on Ubuntu. I just installed redis onto the box and here's how I did it and some things to look out for.
To install:
sudo apt-get install redis-server
That will create a redis user and install the init.d script for it. Since upstart is now the replacement for using init.d, I figure I should convert it to run using upstart.
To disable the default init.d script for redis:
sudo update-rc.d redis-server disable
Then create /etc/init/redis-server.conf with the following script:
description "redis server"
start on runlevel [23]
stop on shutdown
exec sudo -u redis /usr/bin/redis-server /etc/redis/redis.conf
respawn
What this is the script for upstart to know what command to run to start the process. The last line also tells upstart to keep trying to respawn if it dies.
One thing I had to change in /etc/redis/redis.conf is daemonize yes to daemonize no. What happens if you don't change it then redis-server will fork and daemonize itself, and the parent process goes away. When this happens, upstart thinks that the process has died/stopped and you won't have control over the process from within upstart.
Now you can use the following commands to control your redis-server:
sudo start redis-server
sudo restart redis-server
sudo stop redis-server
Hope this was helpful!
redis-server --daemonize yes
I have solved this issue by running this command.
This work for me :
sudo service redis-server start
Date: Dec 2021
There is a couple of reason for this error. I read one article to fix the issue for me. So I just summarize what to check one by one.
1 Check: Redis-Server not Started
redis-server
Also to run Redis in the background, the following command could be used.
redis-server --daemonize yes
2. Check: Firewall Restriction
sudo ufw status (inactive)
sudo ufw active (for making active it might disable ssh when first time active. So enable port 22 to access ssh.)
sudo ufw allow 22
sudo ufw allow 6379
3. Check: Resource usage
ps -aux | grep redis
4. Config setup restriction
sudo vi /etc/redis/redis.conf.
Comment the following line.
# bind 127.0.0.1 ::1
Note: It will be more difficult for malicious actors to make requests or gain access to your server. Make sure you're bound to correct IP address network.
Hope it helps someone. For more information read the following article.
https://bobcares.com/blog/could-not-connect-to-redis-connection-refused/
It's the better way to connect to your redis.
At first, check the ip address of redis server like this.
ps -ef | grep redis
The result is kind of " redis 1184 1 0 .... /usr/bin/redis-server 172.x.x.x:6379
And then you can connect to redis with -h(hostname) option like this.
redis-cli -h 172.x.x.x
Try this :
sudo service redis-server restart
Error connecting Redis on Apple Silicon( Macbook Pro M1 - Dec 2020), you have to just know 2 things:
Run the redis-server using a sudo will remove the server starting error
shell% sudo redis-server
For running it as a service "daemonize" it will allow you to run in the background
shell% sudo redis-server --daemonize yes
Verify using below steps:
shell% redis-cli ping
Hope this helps all Macbook Pro M1 users who are really worried about lack of documentation on this.
I was stuck on this for a long time. After a lot of tries I was able to configure it properly.
There can be different reasons of raising the error. I am trying to provide the reason and the solution to overcome from that situation. Make sure you have installed redis-server properly.
6379 Port is not allowed by ufw firewall.
Solution: type following command sudo ufw allow 6379
The issue can be related to permission of redis user. May be redis user doesn't have permission of modifying necessary redis directories. The redis user should have permissions in the following directories:
/var/lib/redis
/var/log/redis
/run/redis
/etc/redis
To give the owner permission to redis user, type the following commands:
sudo chown -R redis:redis /var/lib/redis
sudo chown -R redis:redis /var/log/redis
sudo chown -R redis:redis /run/redis
sudo chown -R redis:redis /etc/redis.
Now restart redis-server by following command:
sudo systemctl restart redis-server
Hope this will be helpful for somebody.
First you need to up/start the all the redis nodes using below command, one by one for all conf files.
#Note : if you are setting up cluster then you should have 6 nodes, 3 will be master and 3 will be slave.redis-cli will automatically select master and slave out of 6 nodes using --cluster command as shown in my below commands.
[xxxxx#localhost redis-stable]$ redis-server xxxx.conf
then run
[xxxxx#localhost redis-stable]$ redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 127.0.0.1:7003 127.0.0.1:7004 127.0.0.1:7005 --cluster-replicas 1
output of above should be like:
>>> Performing hash slots allocation on 6 nodes...
2nd way to set up all things automatically:
you can use utils/create-cluster scripts to set up every thing for you like
starting all nodes, creating cluster
you an follow https://redis.io/topics/cluster-tutorial
Thanks
Actually you need to run "redis-server &" after instalation to start the service, when you only run "redis-server" the service runs in undetached mode. emphasis on "&"
I just had this same problem because I had used improper syntax in my config file. I meant to add:
maxmemory-policy allkeys-lru
to my config file, but instead only added:
allkeys-lru
which evidently prevented Redis from parsing the config file, which in turn prevented me from connecting through the cli. Fixing this syntax allowed me to connect to Redis.
Had that issue with homebrew MacOS the problem was some sort of permission missing on /usr/local/var/log directory see issue here
In order to solve it I deleted the /usr/local/var/log and reinstall redis brew reinstall redis
In my case, it was the password that contained some characters like ', after changing it the server started without problems.
Just like Aaron, in my case brew services list claimed redis was running, but it wasn't. I found the following information in my log file at /usr/local/var/log/redis.log:
4469:C 28 Feb 09:03:56.197 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
4469:C 28 Feb 09:03:56.197 # Redis version=4.0.9, bits=64, commit=00000000, modified=0, pid=4469, just started
4469:C 28 Feb 09:03:56.197 # Configuration loaded
4469:M 28 Feb 09:03:56.198 * Increased maximum number of open files to 10032 (it was originally set to 256).
4469:M 28 Feb 09:03:56.199 # Creating Server TCP listening socket 192.168.161.1:6379: bind: Can't assign requested address
That turns out to be caused by the following configuration:
bind 127.0.0.1 ::1 192.168.161.1
which was necessary to give my VMWare Fusion virtual machine access to the redis server on macOS, the host. However, if the virtual machine wasn't started, this binding failure caused redis not to start up at all. So starting the virtual machine solved the problem.
I was trying to connect my Redis running in wsl2 from vs code running in Windows.
I have listed down what worked for me and the order in which I have performed these actions:
1) sudo ufw allow 6379
2) Update redis.conf to bind 127.0.0.1 ::1 192.168.1.7
3) sudo service redis-server restart
NOTE: This is the first time I have installed Redis on wsl2 and have not run a single command yet.
Let me know if it works for you.
Thanks.
Redis for Mac:
1- brew install redis
2- brew services start redis
3- redis-cli ping
$ brew services start redis
$ brew services stop redis
$ brew services restart redis
Lunch autostart options:
$ ln -sfv /usr/local/opt/redis/*.plist ~/Library/LaunchAgents
# autostart activate
$ launchctl load ~/Library/LaunchAgents/homebrew.mxcl.redis.plist
# autostart deactivate
$ launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.redis.plist
Redis conf default path : /usr/local/etc/redis.conf
In my case, someone had come along and incorrectly edited the redis.conf file to this:
bind 127.0.0.1 ::1
bind 192.168.1.7
when, it really needed to be this (one line):
bind 127.0.0.1 ::1 192.168.1.7
I am using Ubuntu 18.04
I have just enter this command in CMD
sudo systemctl start redis-server
And it is now working. so I thing my redis server was not started that why it showing me the error
Could not connect to Redis at 127.0.0.1:6379: Connection refused.

After setup password-less login, I still got permission denied error while running a script to login

I have already setup the password-less login to localhost via ssh, and it works and doesn't need me to enter any password.
justinwu#ubuntu:~$ ssh root#localhost
Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 3.13.0-110-generic x86_64)
* Documentation: https://help.ubuntu.com/
2 packages can be updated.
0 updates are security updates.
New release '16.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.
Last login: Tue Mar 7 10:08:22 2017 from localhost
But while running one script and it needs to login as root via ssh, it returns the error 'Permission denied (publickey)'.
...
Launching 20 processes.
Starting test client: 0 Host: localhost Workload: VDI Workdir: /mnt/share
Permission denied (publickey).
I don't know why the script cannot automatically login via root without entering the password.
Please help! Thanks!

ssh timing out on second login

I'm trying to set up Rails on my site via ssh. When everything is set up, I start the server with rails server and I get:
=> Rails 5.0.1 application starting in development on http://localhost:3000
=> Run rails server -h for more startup options
Puma starting in single mode...
* Version 3.6.2 (ruby 2.3.3-p222), codename: Sleepy Sunday Serenity
* Min threads: 5, max threads: 5
* Environment: development
* Listening on tcp://localhost:3000
Use Ctrl-C to stop
It would be fine if the issue didn't stop there, but when I point my browser at the IP address on port 3000, my browser just hangs instead of displaying the Rails smoke page.
Since I can't type in more commands, I open a new terminal and log in via:
ssh -i /path/to/cloud.key user_name#XXX.XXX.XXX.XXX
I think I've seen it work before, but now it's timing out:
ssh: connect to host xxx.xxx.xxx.xxx port 22: Operation timed out
I found similar problems resolved on stackoverflow, but none of them solved the problem for public key authentication and when I try their solutions (ssh user_name#XXX.XXX.XXX.XXX), I turn up Permission denied (publickey).
So, I want to either learn why my browser is hanging (if I need to install nginx or apache2 or configure puma, etc.), and/or why my attempts to log into a second ssh session are failing.
Any help for this one?
Ubuntu Server 16.04
Rails 5.0.1
Ruby 2.3.0p0
You can't point your browser to the ip because the server is binding to localhost. You must enter rails s -b0.0.0.0

Cannot connect to Google Compute Engine instance via SSH in browser

I cannot connect to GCE via ssh. It is showing Connection Failed, and we are unable to connect VM on port 22.
And serial console output its shows
Jul 8 10:09:26 Instance sshd[10103]: error: Could not load host key: /etc/ssh/ssh_host_ed25519_key
Jul 8 10:09:27 Instance sshd[10103]: User username from 0.0.0.0 not allowed because not listed in AllowUsers
Jul 8 10:09:27 Instance sshd[10103]: input_userauth_request: invalid user username [preauth] Jul 8 10:09:27 Instance sshd[10103]: Connection closed by 0.0.0.0 [preauth] –
Yesterday it was working fine, but today it shows this error. I am new to GCE. Any suggestions?
UPDATE
I'd like to post this update to mention that on June 2016 a new feature is released where you can enable interactive access to the serial console so you can more easily troubleshoot instances that are not booting properly or that are otherwise inaccessible. See Interacting with the Serial Console for more information.
-----------------------------------------------------------------------------------
It looks like you've added AllowUsers in /etc/ssh/sshd_config configuration file.
To resolve this issue, you'll need to attach the boot disk of your VM instance to a healthy instance as the second disk. Mount it, edit the configuration file and fix the issue.
Here are the steps you can take to resolve the issue:
First of all, take a snapshot of your instance’s disk, in case if a loss or corruption happens you can recover your disk.
In the Developers Console, click on your instance. Uncheck Delete boot disk when instance is deleted and then delete the instance. The boot disk will remain under “Disks”, and now you can attach the disk to another instance. You can also do this step using gcloud command:
$ gcloud compute instances delete NAME --keep-disks all
Now attach the disk to a healthy instance as an additional disk. You can do this through the Developers Console or using the gcloud command:
$ gcloud compute instances attach-disk EXAMPLE-INSTANCE --disk DISK --zone ZONE
SSH into your healthy instance.
Determine where the secondary disk lives:
$ ls -l /dev/disk/by-id/google-*
Mount the disk:
$ sudo mkdir /mnt/tmp
$ sudo mount /dev/disk/by-id/google-persistent-disk-1-part1 /mnt/tmp
Where google-persistent-disk-1 is the name of the disk
Edit sshd_config configuration file and remove AllowUsers line and save it.
$ sudo nano /mnt/tmp/etc/ssh/sshd_config
Now unmout the disk:
$ sudo umount /mnt/tmp
Detach it from the VM instance. This can be done through the Developers Console or using the command below:
$ gcloud compute instances detach-disk EXAMPLE-INSTANCE --disk DISK
Now create a new instance using your fixed boot disk.