I'm using Terraform to automate build out of an AWS EC2 based docker host and then using its remote exec option to download a docker file, build and run it.
I'd hoped to integrate this with Serverspec but am struggling to work out two things:
The best way to pass the external dns of the newly created AWS EC2 instance to Serverspec.
How to configure the SSH options for Serverspec so that it executes correctly on an Amazon Linux AMI using the ec2-user account.
I would normally connect to the EC2 instance using a pre-defined key pair and never use a password however ServerSpec seems to run commands on the server with a sudo -p format.
Any advice much appreciated.
Contents of spec_helper.rb
require 'serverspec'
require 'net/ssh'
set :ssh_options, :user => 'ec2-user'
Also using edited rakefile as follows to force correct EC2 external dns (masked):
require 'rake'
require 'rspec/core/rake_task'
hosts = %w(
ec2-nn-nn-nn-nnn.eu-west-1.compute.amazonaws.com
)
set :ssh_options, :user => 'ec2-user'
task :spec => 'spec:all'
namespace :spec do
task :all => hosts.map {|h| 'spec:' + h.split('.')[0] }
hosts.each do |host|
short_name = host.split('.')[0]
role = short_name.match(/[^0-9]+/)[0]
desc "Run serverspec to #{host}"
RSpec::Core::RakeTask.new(short_name) do |t|
ENV['TARGET_HOST'] = host
t.pattern = "spec/Nexus/*_spec.rb"
end
end
end
You could make the IP address an output in Terraform. In fact, the link gives an example doing just that to get the IP address of an AWS instance, named web in this case:
output "address" {
value = "${aws_instance.web.public_dns}"
}
Then you can get this value from the command line after a terraform apply with terraform output address.
You can set the sudo password with the config option :sudo_password. If the ec2-user can run sudo without a password, set this to ''. (See this blog post for an example.) Or pass it in the SUDO_PASSWORD environment variable, described here: http://serverspec.org/tutorial.html
I'm using two Vagrant VMs to test some things with Puppet, but when I go to request a cert, I get a cryptic error message that I can't find any information about.
I should note that in correspondence with good Linux server administration I'm use /var/ and /opt/ for storing sensitive cert info, but otherwise a standard Puppet setup.
# Client node details
IP: 192.168.250.10
Hostname: client.example.com
Puppet version: 4.3.2
OS: CentOS Linux release 7.0.1406 (on Vagrant)
# Puppet server details
IP: 192.168.250.6
Hostname: puppet-server.example.com
Puppet version: 4.3.2
OS: CentOS Linux release 7.0.1406 (on Vagrant)
# client's and server's /etc/hosts files are identical
192.168.250.5 puppetmaster.example.com
192.168.250.6 puppet.example.com puppet-server.example.com
192.168.250.7 dashserver.example.com dashboard.example.com
192.168.250.10 client.example.com
192.168.250.20 webserver.example.com
# /etc/puppetlabs/puppet/puppet.conf on both client and server
[main]
logdest = syslog
[user]
bucketdir = $clientbucketdir
vardir = /var/opt/puppetlabs/server
ssldir = $vardir/ssl
[agent]
server = puppet.example.com
[master]
certname = puppet.example.com
vardir = /var/opt/puppetlabs/puppetserver
ssldir = $vardir/ssl
logdir = /var/log/puppetlabs/puppetserver
rundir = /var/run/puppetlabs/puppetserver
pidfile = /var/run/puppetlabs/puppetserver/puppetserver.pid
trusted_server_facts = true
reports = store
cacert = /var/opt/puppetlabs/puppetserver/ssl/certs/ca.pem
cacrl = /var/opt/puppetlabs/puppetserver/ssl/crl.pem
hostcert = /var/opt/puppetlabs/puppetserver/ssl/certs/{puppet, client}.example.com.pem # respectively, obviously
hostprivkey = /var/opt/puppetlabs/puppetserver/ssl/private_keys/{puppet, client}.example.com.pem # respectively, obviously
Finally, the error I get:
$ sudo puppet resource service puppet ensure=stopped enable=false
Notice: /Service[puppet]/ensure: ensure changed 'running' to 'stopped'
service { 'puppet':
ensure => 'stopped',
enable => 'false',
}
$ sudo puppet resource service puppet ensure=running enable=true
Notice: /Service[puppet]/ensure: ensure changed 'stopped' to 'running'
service { 'puppet':
ensure => 'running',
enable => 'true',
}
$ puppet agent --test --server=puppet.example.com
Error: Could not request certificate: Permission denied # dir_initialize - /etc/puppetlabs/puppet/ssl/private_keys
Exiting; failed to retrieve certificate and waitforcert is disabled
First of all, with this setup Puppet should not be using /etc/puppetlabs/puppet/ssl/private_keys. It's not using my configuration file correctly:
$ puppet config print ssldir
/etc/puppetlabs/puppet/ssl
Next, I went through and regenerated the keys on BOTH the server and the client nodes as prescribed in the Puppet docs, however I still got the same error and both the client AND server still think my $ssldir is /etc/puppetlabs/puppet/ssl when it should be /var/opt/puppetlabs/puppetserver/ssl.
Any thoughts?
You need to specify the ssl and vardir config in the agent section as well as master.
the user section is only applicable to the puppet apply commands etc
I have installed the latest version of RabbitMQ on a VPS Debian Linux box. Tried to get login through guest/guest but returned with the message login failed. I did a little research and found that for security reason its prohibited to get login via guest/guest remotely.
I also have tried enabling guest uses on this version to get logged in remotely by creating a rabbitmq.config file manually (because the installation didn't create one) and placing the following entry only
[{rabbit, [{loopback_users, []}]}].
after restart the rabbitmq with the following command.
invoke-rc.d rabbitmq-server stop -- to stop
invoke-rc.d rabbitmq-server start -- to start
It still doesn't logged me in with guest/guest. I also have tried installing RabbitMQ on Windows VPS and tried to get log in via guest/guest through localhost but again i get the same message login failed.
Also provide me a source where I could try installing the old version of RabbitMQ that does support logging remotely via guest/guest.
I had the same Problem..
I installed RabbitMQ and Enabled Web Interface also but still couldn't sign in with any user i newly created, this is because you need to be administrator to access this.
Do not create any config file and mess with it..
This is what i did then,
Add a new/fresh user, say user test and password test:
rabbitmqctl add_user test test
Give administrative access to the new user:
rabbitmqctl set_user_tags test administrator
Set permission to newly created user:
rabbitmqctl set_permissions -p / test ".*" ".*" ".*"
That's it, enjoy :)
I tried on Debian the same configuration with the following steps:
Installed RabbitMQ.
Enabled the web-management plug-in (not necessary).
When I tried to login I had the same error:
So I created a rabbitmq.config file (classic configuration file) inside the /etc/rabbitmq directory with the following content (notice the final dot):
[{rabbit, [{loopback_users, []}]}].
Alternatively, one can create instead a rabbitmq.conf file (new configuration file) inside the same directory with the following content:
loopback_users = none
Then I executed the invoke-rc.d rabbitmq-server start command and both the console and the Java client were able to connect using the guest/guest credentials:
So I think you have some other problem if this procedure doesn't work. For example your RabbitMQ might be unable to read the configuration file if for some reason you have changed the RABBITMQ_CONFIG_FILE environment variable.
This is a new features since the version 3.3.0. You can only login using guest/guest on localhost. For logging from other machines or on ip you'll have to create users and assign the permissions. This can be done as follows:
rabbitmqctl add_user test test
rabbitmqctl set_user_tags test administrator
rabbitmqctl set_permissions -p / test ".*" ".*" ".*"
Adding the below line in the config file and restarting the server worked for me. Kindly try in your setup.
loopback_users.guest = false
I got this line from the example RabbitMQ config file from Github as linked here.
notice: check your PORT is 15672 ! (version > 3.3 ) if 5672 not works
First of all, check the "choosen answer above":
rabbitmqctl add_user test test
rabbitmqctl set_user_tags test administrator
rabbitmqctl set_permissions -p / test ".*" ".*" ".*"
and if still can't make connection work, check if your port is correct!
for me, this command works:
$ rabbitmqadmin -H 10.140.0.2 -P 15672 -u test -p test list vhosts
+------+----------+
| name | messages |
+------+----------+
| / | |
+------+----------+
for the completed ports , check this:
What ports does RabbitMQ use?
to verify your rabbit mq server, check this: Verify version of rabbitmq
p.s.
For me, after I created the "test" user and run set_user_tags, set_permissions , I can't connect to rabbitmq via port 5672. but I can connect via 15672.
However, port 15672 always gives me a "blank response". and my code stop working.
so about 5 minutes later, I switched to 5672, everything worked!
Very wired problem. I have no time to dig deeper. so I wrote it down here for someone meeting the same problems.
for other guys which use Ansible for RabbitMQ provisioning, what I missed for rabbitmq_user module was tags: administrator
here is my working Ansible configuration to recreate "guest" user (for development environment purpose, don't do that in production environment):
- name: Create RabbitMQ user "guest"
become: yes
rabbitmq_user:
user: guest
password: guest
vhost: /
configure_priv: .*
read_priv: .*
write_priv: .*
tags: administrator
force: yes # recreate existing user
state: present
and I also had to setup a file /etc/rabbitmq/rabbitmq.config containing the following:
[{rabbit, [{loopback_users, []}]}].
in order to be able to log using "guest"/"guest" from outside of localhost
#Create rabbitmq.conf file with
rabbitmq.conf
loopback_users = none
Dockerfile:
FROM rabbitmq:3.7-management
#Rabbitmq config
COPY rabbitmq.conf /etc/rabbitmq/rabbitmq.conf
#Install vim (edit file)
RUN ["apt-get", "update"]
RUN ["apt-get", "-y", "install", "vim"]
#Enable plugins rabbitmq
RUN rabbitmq-plugins enable --offline rabbitmq_mqtt rabbitmq_federation_management rabbitmq_stomp
Run:
$ docker build -t my-rabbitmq-image .
$ docker run -d --hostname my-rabbit --name some-rabbit -p 8080:15672 my-rabbitmq-image
Check that the rabbitmq.conf file has been copied correctly.
$ docker exec -it my_container_id /bin/bash
$ vim /etc/rabbitmq/rabbitmq.conf
I had the same problem. I tried what was suggested by Gas and ran "invoke-rc.d rabbitmq-server start" it didn't start. I tried to reboot the server and the webui worked with the guest user. Maybe after adding the rabbitmq.config file, something else also needed to started.
I used rabbitmq version 3.5.3.
One more thing to note: if you're using AWS instance then you need to open inbound port 15672. (The port for RabbitMQ versions prior to 3.0 is 55672.).
Students and I stared at this problem for an hour. Be sure you've named your files correctly. In the /etc/rabbitmq directory, there are two distinct files. There is an /etc/rabbitmq/rabbitmq.config file which you should edit to get the loopback users as described, but there is another file called rabbitmq-env.conf file. Many folks were using tab completion and just adding "ig", which isn't the right file. Double check!
sometimes you don't need the comma , which is there in the configuration file by default , if nothing else is configured below rabbit tag , while starting broker
we will get a crash
like
{loopback_users, []} , I spend many times hours forgetting this and later removing the comma , it is applicable for all other configurations including SSL
Try restart your rabbitmq and login again, for me work.
For a slightly different use, but might be useful for anyone dealing with accessing the API for monitoring purposes:
I can confirm the answer given by #Oliboy50 works well, however make sure you enable it for each vhost you want the user to be able to monitor, such as:
permissions:
- vhost: "{{item.name}}"
configure_priv: .*
write_priv: .*
read_priv: .*
state: present
tags: management
with_items: "{{user_system_users}}"
With this loop I was able to get past the "401 Unauthorized" error when using the API for any vhost.
By default, the guest user is prohibited from connecting from remote hosts; it can only connect over a loopback interface (i.e. localhost). This applies to connections regardless of the protocol. Any other users will not (by default) be restricted in this way.
It is possible to allow the guest user to connect from a remote host
by setting the loopback_users configuration to none
# DANGER ZONE!
#
# allowing remote connections for default user is highly discouraged
# as it dramatically decreases the security of the system. Delete the user
# instead and create a new one with generated secure credentials.
loopback_users = none
Or, in the classic config file format (rabbitmq.config):
%% DANGER ZONE!
%%
%% Allowing remote connections for default user is highly discouraged
%% as it dramatically decreases the security of the system. Delete the user
%% instead and create a new one with generated secure credentials.
[{rabbit, [{loopback_users, []}]}].
See at "guest" user can only connect from localhost
TIP: It is advisable to delete the guest user or at least change its password to reasonably secure generated value that won't be known to the public.
If you will check the log file under info report you will get this.
`config file(s) : /etc/rabbitmq/rabbitmq.config (not found)`.
Change the config file permission using below command then login using guest , it will work
sudo chmod 777 /etc/rabbitmq/rabbitmq.config
How to remove redis specific application cache via capistrano prior to a restart. Redis is running a remote machine and the redis client need not be installed on the machine which performs the deployment.
As long as capistrano can run any command upon deployment, just remove the cache key(s) with redis-cli:
role :redisserver, "127.0.0.1"
...
namespace :deploy do
...
before "deploy:restart", "deploy:reset_redis_cache"
task :reset_redis_cache, :roles => :rediserver do
run "redis-cli DEL cachekey"
end
...
UPD. added role reference
Need some help connecting Resque Web UI (Rack config.ru) to a Redis server with AUTH
Using Resque + Unicorn + Nginx and installed most using apt-get install (Debian) and gem install
So basically Unicorn loads up resque-web (via Rack) using the standard config.ru
http://etagwerker.wordpress.com/2011/06/27/how-to-setup-resque-web-with-nginx-and-unicorn/
#!/usr/bin/env ruby
# Put this in /var/www/resque-web/config.ru
require 'logger'
$LOAD_PATH.unshift ::File.expand_path(::File.dirname(__FILE__) + '/lib')
require 'resque/server'
Resque::Server.use Rack::Auth::Basic do |username, password|
password == '{{password}}' # password
end
# Set the RESQUE_CONFIG env variable if you’ve a `resque.rb` or similar
# config file you want loaded on boot.
if ENV['RESQUECONFIG'] && ::File.exists?(::File.expand_path(ENV['RESQUE_CONFIG']))
load ::File.expand_path(ENV['RESQUE_CONFIG'])
end
use Rack::ShowExceptions
run Resque::Server.new
I'm trying to find out how to connect this to a Redis server with AUTH per the documentation here: http://redis.io/topics/security (basically in /etc/redis/redis.conf)
This rack configuration seem to only connection to a "vanilla" Redis server using defaults (localhost with standard 6379 port) -- how do I specify the Redis connection so I can pass the user/pass in the format below
redis://user:PASSWORD#redis-server:6379
I've tried using ENV['RESQUE_CONFIG'] to load up a resque.rb file
require 'resque'
Resque.redis = Redis.new(:password => '{{password}}')
this gets pulled via /etc/unicorn/resque-web.conf
# Put this in /etc/unicorn/resque-web.conf
RAILS_ROOT=/var/www/resque-web
RAILS_ENV=production
RESQUE_CONFIG=/var/www/resque-web/config/resque.rb
but it's still not really working
BTW, everything works without the Redis AUTH and just using the "vanilla" localhost Redis connection
Try this
redis_client = Redis.new(:url => "redis://user:PASSWORD#redis-server:6379")
and then do this
Resque.redis = redis_client
Hope this help