I am trying to install an old version of RabbitMQ using Chef (cookbook 'rabbitmq', '~> 5.8.5') and Kitchen, below my configuration:
Attributes
#Erlang
default['erlang']['install_method'] = 'source'
default['erlang']['source']['version']='R13B03'
default['erlang']['source']['checksum']='e7c46c8b2778f22064a3b369c1a1b572a1cc0e8a2198166858d4b9a1b488d662'
#RabbitMQ
default['rabbitmq']['erlang']['enabled'] = true
default['rabbitmq']['version'] = "3.4.4"
default['rabbitmq']['rpm_package'] ='rabbitmq-server-3.4.4-1.noarch.rpm'
Recipe:
include_recipe 'rabbitmq::default'
When I run kitchen converge, I am getting the following exception:
Running handlers:
[2020-08-22T22:20:07+00:00] ERROR: Running exception handlers
Running handlers complete
[2020-08-22T22:20:07+00:00] ERROR: Exception handlers complete
Chef Infra Client failed. 9 resources updated in 06 minutes 26 seconds
[2020-08-22T22:20:07+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
[2020-08-22T22:20:07+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2020-08-22T22:20:07+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: rpm_package[/tmp/kitchen/cache/rabbitmq-server-3.4.4-1.noarch.rpm] (rabbitmq::default line 224) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of ["rpm", "-i", "/tmp/kitchen/cache/rabbitmq-server-3.4.4-1.noarch.rpm"] ----
STDOUT:
STDERR: warning: /tmp/kitchen/cache/rabbitmq-server-3.4.4-1.noarch.rpm: Header V4 DSA/SHA1 Signature, key ID 056e8e56: NOKEY
error: Failed dependencies:
erlang >= R13B-03 is needed by rabbitmq-server-3.4.4-1.noarch
---- End output of ["rpm", "-i", "/tmp/kitchen/cache/rabbitmq-server-3.4.4-1.noarch.rpm"] ----
Ran ["rpm", "-i", "/tmp/kitchen/cache/rabbitmq-server-3.4.4-1.noarch.rpm"] returned 1
But when I logged in to the VM, I can see erlang is installed:
[vagrant#kitchen-rmq-server-centos-7 ~]$ erl
Erlang R13B03 (erts-5.7.4) [source] [64-bit] [rq:1] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.7.4 (abort with ^G)
1>
And it is the same version required by RMQ (R13B03)
Any idea how to solve this issue?
Edit: to replicate the issue https://github.com/Proximator/chef-rmq
Firstly, we have to make sure erlang is installed by the rabbitmq cookbook, and not by any other means. This is the note found on Chef supermarket for rabbitmq cookbook:
The packages are cannot be installed alongside with other Erlang packages, for example, those from standard Debian repositories or Erlang Solutions.
To make sure that the Erlang cookbook is not used by rabbitmq::default
Also, there is a compatibility matrix of RabbitMQ and Erlang versions. RabbitMQ 3.7.0 being the lowest supported version, for which the lowest compatible Erlang version is 19.3.
There are zero dependency Erlang RPMs "just enough to run RabbitMQ" as documented here:
https://github.com/rabbitmq/erlang-rpm
For example - to install RabbitMQ 3.7.x with the compatible Erlang 19.3.x:
You should have these attributes:
default['rabbitmq']['erlang']['enabled'] = true
default['rabbitmq']['version'] = '3.7.6'
default['rabbitmq']['erlang']['yum']['baseurl'] = 'https://dl.bintray.com/rabbitmq-erlang/rpm/erlang/19/el/7'
default['rabbitmq']['erlang']['version'] = '19.3.6.13'
Then include below recipes:
include_recipe 'rabbitmq::erlang_package'
include_recipe 'rabbitmq::default'
I have a Flask server that I'm running on AWS Fargate. My task has 2 vCPUs and 8 GB of memory. My server is only able to respond to one request at a time. If I run 2 API requests at the same, each that takes 7 seconds, the first request will take 7 seconds to return and the second will take 14 seconds to return.
This is my Docker file (using this repo):
FROM tiangolo/uwsgi-nginx-flask:python3.7
COPY ./requirements.txt requirements.txt
RUN pip3 install --no-cache-dir -r requirements.txt
RUN python3 -m spacy download en
RUN apt-get update
RUN apt-get install wkhtmltopdf -y
RUN apt-get install poppler-utils -y
RUN apt-get install xvfb -y
COPY ./ /app
I have the following config file:
[uwsgi]
module = main
callable = app
enable-threads = true
These are my logs when I start the server:
Checking for script in /app/prestart.sh
Running script /app/prestart.sh
Running inside /app/prestart.sh, you could add migrations to this file, e.g.:
#! /usr/bin/env bash
# Let the DB start
sleep 10;
# Run migrations
alembic upgrade head
/usr/lib/python2.7/dist-packages/supervisor/options.py:298: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
'Supervisord is running as root and it is searching '
2019-10-05 06:29:53,438 CRIT Supervisor running as root (no user in config file)
2019-10-05 06:29:53,438 INFO Included extra file "/etc/supervisor/conf.d/supervisord.conf" during parsing
2019-10-05 06:29:53,446 INFO RPC interface 'supervisor' initialized
2019-10-05 06:29:53,446 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2019-10-05 06:29:53,446 INFO supervisord started with pid 1
2019-10-05 06:29:54,448 INFO spawned: 'nginx' with pid 9
2019-10-05 06:29:54,450 INFO spawned: 'uwsgi' with pid 10
[uWSGI] getting INI configuration from /app/uwsgi.ini
[uWSGI] getting INI configuration from /etc/uwsgi/uwsgi.ini
;uWSGI instance configuration
[uwsgi]
cheaper = 2
processes = 16
ini = /app/uwsgi.ini
module = main
callable = app
enable-threads = true
ini = /etc/uwsgi/uwsgi.ini
socket = /tmp/uwsgi.sock
chown-socket = nginx:nginx
chmod-socket = 664
hook-master-start = unix_signal:15 gracefully_kill_them_all
need-app = true
die-on-term = true
show-config = true
;end of configuration
*** Starting uWSGI 2.0.18 (64bit) on [Sat Oct 5 06:29:54 2019] ***
compiled with version: 6.3.0 20170516 on 09 August 2019 03:11:53
os: Linux-4.14.138-114.102.amzn2.x86_64 #1 SMP Thu Aug 15 15:29:58 UTC 2019
nodename: ip-10-0-1-217.ec2.internal
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /app
detected binary path: /usr/local/bin/uwsgi
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
Python version: 3.7.4 (default, Jul 13 2019, 14:20:24) [GCC 6.3.0 20170516]
Python main interpreter initialized at 0x55e1e2b181a0
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 1239640 bytes (1210 KB) for 16 cores
*** Operational MODE: preforking ***
2019-10-05 06:29:55,483 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-10-05 06:29:55,484 INFO success: uwsgi entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
I cannot seem to start or install my RabbitMQ server anymore for my Ubuntu 18.04 anymore. I tried to remove and install it again, but it cannot finish the install because configuration fails. When I try to run sudo apt-get install --fix-broken. This is the result of it failing:
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 61 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Setting up rabbitmq-server (3.6.10-1) ...
Job for rabbitmq-server.service failed because the control process exited with error code.
See "systemctl status rabbitmq-server.service" and "journalctl -xe" for details.
invoke-rc.d: initscript rabbitmq-server, action "start" failed.
● rabbitmq-server.service - RabbitMQ Messaging Server
Loaded: loaded (/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2018-08-22 09:16:51 EEST; 5ms ago
Process: 20997 ExecStartPost=/usr/lib/rabbitmq/bin/rabbitmq-server-wait (code=exited, status=70)
Process: 20996 ExecStart=/usr/sbin/rabbitmq-server (code=exited, status=0/SUCCESS)
Main PID: 20996 (code=exited, status=0/SUCCESS)
elo 22 09:16:48 ubuntu-dev systemd[1]: Starting RabbitMQ Messaging Server...
elo 22 09:16:49 ubuntu-dev rabbitmq[20997]: Waiting for 'rabbit#ubuntu-dev'
elo 22 09:16:49 ubuntu-dev rabbitmq[20997]: pid is 21001
elo 22 09:16:51 ubuntu-dev rabbitmq[20997]: Error: process_not_running
elo 22 09:16:51 ubuntu-dev systemd[1]: rabbitmq-server.service: Control process exited, code=exited status=70
elo 22 09:16:51 ubuntu-dev systemd[1]: rabbitmq-server.service: Failed with result 'exit-code'.
elo 22 09:16:51 ubuntu-dev systemd[1]: Failed to start RabbitMQ Messaging Server.
dpkg: error processing package rabbitmq-server (--configure):
installed rabbitmq-server package post-installation script subprocess returned error exit status 1
Errors were encountered while processing:
rabbitmq-server
E: Sub-process /usr/bin/dpkg returned an error code (1)
Then when checking the log files they doesn't provide much more information either. Here is startup_err log file content:
init terminating in do_boot (noproc)
Crash dump is being written to: erl_crash.dump...done'
And here is startup_log file content:
BOOT FAILED
===========
Error description:
noproc
Log files (may contain more information):
/var/log/rabbitmq/rabbit.log
/var/log/rabbitmq/rabbit-sasl.log
Stack trace:
[{gen,do_for_proc,2,[{file,"gen.erl"},{line,228}]},
{gen_event,rpc,2,[{file,"gen_event.erl"},{line,239}]},
{rabbit,ensure_working_log_handlers,0,
[{file,"src/rabbit.erl"},{line,842}]},
{rabbit,'-boot/0-fun-0-',0,[{file,"src/rabbit.erl"},{line,281}]},
{rabbit,start_it,1,[{file,"src/rabbit.erl"},{line,417}]},
{init,start_em,1,[]},
{init,do_boot,3,[]}]
=INFO REPORT==== 22-Aug-2018::09:16:49.691453 ===
Error description:
noproc
Log files (may contain more information):
/var/log/rabbitmq/rabbit.log
/var/log/rabbitmq/rabbit-sasl.log
Stack trace:
[{gen,do_for_proc,2,[{file,"gen.erl"},{line,228}]},
{gen_event,rpc,2,[{file,"gen_event.erl"},{line,239}]},
{rabbit,ensure_working_log_handlers,0,
[{file,"src/rabbit.erl"},{line,842}]},
{rabbit,'-boot/0-fun-0-',0,[{file,"src/rabbit.erl"},{line,281}]},
{rabbit,start_it,1,[{file,"src/rabbit.erl"},{line,417}]},
{init,start_em,1,[]},
{init,do_boot,3,[]}]
{"init terminating in do_boot",noproc}
The other log files it claim to use for logging are empty. For example log file rabbit#ubuntu-dev.log and rabbit#ubuntu-dev-sasl.log.
I also found this post, which explains to check your hostname in /etc/hostname file but I checked and it's correct.
kazhu#ubuntu-dev:/var/log/rabbitmq$ cat /etc/hostname
ubuntu-dev
I also checked RabbitMQ troubleshoot guide and they said to check log folder permissions and they are right to my eye:
kazhu#ubuntu-dev:/var/log/rabbitmq$ ll
total 48
drwxr-xr-x 2 rabbitmq rabbitmq 4096 kesä 14 06:16 ./
drwxrwxr-x 16 root syslog 4096 elo 22 00:09 ../
-rw-r--r-- 1 rabbitmq rabbitmq 0 kesä 14 06:16 'rabbit#ubuntu-dev.log'
-rw-r--r-- 1 rabbitmq rabbitmq 5247 kesä 14 06:16 'rabbit#ubuntu-dev.log.1'
-rw-r--r-- 1 rabbitmq rabbitmq 954 touko 28 08:36 'rabbit#ubuntu-dev.log.2.gz'
-rw-r--r-- 1 rabbitmq rabbitmq 768 touko 21 07:11 'rabbit#ubuntu-dev.log.3.gz'
-rw-r--r-- 1 rabbitmq rabbitmq 708 touko 16 00:12 'rabbit#ubuntu-dev.log.4.gz'
-rw-r--r-- 1 rabbitmq rabbitmq 955 touko 7 07:26 'rabbit#ubuntu-dev.log.5.gz'
-rw-r--r-- 1 rabbitmq rabbitmq 4264 huhti 22 00:07 'rabbit#ubuntu-dev.log.6.gz'
-rw-r--r-- 1 rabbitmq rabbitmq 0 huhti 17 15:58 'rabbit#ubuntu-dev-sasl.log'
-rw-r--r-- 1 rabbitmq rabbitmq 95 elo 22 09:16 startup_err
-rw-r--r-- 1 rabbitmq rabbitmq 1212 elo 22 09:16 startup_log
Guide also stated that perl chrash dump file contains detailed information of the problem and requires Erlang expertises, which I don't have. So decided to upload the file to my Dropbox for you to see.
Can somebody help me solve this? I've tried some time myself but gave up because cannot figure out what the problem seems to be :/
I solved the problem with help of my colleague. I had installed newest erlang and rabbitmq from outside apt source separately. Now when I removed and purged everything related to rabbitmq and erlang, and removed the added apt sources too. Then I just ran sudo apt install rabbitmq-server and it wanted to install erlang packages too because of the dependency. And it installed and everything is working fine after that.
Wanted to share this solution if somebody else has the same problem as me.
UPDATE 9.12.2020:
Someone asked how I removed RabbitMQ and Erlang. I don't fully remember but I think I was following this guide: https://www.rabbitmq.com/install-debian.html.
The point is to remove the installed RabbitMQ and Erlang packages from added repositories and their configuration with
sudo apt purge rabbitmq-server erlang
You might need to search for rest of the erlang packages with
apt list | grep erlang
Then you need to remove added apt repositories. Usually added repositories in Ubuntu goes under /etc/apt/sources.list.d/ folder. Look for files names like rabbitmq and erlang. Make sure you are not deleting any other files!
After this run sudo apt update and apt should remove removed apt repositories. Then just running sudo apt install rabbitmq-server should do the trick and install Erlang package as a dependence. Of course installing this way you get much older version than using added repositories.
I meet a problem when booting rabbitmq which is driving me crazy...
Env:redhat4
erlang version:R16B03 installed from source code
rabbitmq version:rabbitmq-server-3.6.1 from source code
when I ran 'rabbitmq-server start' command, it gave me some error
information, then I ran that again it showed:
RabbitMQ 3.6.1. Copyright (C) 2007-2016 Pivotal Software, Inc.
## ## Licensed under the MPL. See http://www.rabbitmq.com/
## ##
########## Logs: /var/log/rabbitmq/rabbit#bogon.log
###### ## /var/log/rabbitmq/rabbit#bogon-sasl.log
##########
Starting broker...
And it stucked there, I thought may be it had started, so I ran './rabbitmq-plugins enable rabbitmq_management' to enable web plugins, but I got:
The following plugins have been enabled:
mochiweb
webmachine
rabbitmq_web_dispatch
amqp_client
rabbitmq_management_agent
rabbitmq_management
Applying plugin configuration to rabbit#bogon... failed.
Error: {undef,[{crypto,module_info,[attributes],[]},
{rabbit_misc,module_attributes,1,
[{file,"src/rabbit_misc.erl"},{line,805}]},
{rabbit_misc,'-all_module_attributes/1-fun-0-',3,
[{file,"src/rabbit_misc.erl"},{line,825}]},
{lists,foldl,3,[{file,"lists.erl"},{line,1248}]},
{rabbit_boot_steps,find_steps,1,
[{file,"src/rabbit_boot_steps.erl"},
{line,40}]},
{rabbit_boot_steps,run_boot_steps,1,
[{file,"src/rabbit_boot_steps.erl"},
{line,26}]},
{rabbit,start_apps,1,[{file,"src/rabbit.erl"},{line,343}]},
{rabbit_plugins,ensure,1,
[{file,"src/rabbit_plugins.erl"},{line,52}]}]}
I don't know why... so I stopped the rabbitmq server and started it again, and it gave me this:
RabbitMQ 3.6.1. Copyright (C) 2007-2016 Pivotal Software, Inc.
## ## Licensed under the MPL. See http://www.rabbitmq.com/
## ##
########## Logs: /var/log/rabbitmq/rabbit#bogon.log
###### ## /var/log/rabbitmq/rabbit#bogon-sasl.log
##########
Starting broker...
BOOT FAILED
===========
Error description:
{could_not_start,rabbit,
{undef,
[{crypto,module_info,[attributes],[]},
{rabbit_misc,module_attributes,1,
[{file,"src/rabbit_misc.erl"},{line,805}]},
{rabbit_misc,'-all_module_attributes/1-fun-0-',3,
[{file,"src/rabbit_misc.erl"},{line,825}]},
{lists,foldl,3,[{file,"lists.erl"},{line,1248}]},
{rabbit_boot_steps,find_steps,1,
[{file,"src/rabbit_boot_steps.erl"},{line,40}]},
{rabbit_boot_steps,run_boot_steps,1,
[{file,"src/rabbit_boot_steps.erl"},{line,26}]},
{rabbit,start,2,[{file,"src/rabbit.erl"},{line,477}]},
{application_master,start_it_old,4,
[{file,"application_master.erl"},{line,269}]}]}}
Log files (may contain more information):
/var/log/rabbitmq/rabbit#bogon.log
/var/log/rabbitmq/rabbit#bogon-sasl.log
{"init terminating in do_boot",{could_not_start,rabbit,{undef,[{crypto,module_info,[attributes],[]},{rabbit_misc,module_attributes,1,[{file,"src/rabbit_misc.erl"},{line,805}]},{rabbit_misc,'-all_module_attributes/1-fun-0-',3,[{file,"src/rabbit_misc.erl"},{line,825}]},{lists,foldl,3,[{file,"lists.erl"},{line,1248}]},{rabbit_boot_steps,find_steps,1,[{file,"src/rabbit_boot_steps.erl"},{line,40}]},{rabbit_boot_steps,run_boot_steps,1,[{file,"src/rabbit_boot_steps.erl"},{line,26}]},{rabbit,start,2,[{file,"src/rabbit.erl"},{line,477}]},{application_master,start_it_old,4,[{file,"application_master.erl"},{line,269}]}]}}}
Crash dump was written to: erl_crash.dump
init terminating in do_boot ()
I don't understand... Why there are a lot of issues? Is there any procedure that was not correct?
{could_not_start,rabbit,{undef,[{crypto,module_info,[attributes]
means that you need the crypto Erlang.
Check this https://www.rabbitmq.com/which-erlang.html.
use SSL/TLS reliably 17.0
you need the version >= 17.0
I got same issue because of mismatch of RabbitMQ version installed on 3rd node other than two nodes. Install same versions on all nodes of cluster.