I changed disk limit of rabbitmq and now can't restart it. - rabbitmq
I am very new to rabbitMQ. I get getting disk limit full error so I thought of changing the disk limit. I executed following command in sequence:
#1. rabbitmqctl set_disk_free_limit 1GB
#2. sudo systemctl stop rabbitmq-server.service
#3. sudo systemctl disable rabbitmq-server.service
#4. sudo systemctl start rabbitmq-server.service --> failing
I saw rabbitmq process was still running (by using top command). So I executed following:
#5. sudo service rabbitmq-server stop
#6 sudo service rabbitmq-server start --> failing
I am getting following error:
> Jul 11 15:13:37 sk-backend-vm rabbitmq-server[31810]:
> Starting broker... Jul 11 15:13:40 sk-backend-vm
> rabbitmq-server[31810]: erl_child_setup closed Jul 11 15:13:40
> sk-backend-vm rabbitmq-server[31810]: [1B blob data] Jul 11 15:13:40
> sk-backend-vm rabbitmq-server[31810]: Crash dump is being written to:
> /var/log/rabbitmq/erl_crash.dump...done Jul 11 15:13:40 sk-backend-vm
> systemd[1]: rabbitmq-server.service: Main process exited, code=exited,
> status=1/FAILURE Jul 11 15:13:40 sk-backend-vm systemd[1]: Stopped
> RabbitMQ broker. Jul 11 15:13:40 sk-backend-vm systemd[1]:
> rabbitmq-server.service: Unit entered failed
Can you please help me here. I am so stuck.
Related
Centos 7 - httpd service fails once every 5 days with status code 254/n/a
That's weird because it happens only once every 5 nights. I have a cron job which restarts httpd service every night, but I'm receiving this error once every 5 nights. Apr 24 04:33:03 system systemd: httpd.service: main process exited, code=exited, status=254/n/a Apr 24 04:33:03 system systemd: Unit httpd.service entered failed state. Apr 24 04:33:03 system systemd: httpd.service failed. And this is my script to restart httpd ... /usr/sbin/httpd -k restart ... I've checked permissions of /tmp folder and it all looks good. File: ‘/tmp’ Size: 12288 Blocks: 24 IO Block: 4096 directory Device: 803h/2051d Inode: 2359297 Links: 24 Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2022-04-25 10:05:41.726022978 +0300 Modify: 2022-04-25 10:07:02.568018159 +0300 Change: 2022-04-25 10:07:02.568018159 +0300 Birth: - And this is what i found in error_log of httpd. Fatal Error Unable to create lock file: Bad file descriptor (9)
Cloudstack KVM installation failed
I'm installing cloudstack on ubuntu 20.04 by following this document. I installed qemu-kvm and cloudstack-agent successfully but I'm not able to start libvirtd.service, on seeing the status I'm getting following errors ● libvirtd.service - Virtualization daemon Loaded: loaded (/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Tue 2021-03-16 18:00:09 IST; 1min 28s ago TriggeredBy: ● libvirtd-admin.socket ● libvirtd.socket ● libvirtd-ro.socket Docs: man:libvirtd(8) https://libvirt.org Process: 232313 ExecStart=/usr/sbin/libvirtd $libvirtd_opts (code=exited, status=6) Main PID: 232313 (code=exited, status=6) Mar 16 18:00:09 host systemd[1]: libvirtd.service: Scheduled restart job, restart counter is at 5. Mar 16 18:00:09 host systemd[1]: Stopped Virtualization daemon. Mar 16 18:00:09 host systemd[1]: libvirtd.service: Start request repeated too quickly. Mar 16 18:00:09 host systemd[1]: libvirtd.service: Failed with result 'exit-code'. Mar 16 18:00:09 host systemd[1]: Failed to start Virtualization daemon. on seeing the log of journalctl -xe it is showing cloudstack-usage.service: Failed with result 'exit-code' can any one suggest what whould be the issue.
Are you trying this on a virtualised VM, or baremetal host, or on a raspberrypi? This means some other service hasn't started which libvirtd may depend on. See if you can run "systemctl daemon-reload" and try to start libvirtd manually "systemctl start libvirtd", and then try rest. The cloudstack-usage service can be started once the mysql server is running. If you've further questions I encourage you to join the CloudStack users mailing list and ask questions there - http://cloudstack.apache.org/mailing-lists.html
I got that same error message when following the official install guide when starting the mysql server. The problem was for me that [mysqld] was missing in the my.conf file before the config snippet. The documentation is misleading in that case (like the secion header is only relevant when editing that alternative mysql config file mentioned later there).
Unable to start httpd service on Centos 7 "GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: Cannot determine user of subject"
When trying to (re)start httpd service I get the following error: ** (pkttyagent:2574): WARNING **: 01:33:55.345: Unable to register authentication agent: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: Cannot determine user of subject Error registering authentication agent: GDBus.Error:org.freedesktop.PolicyKit1.Error.Failed: Cannot determine user of subject (polkit-error-quark, 0) Job for httpd.service failed because the control process exited with error code. See "systemctl status httpd.service" and "journalctl -xe" for details. Then systemctl status httpd.service sheds no light on the problem: httpd.service - Web server Apache Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Wed 2020-07-22 01:33:55 BST; 26min ago Process: 2578 ExecStart=/usr/local/apache/bin/apachectl start (code=exited, status=1/FAILURE) Jul 22 01:33:55 server... systemd[1]: Starting Web server Apache... Jul 22 01:33:55 server... systemd[1]: httpd.service: control process exited, code=exited status=1 Jul 22 01:33:55 server... systemd[1]: Failed to start Web server Apache. Jul 22 01:33:55 server... systemd[1]: Unit httpd.service entered failed state. Jul 22 01:33:55 server... systemd[1]: httpd.service failed. What might be causing this? Where should I look to debug?
[Thu Jul 23 05:40:44.885963 2020] [ssl:emerg] [pid 2877:tid 140514669713280] AH02565: Certificate and private key do not match Looks like key and certificate do not match. Please check md5 of the key and cert file.
This usually happens when trying to restart a systemd service. The workaround is as follows: Run this as root groupadd -g 23 nohidproc usermod -a -G nohidproc polkitd mount -o remount,rw,hidepid=2,gid=nohidproc /proc systemctl restart polkit
apache2.service: Failed to run 'start' task: No such file or directory
I can't start my apache server on debian 9. I tried reinstall : sudo apt-get autoremove --purge apache2 && sudo apt-get install apache2 but no change... Job for apache2.service failed because of unavailable resources or another system error. See "systemctl status apache2.service" and "journalctl -xe" for details. invoke-rc.d: initscript apache2, action "restart" failed. systemctl status apache2.service ● apache2.service - The Apache HTTP Server Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled) Active: failed (Result: resources) journalctl -xeu apache2.service (I set loglevel to debug mod) Sep 05 11:45:44 systemd[1]: apache2.service: Failed with result 'resources'. Sep 05 11:50:26 systemd[1]: apache2.service: Changed dead -> failed Sep 05 11:50:27 systemd[1]: apache2.service: Trying to enqueue job apache2.service/stop/replace Sep 05 11:50:27 systemd[1]: apache2.service: Installed new job apache2.service/stop as 1415 Sep 05 11:50:27 systemd[1]: apache2.service: Enqueued job apache2.service/stop as 1415 Sep 05 11:50:27 systemd[1]: apache2.service: Job apache2.service/stop finished, result=done Sep 05 11:50:27 systemd[1]: apache2.service: Changed dead -> failed Sep 05 11:50:30 systemd[1]: apache2.service: Failed to run 'start' task: No such file or directory Sep 05 11:50:30 systemd[1]: Failed to start The Apache HTTP Server. -- Subject: Unit apache2.service has failed -- Defined-By: systemd -- Support: https://www.debian.org/support -- -- Unit apache2.service has failed. -- -- The result is failed. Sep 05 11:50:30 systemd[1]: apache2.service: Failed with result 'resources'. what's wrong?
Maybe this is a problem with service 'tmp' directory. I has a similar error with systemd-resolved.service, and reason was missing '/var/tmp' directory after system migration. Check what temp directory the service is using and create it if necessary.
Also, if there is systemd newly running and crap in /var/tmp/, you might have to clear up this crap and try running the service again. In my case it turned out to be this (without apache2 running at the time): root#www:/var/tmp # ls -al total 32 drwxrwxrwt 8 root root 4096 Dec 15 12:48 . drwxr-xr-x 14 root root 4096 Jul 8 21:43 .. drwx------ 2 root root 4096 Dec 15 12:48 systemd-private-1dcdfe608b6c41f387936225d86126c7-apache2.service-L0KeaS drwx------ 2 root root 4096 Dec 8 03:09 systemd-private-39294ac7bf4b44198d87d45660dcbac2-phpsessionclean.service-4ShLZm drwx------ 2 root root 4096 Dec 15 04:00 systemd-private-451ad0c3bfe6435891a80a6c714a222b-apache2.service-YQyZes drwx------ 2 root root 4096 Dec 15 07:09 systemd-private-451ad0c3bfe6435891a80a6c714a222b-phpsessionclean.service-5L25TU drwx------ 3 root root 4096 Dec 15 03:53 systemd-private-68bc1493e8804c968af642a2319c4e79-apache2.service-RY1iLF
Restoring rabbitmq-server database to a different host
Im running rabbitmq-server-3.6.6-1.el7.noarch on a CentOS 7.4.1708 server.
The /var/lib/ directory, on an ext4 lvm partition, had insufficient storage issues which required us to extend the lvm online, in order to make more space.
This seemed to fix the problems at the time, but a rabbitmq-server service restart was needed, which, when attempted, hung.
The service never started up anymore.
In order to get rabbitmq working again, the old mnesia directory was backed up, and a new one was created.
In order to recover the messages in the broken service, I have moved the old mnesia to a new server, added NODENAME=rabbit#oldserver to /etc/rabbitmq/rabbitmq-env.conf on the new server and tried to start it, but it keeps failing to start.
Please how can I start the old rabbitmq database on the new host?
[root#newserver]# cat /etc/rabbitmq/rabbitmq-env.conf
NODENAME=rabbit#oldserver
When I try to start the service on the new server:
[root#newserver rabbitmq]# systemctl status rabbitmq-server.service -l
? rabbitmq-server.service - RabbitMQ broker
Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Mon 2018-05-07 16:25:23 WAT; 1h 32min ago
Process: 6484 ExecStop=/usr/sbin/rabbitmqctl stop (code=exited, status=0/SUCCESS)
Process: 5997 ExecStart=/usr/sbin/rabbitmq-server (code=exited, status=1/FAILURE)
Main PID: 5997 (code=exited, status=1/FAILURE)
Status: "Exited."
May 07 16:25:23 newserver.dom.local rabbitmqctl[6484]: * epmd reports: node 'rabbit' not running at all
May 07 16:25:23 newserver.dom.local rabbitmqctl[6484]: other nodes on oldserver: ['rabbitmq-cli-20']
May 07 16:25:23 newserver.dom.local rabbitmqctl[6484]: * suggestion: start the node
May 07 16:25:23 newserver.dom.local rabbitmqctl[6484]: current node details:
May 07 16:25:23 newserver.dom.local rabbitmqctl[6484]: - node name: 'rabbitmq-cli-20#newserver'
May 07 16:25:23 newserver.dom.local rabbitmqctl[6484]: - home dir: .
May 07 16:25:23 newserver.dom.local rabbitmqctl[6484]: - cookie hash: edMXQlaNlKXH72ZvAXFhbw==
May 07 16:25:23 newserver.dom.local systemd[1]: Failed to start RabbitMQ broker.
May 07 16:25:23 newserver.dom.local systemd[1]: Unit rabbitmq-server.service entered failed state.
May 07 16:25:23 newserver.dom.local systemd[1]: rabbitmq-server.service failed.
=ERROR REPORT==== 7-May-2018::16:16:43 ===
** Generic server <0.145.0> terminating
** Last message in was {'$gen_cast',
{submit_async,
#Fun<rabbit_queue_index.32.103862237>}}
** When Server state == undefined
** Reason for termination ==
** {function_clause,
[{rabbit_queue_index,parse_segment_entries,
[<<1,23,0,255,54,241,95,251,201,20,69,202,0,0,0,0,0,0,0,0,0,0,0,
240,0,0,1,176,131,104,6,100,0,13,98,97,115,105,99,95,109,101,
115,115,97,103,101,104,4,100,0,8,114,101,115,111,117,114,99,101,
109,0,0,0,8,47,98,105,108,108,105,110,103,100,0,8,101,120,99,
104,97,110,103,101,109,0,0,0,7,98,105,108,108,105,110,103,108,0,
0,0,1,109,0,0,0,13,98,105,108,108,105,110,103,95,113,117,101,
117,101,106,104,6,100,0,7,99,111,110,116,101,110,116,97,60,100,
0,4,110,111,110,101,109,0,0,0,7,48,0,0,0,0,0,2,100,0,25,114,97,
98,98,105,116,95,102,114,97,109,105,110,103,95,97,109,113,112,
95,48,95,57,95,49,108,0,0,0,1,109,0,0,0,240,123,34,115,104,111,
114,116,67,111,100,101,34,58,34,52,50,54,95,109,101,110,117,115,
34,44,34,116,105,109,101,115,116,97,109,112,34,58,34,50,48,49,
56,45,48,52,45,49,56,84,49,48,58,53,57,58,50,55,46,57,51,53,90,
34,44,34,109,115,105,115,100,110,34,58,34,50,51,52,57,48,57,51,
52,52,57,50,57,51,34,44,34,105,100,34,58,34,50,55,50,49,56,95,
50,51,52,57,48,57,51,52,52,57,50,57,51,95,49,53,50,52,48,52,57,
48,52>>,
------snip-------goes-on-forever----
100,0,4,116,114,117,101>>},
no_del,no_ack},
undefined,undefined,undefined,undefined,undefined,
undefined,undefined,undefined},
10,10,10,10},
100,100,100,100,100},
1000,1000,1000,1000,1000,1000,1000},
10000,10000,10000,10000,10000,10000,10000,10000,10000}},
8988}],
[{file,"src/rabbit_queue_index.erl"},{line,1067}]},
{rabbit_queue_index,'-recover_journal/1-fun-0-',1,
[{file,"src/rabbit_queue_index.erl"},{line,863}]},
{lists,map,2,[{file,"lists.erl"},{line,1224}]},
{rabbit_queue_index,segment_map,2,
[{file,"src/rabbit_queue_index.erl"},{line,989}]},
{rabbit_queue_index,recover_journal,1,
[{file,"src/rabbit_queue_index.erl"},{line,856}]},
{rabbit_queue_index,scan_segments,3,
[{file,"src/rabbit_queue_index.erl"},{line,676}]},
{rabbit_queue_index,queue_index_walker_reader,2,
[{file,"src/rabbit_queue_index.erl"},{line,664}]},
{rabbit_queue_index,'-queue_index_walker/1-fun-0-',2,
[{file,"src/rabbit_queue_index.erl"},{line,645}]}]}