I have a very simple chef recipe which attempts to install RabbitMQ and several RabbitMQ Plugins on an EC2 & Vagrant VM (both running Ubuntu 12.04).
I'm using the OpsCode RabbitMQ cookbook (https://github.com/opscode-cookbooks/rabbitmq), and when I provision the machine(s) I get the following stack trace:
INFO: Processing apt_repository[rabbitmq] action add (rabbitmq::default line 43)
INFO: Processing remote_file[/srv/chef/file_store/rabbitmq-signing-key-public.asc] action create (/srv/chef/file_store/cookbooks/apt/providers/repository.rb line 36)
INFO: remote_file[/srv/chef/file_store/rabbitmq-signing-key-public.asc] updated
INFO: remote_file[/srv/chef/file_store/rabbitmq-signing-key-public.asc] mode changed to 644
INFO: Processing execute[install-key rabbitmq-signing-key-public.asc] action run (/srv/chef/file_store/cookbooks/apt/providers/repository.rb line 52)
INFO: execute[install-key rabbitmq-signing-key-public.asc] ran successfully
INFO: apt_repository[rabbitmq] sending run action to execute[apt-get update] (immediate)
INFO: Processing execute[apt-get update] action run (apt::default line 29)
INFO: execute[apt-get update] ran successfully
INFO: Processing remote_file[/srv/chef/file_store/rabbitmq-signing-key-public.asc] action nothing (/srv/chef/file_store/cookbooks/apt/providers/repository.rb line 36)
INFO: Processing execute[install-key rabbitmq-signing-key-public.asc] action nothing (/srv/chef/file_store/cookbooks/apt/providers/repository.rb line 52)
INFO: Processing execute[apt-get update] action nothing (/srv/chef/file_store/cookbooks/apt/providers/repository.rb line 78)
INFO: Processing file[/etc/apt/sources.list.d/rabbitmq-source.list] action create (/srv/chef/file_store/cookbooks/apt/providers/repository.rb line 89)
INFO: file[/etc/apt/sources.list.d/rabbitmq-source.list] created file /etc/apt/sources.list.d/rabbitmq-source.list
INFO: Processing package[util-linux] action install (rabbitmq::default line 53)
INFO: Processing package[rabbitmq-server] action install (rabbitmq::default line 54)
ERROR: package[rabbitmq-server] (rabbitmq::default line 54) has had an error
...
FATAL: Chef::Exceptions::Exec: package[rabbitmq-server] (rabbitmq::default line 54) had an error: Chef::Exceptions::Exec: apt-get -q -y install rabbitmq-server=2.7.1-0ubuntu4 returned 100, expected 0
As you can see by the logs the apt-get update is happening before the new rabbitmq source is added (meaning it tries to install rabbitmq-server version 2.7.1 vs. the 2.8.x version from the RabbitMQ repository.
This is the cookbook code: https://github.com/opscode-cookbooks/rabbitmq/blob/master/recipes/default.rb#L43-54
Any ideas how to resolve the issue?
I've got the 2 fixes you need.
apt_repository wasn't kicking off a proper apt-get update when new repos were added.
http://tickets.opscode.com/browse/COOK-1530 has the fix.
rabbitmq's apt_repository wasn't triggering the apt-get update
http://tickets.opscode.com/browse/COOK-1496 with the apt cookbook update fixes it. This was merged to https://github.com/opscode-cookbooks/rabbitmq master today.
New releases of the 2 cookbooks should fix everything, hopefully this week. Feel free to grab from github in the meantime and ping me with any further questions.
matt#opscode
Related
HDP version: 3.1.0.78, using ambari 2.7.3 to install.
yarn version: 3.1.1
The application is finished with failed status.
Here is the complete log:
yarn logs -applicationId application_1565920695836_0005
19/08/16 11:49:57 INFO client.AHSProxy: Connecting to Application History server at dn1003.jja.bigo/10.221.117.44:10300
Not a valid BCFile.
Can not find any log file matching the pattern: [ALL] for the application: application_1565920695836_0005
Can not find the logs for the application: application_1565920695836_0005 with the appOwner: hadoop
Thanks for your reply.
UPDATE:
After using debug with export YARN_ROOT_LOGGER="DEBUG,console", it says "/data1/app-logs/hadoop/logs/application_1566555356033_0034/meta" is not valid bcfile.
I'm trying to build a reactor sls file, which starts running when an event occurs.
The content of the sls file should be as the following cli commands:
sudo salt minion git.add /srv/salt .
sudo salt minion git.commit /srv/salt test
sudo salt minion git.push /srv/salt origin master identity=/home/autogit/.ssh/id_rsa
If i run the code bellow triggered by the reactor. I get the following error message.
[DEBUG ] Reactor is populating module client cache
[ERROR ] An un-handled exception from the multiprocessing process 'Reactor-9:1' was caught:
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/utils/process.py", line 765, in _run
return self._original_run()
File "/usr/lib/python2.7/dist-packages/salt/utils/reactor.py", line 271, in run
self.call_reactions(chunks)
File "/usr/lib/python2.7/dist-packages/salt/utils/reactor.py", line 228, in call_reactions
self.wrap.run(chunk)
File "/usr/lib/python2.7/dist-packages/salt/utils/reactor.py", line 330, in run
self.populate_client_cache(low)
File "/usr/lib/python2.7/dist-packages/salt/utils/reactor.py", line 324, in populate_client_cache
self.reaction_class[reaction_type](self.opts['conf_file'])
KeyError: u'module'
[CRITICAL] Engine 'reactor' could not be started!
I've tried different syntax (old style and new style) but couldn't figure out what the problem is. Always getting an KeyError: u'module' or u'git'.
Also tried it with runner function to run it locally on the master.
git pull:
module.run:
- git.pull:
- cwd: /srv/salt
- remote: git#git.xyz.com:user/sbt.git
- identity: /home/autogit/.ssh/id_rsa
- git.add:
- cwd: /srv/salt
- filename: .
- git.commit:
- cwd: /srv/salt
- remote: git#git.xyz.com:user/sbt.git
- git.push:
- cwd: /srv/salt
- remote: git#git.xyz.com:user/sbt.git
- identity: /home/autogit/.ssh/id_rsa
salt --versions-report
Salt Version:
Salt: 2019.2.0
Dependency Versions:
cffi: Not Installed
cherrypy: unknown
dateutil: 2.6.1
docker-py: Not Installed
gitdb: 2.0.3
gitpython: 2.1.8
ioflo: Not Installed
Jinja2: 2.10
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: 1.0.7
msgpack-pure: Not Installed
msgpack-python: 0.5.6
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.15rc1 (default, Nov 12 2018, 14:31:15)
python-gnupg: 0.4.1
PyYAML: 3.12
PyZMQ: 16.0.2
RAET: Not Installed
smmap: 2.0.3
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.2.5
System Versions:
dist: Ubuntu 18.04 bionic
locale: UTF-8
machine: x86_64
release: 4.15.0-46-generic
system: Linux
version: Ubuntu 18.04 bionic
Since i'm quite new to Salt, hopefully you can give me a hint what i'm doing wrong:
You didn't provide the master config.
About module.run confusion: add in your settings (minion and maybe to master since I don't know your use-case)
use_superseded:
- module.run
That will enable your syntax, more doc about this here: https://docs.saltstack.com/en/latest/ref/states/all/salt.states.module.html#salt.states.module.run
In general: you are executing execution modules from the place that state modules are allowed only (the term module is heavily overused in salt...)
You didn't provide the full Master config. Reactor requires dedicated config to match events to sls files:
https://docs.saltstack.com/en/latest/ref/configuration/master.html#master-reactor-settings
You can also check the doc I've written some time ago about events and reactors:
https://github.com/kiemlicz/util/wiki/Salt-Events-and-Reactor
Assuming you've configured your events-to-sls-files-matching in master config, your provided sls:
git pull:
module.run:
- git.pull:
- cwd: /srv/salt
- remote: git#git.xyz.com:user/
...
will not work.
Mind that reaction happens on Salt Master thus the reaction sls file need to provide type of reaction (local, runner etc.) since it's no longer 'view of one minion' but possibly of tons of minions!
First create runner reaction type (which delegates to some orchestration sls file which will contain your logic wrapped with (I think) salt.function )
Help yourself with aforementioned github link to my attempt of explaining Reactor.
Refer to official doc as well: https://docs.saltstack.com/en/latest/topics/reactor/index.html
When I tried to deploy the Content Runtime it failed with the error:
null_resource.singlenode (remote-exec): ERROR: CONFIGURATION ERROR:Specified config file /etc/opscode/pivotal.rb does not exist
null_resource.singlenode (remote-exec): Creating admin user: chef-admin
null_resource.singlenode: Still creating... (8m30s elapsed)
null_resource.singlenode (remote-exec): ERROR: CONFIGURATION ERROR:Specified config file /etc/opscode/pivotal.rb does not exist
Error applying plan:
I see this error when I try to create content-runtime using vSphere or Other template. What could be the cause?
This looks like an issue while installing chef. Run the command on the failed vm:
~/advanced-content-runtime/verify-installation.sh
And check the results.
The command will indicate a failure with chef, but will show you the location of the chef install log:
~/advanced-content-runtime/.advanced-runtime-config/chef-install.log
Then check the log for pivotal:
egrep pivotal ~/advanced-content-runtime/.advanced-runtime-config/chef-install.log
On a system where chef installed correctly, the result seen with the commands above are:
[2017-11-07T16:44:10-06:00] INFO: Storing updated cookbooks/private-chef/templates/default/pivotal.rb.erb in the cache.
[2017-11-07T16:44:13-06:00] INFO: Processing file[/etc/opscode/pivotal.pem] action create (private-chef::private_keys line 33)
[2017-11-07T16:44:13-06:00] INFO: file[/etc/opscode/pivotal.pem] created file /etc/opscode/pivotal.pem
[2017-11-07T16:44:13-06:00] INFO: file[/etc/opscode/pivotal.pem] updated file contents /etc/opscode/pivotal.pem
[2017-11-07T16:44:13-06:00] INFO: file[/etc/opscode/pivotal.pem] owner changed to 999
[2017-11-07T16:44:13-06:00] INFO: file[/etc/opscode/pivotal.pem] group changed to 0
[2017-11-07T16:44:13-06:00] INFO: file[/etc/opscode/pivotal.pem] mode changed to 600
[2017-11-07T16:47:51-06:00] INFO: Processing template[/etc/opscode/pivotal.rb] action create (private-chef::ctl_config line 32)
[2017-11-07T16:47:51-06:00] INFO: template[/etc/opscode/pivotal.rb] created file /etc/opscode/pivotal.rb
[2017-11-07T16:47:51-06:00] INFO: template[/etc/opscode/pivotal.rb] updated file contents /etc/opscode/pivotal.rb
[2017-11-07T16:47:51-06:00] INFO: template[/etc/opscode/pivotal.rb] owner changed to 0
[2017-11-07T16:47:51-06:00] INFO: template[/etc/opscode/pivotal.rb] group changed to 0
[2017-11-07T16:47:51-06:00] INFO: template[/etc/opscode/pivotal.rb] mode changed to 644
From review of the chef logs, you might find a failed chef config, specifically this issue: https://github.com/chef/chef-server/issues/987.
You can clean the chef install (chef-server-ctl cleanse), and launch the command line to then successfully complete the install. From CAM perspective, you can resubmit the other template create of the content runtime, and the VM will then become usable and known to CAM.
I was following instruction given in the link. Updated my repository list and tried to upgrade the version but it gives the following error.
Earlier, cluster of two node of rabbitmq 3.2.4 was running.
Error:
Reading package lists... Done
Building dependency tree
Reading state information... Done
rabbitmq-server is already the newest version.
0 upgraded, 0 newly installed, 0 to remove and 467 not upgraded.
1 not fully installed or removed.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] Y
Setting up rabbitmq-server (3.6.1-1) ...
* Starting message broker rabbitmq-server * FAILED - check /var/log/rabbitmq/startup_\{log, _err\}
[fail]
invoke-rc.d: initscript rabbitmq-server, action "start" failed.
dpkg: error processing package rabbitmq-server (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
rabbitmq-server
E: Sub-process /usr/bin/dpkg returned an error code (1)
[1]: http://monkeyhacks.com/post/installing-rabbitmq-on-ubuntu-14-04
In the error log i am getting this :
Cluster upgrade needed but other nodes are running [add_ip_to_listener,
exchange_decorators,
exchange_event_serial,gm,
mirrored_supervisor,
policy_apply_to,
queue_decorators,
remove_user_scope,
semi_durable_route,
topic_trie,
topic_trie_node,
user_admin_to_tags]
and I want [add_ip_to_listener,cluster_name,exchange_event_serial,gm,
internal_system_x,mirrored_supervisor,policy_apply_to,
recoverable_slaves,remove_user_scope,semi_durable_route,
topic_trie,topic_trie_node,user_admin_to_tags,
user_password_hashing]
I'm working on a project with Vagrant and Ansible and Virtualbox.
When I try to install Apache on an ubuntu precise (14.04) box, Vagrant fails. I improved the answer after.
It seems a known bug, but even if I'm installing a newer version, the error shows up.
I tried also as stated here, but with no luck.
How can I resolve this issue?
Thank you.
UPDATED ANSWER
This is the Ansible task.
Version 1:
- name: Install Apache
sudo: yes
apt: pkg=apache2 state=latest
register: apache2_apt
Output:
failed: [default] => {"failed": true}
stderr: E: Sub-process /usr/bin/dpkg returned an error code (1)
stdout: Reading package lists...
Building dependency tree...
Reading state information...
Suggested packages:
www-browser apache2-doc apache2-suexec-pristine apache2-suexec-custom
The following NEW packages will be installed:
apache2
0 upgraded, 1 newly installed, 0 to remove and 183 not upgraded.
Need to get 0 B/146 kB of archives.
After this operation, 460 kB of additional disk space will be used.
(Reading database ... 52932 files and directories currently installed.)
Unpacking apache2 (from .../apache2_2.4.12-1+deb.sury.org~precise+5_amd64.deb) ...
dpkg: error processing /var/cache/apt/archives/apache2_2.4.12-1+deb.sury.org~precise+5_amd64.deb (--unpack):
error setting ownership of `/var/www/html.dpkg-new': Operation not permitted
Processing triggers for man-db ...
Processing triggers for ureadahead ...
Errors were encountered while processing:
/var/cache/apt/archives/apache2_2.4.12-1+deb.sury.org~precise+5_amd64.deb
msg: '/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" install 'apache2'' failed: E: Sub-process /usr/bin/dpkg returned an error code (1)
FATAL: all hosts have already failed -- aborting
Version 2:
- name: Install Apache
command: "sudo apt-get install apache2"
register: apache2_apt
Output:
failed: [default] => {"changed": true, "cmd": ["sudo", "apt-get", "install", "apache2"], "delta": "0:00:07.745095", "end": "2015-06-09 11:08:53.726031", "rc": 100, "start": "2015-06-09 11:08:45.980936", "warnings": []}
stderr: E: Sub-process /usr/bin/dpkg returned an error code (1)
stdout: Reading package lists...
Building dependency tree...
Reading state information...
Suggested packages:
www-browser apache2-doc apache2-suexec-pristine apache2-suexec-custom
The following NEW packages will be installed:
apache2
0 upgraded, 1 newly installed, 0 to remove and 183 not upgraded.
Need to get 0 B/146 kB of archives.
After this operation, 460 kB of additional disk space will be used.
(Reading database ... 52932 files and directories currently installed.)
Unpacking apache2 (from .../apache2_2.4.12-1+deb.sury.org~precise+5_amd64.deb) ...
dpkg: error processing /var/cache/apt/archives/apache2_2.4.12-1+deb.sury.org~precise+5_amd64.deb (--unpack):
error setting ownership of `/var/www/html.dpkg-new': Operation not permitted
Processing triggers for man-db ...
Processing triggers for ureadahead ...
ureadahead will be reprofiled on next reboot
Errors were encountered while processing:
/var/cache/apt/archives/apache2_2.4.12-1+deb.sury.org~precise+5_amd64.deb
FATAL: all hosts have already failed -- aborting
There are few possible issues for this
You need to disable apparmor or better add a rule to apparmor service for ability to have access by the script to /var/www within guest machine
There is a trouble with host machine permissions for /var/www folder. Try to check if the user has access to local folder, mapped as shared folder from host to guest - possibly you need to add permissions for local user at host machine.
try to use ansible-galaxy and search for already created role with fixes for both previous issues