Apache Metrics Collector install failed while deploying Apache Ambari 2.5.1 - ambari

I've tried to deploy Apache Ambari 2.5.1, and Apache Metrics Collector install failed. I have researched this issue and I can not find the same issue in the Internet. Can you help me to solve this problem? Thanks!
stderr:
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py", line 86, in <module> AmsCollector().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 329, in execute method(env)
File "/var/lib/ambari-agent/cache/common-services/AMBARI_METRICS/0.1.0/package/scripts/metrics_collector.py", line 36, in install self.install_packages(env)
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 693, in install_packages retry_count=agent_stack_retry_count)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 155, in __init__ self.env.run()
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 160, in run self.run_action(resource, action)
File "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", line 124, in run_action provider_action()
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 54, in action_install self.install_package(package_name, self.resource.use_repos, self.resource.skip_repos)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/yumrpm.py", line 51, in install_package self.checked_call_with_retries(cmd, sudo=True, logoutput=self.get_logoutput())
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 86, in checked_call_with_retries return self._call_with_retries(cmd, is_checked=True, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/providers/package/__init__.py", line 98, in _call_with_retries code, out = func(cmd, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 72, in inner result = function(command, **kwargs)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 102, in checked_call tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 150, in _call_wrapper result = _call(command, **kwargs_copy)
File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 303, in _call raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 -y install ambari-metrics-collector' returned 1.
Error: Nothing to do
stdout:
2017-07-19 17:09:34,336 - Stack Feature Version Info: stack_version=2.6, version=None, current_cluster_version=None -> 2.6
2017-07-19 17:09:34,338 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf User Group mapping (user_group) is missing in the hostLevelParams
2017-07-19 17:09:34,341 - Group['livy'] {}
2017-07-19 17:09:34,343 - Group['spark'] {}
2017-07-19 17:09:34,343 - Group['hadoop'] {}
2017-07-19 17:09:34,344 - Group['users'] {}
2017-07-19 17:09:34,344 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-19 17:09:34,345 - User['livy'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-19 17:09:34,345 - User['zookeeper'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-19 17:09:34,346 - User['spark'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-19 17:09:34,347 - User['ams'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-19 17:09:34,348 - User['ambari-qa'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2017-07-19 17:09:34,348 - User['tez'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['users']}
2017-07-19 17:09:34,349 - User['hdfs'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-19 17:09:34,350 - User['yarn'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-19 17:09:34,350 - User['hcat'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-19 17:09:34,351 - User['mapred'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2017-07-19 17:09:34,352 - File['/var/lib/ambari-agent/tmp/changeUid.sh'] {'content': StaticFile('changeToSecureUid.sh'), 'mode': 0555}
2017-07-19 17:09:34,354 - Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] {'not_if': '(test $(id -u ambari-qa) -gt 1000) || (false)'}
2017-07-19 17:09:34,361 - Skipping Execute['/var/lib/ambari-agent/tmp/changeUid.sh ambari-qa /tmp/hadoop-ambari-qa,/tmp/hsperfdata_ambari-qa,/home/ambari-qa,/tmp/ambari-qa,/tmp/sqoop-ambari-qa'] due to not_if
2017-07-19 17:09:34,362 - Group['hdfs'] {}
2017-07-19 17:09:34,363 - User['hdfs'] {'fetch_nonlocal_groups': True, 'groups': ['hadoop', 'hdfs']}
2017-07-19 17:09:34,363 - FS Type:
2017-07-19 17:09:34,364 - Directory['/etc/hadoop'] {'mode': 0755}
2017-07-19 17:09:34,395 - File['/usr/hdp/current/hadoop-client/conf/hadoop-env.sh'] {'content': InlineTemplate(...), 'owner': 'hdfs', 'group': 'hadoop'}
2017-07-19 17:09:34,396 - Directory['/var/lib/ambari-agent/tmp/hadoop_java_io_tmpdir'] {'owner': 'hdfs', 'group': 'hadoop', 'mode': 01777}
2017-07-19 17:09:34,421 - Initializing 2 repositories
2017-07-19 17:09:34,422 - Repository['HDP-2.6'] {'base_url': 'http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/BUILDS/2.6.3.0-63', 'action': ['create'], 'components': ['HDP', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP', 'mirror_list': None}
2017-07-19 17:09:34,430 - File['/etc/yum.repos.d/HDP.repo'] {'content': '[HDP-2.6]\nname=HDP-2.6\nbaseurl=http://s3.amazonaws.com/dev.hortonworks.com/HDP/centos6/2.x/BUILDS/2.6.3.0-63\n\npath=/\nenabled=1\ngpgcheck=0'}
2017-07-19 17:09:34,432 - Repository['HDP-UTILS-1.1.0.21'] {'base_url': 'http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6', 'action': ['create'], 'components': ['HDP-UTILS', 'main'], 'repo_template': '[{{repo_id}}]\nname={{repo_id}}\n{% if mirror_list %}mirrorlist={{mirror_list}}{% else %}baseurl={{base_url}}{% endif %}\n\npath=/\nenabled=1\ngpgcheck=0', 'repo_file_name': 'HDP-UTILS', 'mirror_list': None}
2017-07-19 17:09:34,436 - File['/etc/yum.repos.d/HDP-UTILS.repo'] {'content': '[HDP-UTILS-1.1.0.21]\nname=HDP-UTILS-1.1.0.21\nbaseurl=http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos6\n\npath=/\nenabled=1\ngpgcheck=0'}
2017-07-19 17:09:34,437 - Package['unzip'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2017-07-19 17:09:34,511 - Skipping installation of existing package unzip
2017-07-19 17:09:34,511 - Package['curl'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2017-07-19 17:09:34,519 - Skipping installation of existing package curl
2017-07-19 17:09:34,519 - Package['hdp-select'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2017-07-19 17:09:34,526 - Skipping installation of existing package hdp-select
2017-07-19 17:09:34,708 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2017-07-19 17:09:34,712 - checked_call['hostid'] {}
2017-07-19 17:09:34,729 - checked_call returned (0, 'a8c02132')
2017-07-19 17:09:34,733 - Package['ambari-metrics-collector'] {'retry_on_repo_unavailability': False, 'retry_count': 5}
2017-07-19 17:09:34,809 - Installing package ambari-metrics-collector ('/usr/bin/yum -d 0 -e 0 -y install ambari-metrics-collector')
2017-07-19 17:09:37,830 - Execution of '/usr/bin/yum -d 0 -e 0 -y install ambari-metrics-collector' returned 1. Error: Nothing to do
2017-07-19 17:09:37,830 - Failed to install package ambari-metrics-collector. Executing '/usr/bin/yum clean metadata'
2017-07-19 17:09:38,131 - Retrying to install package ambari-metrics-collector after 30 seconds
Command failed after 1 tries

resource_management.core.exceptions.ExecutionFailed: Execution of '/usr/bin/yum -d 0 -e 0 -y install ambari-metrics-collector' returned 1.
Error: Nothing to do
Usually, this error means that some package is not available in repo, already installed and so on. Try running command manually in verbose mode like
/usr/bin/yum -y install ambari-metrics-collector
and post entire yum output.

Taking the analysis part of Dmitriusan's answer and adding that the suggestion to run the yum manually may not work as the issue is about the package not being available in the repo. You may have to either add a repo that contains package in /etc/yum.repos.d/ and then retry or run the command in Dmitriusan's answer. Or manually download the rpm and install it as "rpm -i /path/to/package.rpm"

Related

Catch Scrapy exception when crawling from Airflow

I'm trying to catch the exception that occurs on my spider in a manner that I can mark the task instance as failed. Currently the task finishes and is marked as succeeded. I'm calling the crawl() from PythonOperator in Airflow, as follow:
with DAG(
'MySpider',
default_args=default_args,
schedule_interval=None) as dag:
t1 = python_task = PythonOperator(
task_id="crawler_task",
python_callable=run_crawler,
op_kwargs=dag_kwargs
)
Here is my run_crawler() method:
def run_crawler(**kwargs):
project_settings = set_project_settings({
'FEEDS': {
f'{kwargs["bucket"]}%(time)s.{kwargs["format"]}': {
'format': kwargs["format"],
'encoding': 'utf8',
'store_empty': kwargs["store_empty"]
}
}
})
print("Project settings: ")
pprint(project_settings.attributes.items())
set_connection("airflow", kwargs["gcs_connection_id"])
process = CrawlerProcess(project_settings)
process.crawl(spider.MySpider)
print("Starting crawler...")
process.start()
When running, I'm having problems with GCS credentials, which leads me to an Exception, as follow:
google.auth.exceptions.DefaultCredentialsError: The file /tmp/file_my_credentials.json does not have a valid type. Type is None, expected one of ('authorized_user', 'service_account', 'external_account', 'external_account_authorized_user', 'impersonated_service_account', 'gdch_service_account').
{logging_mixin.py:115} WARNING - [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 21087,
'downloader/request_count': 68,
'downloader/request_method_count/GET': 68,
'downloader/response_bytes': 1863876,
'downloader/response_count': 68,
'downloader/response_status_count/200': 68,
'elapsed_time_seconds': 25.647386,
'feedexport/failed_count/GCSFeedStorage': 1,
'httpcompression/response_bytes': 9212776,
'httpcompression/response_count': 68,
'item_scraped_count': 66,
'log_count/DEBUG': 136,
'log_count/ERROR': 1,
'log_count/INFO': 10,
'log_count/WARNING': 3,
'memusage/max': 264441856,
'memusage/startup': 264441856,
'request_depth_max': 1,
'response_received_count': 68,
'scheduler/dequeued': 68,
'scheduler/dequeued/memory': 68,
'scheduler/enqueued': 68,
'scheduler/enqueued/memory': 68,
[2032-13-13, 09:04:28 UTC] {engine.py:389} INFO - Spider closed (finished)
[2032-13-13, 09:04:28 UTC] {logging_mixin.py:115} WARNING -
[scrapy.core.engine] INFO: Spider closed (finished)
[2032-13-13, 09:04:28 UTC] {python.py:173} INFO - Done. Returned value was: None
[2032-13-13, 09:04:28 UTC] {taskinstance.py:1408} INFO - Marking task as SUCCESS. dag_id=MySpider, task_id=crawler_task, execution_date=2032-13-13, start_date=2032-13-13, end_date=2032-13-13
[2032-13-13, 09:04:28 UTC] {local_task_job.py:156} INFO - Task exited with return code 0
[2032-13-13, 09:04:28 UTC] {local_task_job.py:279} INFO - 0 downstream tasks scheduled from follow-on schedule check
As you can see, even having this exception, the task itself is marked as "SUCCESS". Is it possible to catch it in order to mark as FAILED, then we can follow it on airflow (Composer) interface?
Thank you
I don't understand why in this case the exception doesn't break the task.
You can add a try except in the run_crawler method and then raise you own exception in the except bloc :
import logging
def run_crawler(**kwargs):
class CustomException(Exception):
pass
try:
project_settings = set_project_settings({
'FEEDS': {
f'{kwargs["bucket"]}%(time)s.{kwargs["format"]}': {
'format': kwargs["format"],
'encoding': 'utf8',
'store_empty': kwargs["store_empty"]
}
}
})
print("Project settings: ")
pprint(project_settings.attributes.items())
set_connection("airflow", kwargs["gcs_connection_id"])
process = CrawlerProcess(project_settings)
process.crawl(spider.MySpider)
print("Starting crawler...")
process.start()
except Exception as err:
logging.error("Error in the Airflow task !!!!!", err)
raise CustomException("Error in the Airflow task !!!!!", err)
In this case when your custom exception will be raised, it will break the Airflow and mark it as failed.

using NAPALM to connect to Cisco Router using SSH Key not working

im trying to get netbox to get configs using NAPALM, but even the basic connection using napalm isnt working, always errors around the key file, which we need to use (cannot use password).
napalm==3.4.1
python=3.8
referred to the documentation at https://napalm.readthedocs.io/en/latest/support/ for use of arguments
I can directly SSH to the router using the key in ~/.ssh/ for the user netbox
netbox#netbox:~/.ssh$ ssh -c aes128-cbc netbox#172.1.1.1
The authenticity of host '172.1.1.1 (172.1.1.1)' can't be established.
RSA key fingerprint is SHA256:<fingerprinthere>.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.1.1.1' (RSA) to the list of known hosts.
PHC-SW01>
Prompts for password and fails
napalm --user netbox --vendor ios --optional_args 'use_keys=true, secret=passwordhere' --debug 172.1.1.1 call get_facts
/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.18ubuntu0.18.04.1 is an invalid version and will not be supported in a future release
warnings.warn(
/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.18ubuntu0.18.04.1 is an invalid version and will not be supported in a future release
warnings.warn(
Enter password:
2022-08-24 13:03:01,926 - napalm - DEBUG - Starting napalm's debugging tool
2022-08-24 13:03:01,926 - napalm - DEBUG - Gathering napalm packages
2022-08-24 13:03:01,926 - napalm - DEBUG - napalm==3.4.1
2022-08-24 13:03:01,926 - napalm - DEBUG - get_network_driver - Calling with args: ('ios',), {}
2022-08-24 13:03:01,927 - napalm - DEBUG - get_network_driver - Successful
Traceback (most recent call last):
File "/usr/local/bin/napalm", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/cl_napalm.py", line 308, in main
run_tests(args)
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/cl_napalm.py", line 269, in run_tests
optional_args = helpers.parse_optional_args(args.optional_args)
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/helpers.py", line 131, in parse_optional_args
return {
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/helpers.py", line 132, in <dictcomp>
x.split("=")[0]: ast.literal_eval(x.split("=")[1])
File "/usr/lib/python3.8/ast.py", line 99, in literal_eval
return _convert(node_or_string)
File "/usr/lib/python3.8/ast.py", line 98, in _convert
return _convert_signed_num(node)
File "/usr/lib/python3.8/ast.py", line 75, in _convert_signed_num
return _convert_num(node)
File "/usr/lib/python3.8/ast.py", line 66, in _convert_num
_raise_malformed_node(node)
File "/usr/lib/python3.8/ast.py", line 63, in _raise_malformed_node
raise ValueError(f'malformed node or string: {node!r}')
ValueError: malformed node or string: <_ast.Name object at 0x7fa719a35af0>
napalm --user netbox --password '' --vendor ios --optional_args 'use_keys=true, secret=passwordhere' --debug 172.1.1.1 call get_facts
warnings.warn(
/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.18ubuntu0.18.04.1 is an invalid version and will not be supported in a future release
warnings.warn(
2022-08-24 13:05:13,330 - napalm - DEBUG - Starting napalm's debugging tool
2022-08-24 13:05:13,330 - napalm - DEBUG - Gathering napalm packages
2022-08-24 13:05:13,331 - napalm - DEBUG - napalm==3.4.1
2022-08-24 13:05:13,331 - napalm - DEBUG - get_network_driver - Calling with args: ('ios',), {}
2022-08-24 13:05:13,331 - napalm - DEBUG - get_network_driver - Successful
Traceback (most recent call last):
File "/usr/local/bin/napalm", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/cl_napalm.py", line 308, in main
run_tests(args)
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/cl_napalm.py", line 269, in run_tests
optional_args = helpers.parse_optional_args(args.optional_args)
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/helpers.py", line 131, in parse_optional_args
return {
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/helpers.py", line 132, in <dictcomp>
x.split("=")[0]: ast.literal_eval(x.split("=")[1])
File "/usr/lib/python3.8/ast.py", line 99, in literal_eval
return _convert(node_or_string)
File "/usr/lib/python3.8/ast.py", line 98, in _convert
return _convert_signed_num(node)
File "/usr/lib/python3.8/ast.py", line 75, in _convert_signed_num
return _convert_num(node)
File "/usr/lib/python3.8/ast.py", line 66, in _convert_num
_raise_malformed_node(node)
File "/usr/lib/python3.8/ast.py", line 63, in _raise_malformed_node
raise ValueError(f'malformed node or string: {node!r}')
ValueError: malformed node or string: <_ast.Name object at 0x7f86f4974b80>
napalm --user netbox --vendor ios --optional_args 'alt_host_keys=True, alt_key_file=/home/netbox/.ssh/id_rsa' --debug 172.1.1.1 call get_facts
Prompts for password, when entered gives below.. shouldnt need password when keyfile is used.
/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.18ubuntu0.18.04.1 is an invalid version and will not be supported in a future release
warnings.warn(
/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.18ubuntu0.18.04.1 is an invalid version and will not be supported in a future release
warnings.warn(
Enter password:
2022-08-24 13:14:59,741 - napalm - DEBUG - Starting napalm's debugging tool
2022-08-24 13:14:59,741 - napalm - DEBUG - Gathering napalm packages
2022-08-24 13:14:59,741 - napalm - DEBUG - napalm==3.4.1
2022-08-24 13:14:59,741 - napalm - DEBUG - get_network_driver - Calling with args: ('ios',), {}
2022-08-24 13:14:59,742 - napalm - DEBUG - get_network_driver - Successful
Traceback (most recent call last):
File "/usr/local/bin/napalm", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/cl_napalm.py", line 308, in main
run_tests(args)
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/cl_napalm.py", line 269, in run_tests
optional_args = helpers.parse_optional_args(args.optional_args)
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/helpers.py", line 131, in parse_optional_args
return {
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/helpers.py", line 132, in <dictcomp>
x.split("=")[0]: ast.literal_eval(x.split("=")[1])
File "/usr/lib/python3.8/ast.py", line 59, in literal_eval
node_or_string = parse(node_or_string, mode='eval')
File "/usr/lib/python3.8/ast.py", line 47, in parse
return compile(source, filename, mode, flags,
File "<unknown>", line 1
/home/netbox/.ssh/id_rsa
^
SyntaxError: invalid syntax
napalm --user netbox --vendor ios --optional_args "hostkey_verify=False, look_for_keys=true, key_file=~/.ssh/id_rsa" --debug 172.1.1.1 call get_facts
/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.18ubuntu0.18.04.1 is an invalid version and will not be supported in a future release
warnings.warn(
/usr/local/lib/python3.8/dist-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.18ubuntu0.18.04.1 is an invalid version and will not be supported in a future release
warnings.warn(
Enter password:
2022-08-24 13:18:37,500 - napalm - DEBUG - Starting napalm's debugging tool
2022-08-24 13:18:37,501 - napalm - DEBUG - Gathering napalm packages
2022-08-24 13:18:37,501 - napalm - DEBUG - napalm==3.4.1
2022-08-24 13:18:37,501 - napalm - DEBUG - get_network_driver - Calling with args: ('ios',), {}
2022-08-24 13:18:37,501 - napalm - DEBUG - get_network_driver - Successful
Traceback (most recent call last):
File "/usr/local/bin/napalm", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/cl_napalm.py", line 308, in main
run_tests(args)
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/cl_napalm.py", line 269, in run_tests
optional_args = helpers.parse_optional_args(args.optional_args)
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/helpers.py", line 131, in parse_optional_args
return {
File "/usr/local/lib/python3.8/dist-packages/napalm/base/clitools/helpers.py", line 132, in <dictcomp>
x.split("=")[0]: ast.literal_eval(x.split("=")[1])
File "/usr/lib/python3.8/ast.py", line 99, in literal_eval
return _convert(node_or_string)
File "/usr/lib/python3.8/ast.py", line 98, in _convert
return _convert_signed_num(node)
File "/usr/lib/python3.8/ast.py", line 75, in _convert_signed_num
return _convert_num(node)
File "/usr/lib/python3.8/ast.py", line 66, in _convert_num
_raise_malformed_node(node)
File "/usr/lib/python3.8/ast.py", line 63, in _raise_malformed_node
raise ValueError(f'malformed node or string: {node!r}')
ValueError: malformed node or string: <_ast.Name object at 0x7ff3a5c5e6d0>
If i specify --password '' it doesnt prompt for the password.
If i use "alt_key_file" or "key_file" always has the same error:
SyntaxError: invalid syntax
running napalm as user 'netbox' accessing the keyfile should work.
Update:
tried putting quotes within quotes re: https://github.com/napalm-automation/napalm-base/issues/309#issuecomment-461185980
but didnt work
--optional_args "hostkey_verify=False, look_for_keys=true, key_file='~/.ssh/id_rsa'"
by putting quotes within each variable i was able to get the command to work
napalm --user netbox --password '' --vendor ios --optional_args "alt_host_keys='True',alt_key_file='/home/netbox/.ssh/id_rsa'" --debug 172.16.1.1 call get_facts
its failing on the enable after connection, but its connecting using the SSH Key now which is what this was about.

Can't create a database with Odoo: Database creation error: OperationalError()

I'm trying to create my first database here, but I get this error:(error image)
I also checked the log and this is what I found:
022-08-14 23:10:15,799 6576 INFO ? odoo.sql_db: Connection to the database failed
2022-08-14 23:10:15,839 6576 INFO None odoo.service.db: Create database `odoo_db`.
2022-08-14 23:10:15,874 6576 INFO None odoo.sql_db: Connection to the database failed
2022-08-14 23:10:15,875 6576 ERROR None odoo.http:
Traceback (most recent call last):
File "C:\Program Files\Odoo\server\odoo\http.py", line 141, in dispatch_rpc
result = dispatch(method, params)
File "C:\Program Files\Odoo\server\odoo\service\db.py", line 462, in dispatch
return g[exp_method_name](*params)
File "<decorator-gen-14>", line 2, in exp_create_database
File "C:\Program Files\Odoo\server\odoo\service\db.py", line 41, in if_db_mgt_enabled
return method(self, *args, **kwargs)
File "C:\Program Files\Odoo\server\odoo\service\db.py", line 130, in exp_create_database
_create_empty_database(db_name)
File "C:\Program Files\Odoo\server\odoo\service\db.py", line 99, in _create_empty_database
with closing(db.cursor()) as cr:
File "C:\Program Files\Odoo\server\odoo\sql_db.py", line 709, in cursor
return Cursor(self.__pool, self.dbname, self.dsn, serialized=serialized)
File "C:\Program Files\Odoo\server\odoo\sql_db.py", line 259, in __init__
self._cnx = pool.borrow(dsn)
File "C:\Program Files\Odoo\server\odoo\sql_db.py", line 592, in _locked
return fun(self, *args, **kwargs)
File "C:\Program Files\Odoo\server\odoo\sql_db.py", line 660, in borrow
**connection_info)
File "C:\Program Files\Odoo\python\lib\site-packages\psycopg2\__init__.py", line 127, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError
2022-08-14 23:10:15,939 6576 INFO None odoo.sql_db: Connection to the database failed
2022-08-14 23:10:15,964 6576 INFO None werkzeug: 127.0.0.1 - - [14/Aug/2022 23:10:15] "POST /web/database/create HTTP/1.1" 200 - 0 0.000 5.881
2022-08-14 23:10:16,162 6576 INFO ? odoo.sql_db: Connection to the database failed
2022-08-14 23:10:16,165 6576 INFO ? werkzeug: 127.0.0.1 - - [14/Aug/2022 23:10:16] "GET /web/database/Roboto-Regular.ttf HTTP/1.1" 404 - 0 0.000 0.047

Celery RabbitMQ broker failover connect issue

I have 3 RabbitMQ nodes in cluster in HA mode. Each node is on separate Docker container.
I am using Celery version 4 and kombu version 4.
I have used this command to set HA policy:
rabbitmqctl set_policy ha-all "" '{"ha-mode":"all","ha-sync-mode":"automatic"}'
Celery config looks like this:
CELERY = dict(
broker_url=[
'amqp://guest#rabbitmq1:5672',
'amqp://guest#rabbitmq2:5672',
'amqp://guest#rabbitmq3:5672',
],
celery_queue_ha_policy='all',
...
)
Everything works fine until I stop master RabbitMQ application in order to test Celery failover feature using command:
rabbitmqctl stop_app
Immediately after RabbitMQ application is stopped I started seeing errors in log bellow. Frequency of log messages is very high and it doesn't slow down with number of attempts.
According to logs Celery tries to reconnect using next failover, but it gets interrupted by another try to reconnect to master node that was stopped. The same thing happens over and over like in infinite loop.
[2017-03-17 15:10:28,084: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**#rabbitmq1:5672//: [Errno 111] Connection refused.
Will retry using next failover.
[2017-03-17 15:10:28,300: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'information': 'Licensed under the MPL. See http://www.rabbitmq.com/', 'product': 'RabbitMQ', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'capabilities': {'exchange_exchange_bindings': True, 'connection.blocked': True, 'authentication_failure_close': True, 'direct_reply_to': True, 'basic.nack': True, 'per_consumer_qos': True, 'consumer_priorities': True, 'consumer_cancel_notify': True, 'publisher_confirms': True}, 'cluster_name': 'rabbit#rabbitmq1', 'platform': 'Erlang/OTP', 'version': '3.6.6'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']
[2017-03-17 15:10:28,302: DEBUG/MainProcess] ^-- substep ok
[2017-03-17 15:10:28,303: DEBUG/MainProcess] | Consumer: Starting Mingle
[2017-03-17 15:10:28,303: INFO/MainProcess] mingle: searching for neighbors
[2017-03-17 15:10:28,303: DEBUG/MainProcess] using channel_id: 1
[2017-03-17 15:10:28,318: DEBUG/MainProcess] Channel open
[2017-03-17 15:10:28,470: WARNING/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection...
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 318, in start
blueprint.start(self)
File "/usr/local/lib/python2.7/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "/usr/local/lib/python2.7/site-packages/celery/worker/consumer/mingle.py", line 38, in start
self.sync(c)
File "/usr/local/lib/python2.7/site-packages/celery/worker/consumer/mingle.py", line 42, in sync
replies = self.send_hello(c)
File "/usr/local/lib/python2.7/site-packages/celery/worker/consumer/mingle.py", line 55, in send_hello
replies = inspect.hello(c.hostname, our_revoked._data) or {}
File "/usr/local/lib/python2.7/site-packages/celery/app/control.py", line 129, in hello
return self._request('hello', from_node=from_node, revoked=revoked)
File "/usr/local/lib/python2.7/site-packages/celery/app/control.py", line 81, in _request
timeout=self.timeout, reply=True,
File "/usr/local/lib/python2.7/site-packages/celery/app/control.py", line 436, in broadcast
limit, callback, channel=channel,
File "/usr/local/lib/python2.7/site-packages/kombu/pidbox.py", line 315, in _broadcast
serializer=serializer)
File "/usr/local/lib/python2.7/site-packages/kombu/pidbox.py", line 290, in _publish
serializer=serializer,
File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 181, in publish
exchange_name, declare,
File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 187, in _publish
channel = self.channel
File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 209, in _get_channel
channel = self._channel = channel()
File "/usr/local/lib/python2.7/site-packages/kombu/utils/functional.py", line 38, in __call__
value = self.__value__ = self.__contract__()
File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 224, in <lambda>
channel = ChannelPromise(lambda: connection.default_channel)
File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 819, in default_channel
self.connection
File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 802, in connection
self._connection = self._establish_connection()
File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 757, in _establish_connection
conn = self.transport.establish_connection()
File "/usr/local/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
conn.connect()
File "/usr/local/lib/python2.7/site-packages/amqp/connection.py", line 294, in connect
self.transport.connect()
File "/usr/local/lib/python2.7/site-packages/amqp/transport.py", line 120, in connect
self._connect(self.host, self.port, self.connect_timeout)
File "/usr/local/lib/python2.7/site-packages/amqp/transport.py", line 161, in _connect
self.sock.connect(sa)
File "/usr/local/lib/python2.7/socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
error: [Errno 111] Connection refused
[2017-03-17 15:10:28,508: DEBUG/MainProcess] Closed channel #1
[2017-03-17 15:10:28,570: DEBUG/MainProcess] | Consumer: Restarting event loop...
[2017-03-17 15:10:28,572: DEBUG/MainProcess] | Consumer: Restarting Gossip...
[2017-03-17 15:10:28,575: DEBUG/MainProcess] | Consumer: Restarting Heart...
[2017-03-17 15:10:28,648: DEBUG/MainProcess] | Consumer: Restarting Control...
[2017-03-17 15:10:28,655: DEBUG/MainProcess] | Consumer: Restarting Tasks...
[2017-03-17 15:10:28,655: DEBUG/MainProcess] Canceling task consumer...
[2017-03-17 15:10:28,655: DEBUG/MainProcess] | Consumer: Restarting Mingle...
[2017-03-17 15:10:28,655: DEBUG/MainProcess] | Consumer: Restarting Events...
[2017-03-17 15:10:28,672: DEBUG/MainProcess] | Consumer: Restarting Connection...
[2017-03-17 15:10:28,673: DEBUG/MainProcess] | Consumer: Starting Connection
[2017-03-17 15:10:28,947: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**#rabbitmq1:5672//: [Errno 111] Connection refused.
Will retry using next failover.
[2017-03-17 15:10:29,345: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'information': 'Licensed under the MPL. See http://www.rabbitmq.com/', 'product': 'RabbitMQ', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'capabilities': {'exchange_exchange_bindings': True, 'connection.blocked': True, 'authentication_failure_close': True, 'direct_reply_to': True, 'basic.nack': True, 'per_consumer_qos': True, 'consumer_priorities': True, 'consumer_cancel_notify': True, 'publisher_confirms': True}, 'cluster_name': 'rabbit#rabbitmq1', 'platform': 'Erlang/OTP', 'version': '3.6.6'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']
[2017-03-17 15:10:29,506: INFO/MainProcess] Connected to amqp://guest:**#rabbitmq2:5672//
[2017-03-17 15:10:29,535: DEBUG/MainProcess] ^-- substep ok
[2017-03-17 15:10:29,569: DEBUG/MainProcess] | Consumer: Starting Events
[2017-03-17 15:10:29,682: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**#rabbitmq1:5672//: [Errno 111] Connection refused.
Will retry using next failover.
[2017-03-17 15:10:29,740: DEBUG/MainProcess] Start from server, version: 0.9, properties: {'information': 'Licensed under the MPL. See http://www.rabbitmq.com/', 'product': 'RabbitMQ', 'copyright': 'Copyright (C) 2007-2016 Pivotal Software, Inc.', 'capabilities': {'exchange_exchange_bindings': True, 'connection.blocked': True, 'authentication_failure_close': True, 'direct_reply_to': True, 'basic.nack': True, 'per_consumer_qos': True, 'consumer_priorities': True, 'consumer_cancel_notify': True, 'publisher_confirms': True}, 'cluster_name': 'rabbit#rabbitmq1', 'platform': 'Erlang/OTP', 'version': '3.6.6'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']
[2017-03-17 15:10:29,768: DEBUG/MainProcess] ^-- substep ok
[2017-03-17 15:10:29,770: DEBUG/MainProcess] | Consumer: Starting Mingle
[2017-03-17 15:10:29,770: INFO/MainProcess] mingle: searching for neighbors
[2017-03-17 15:10:29,771: DEBUG/MainProcess] using channel_id: 1
[2017-03-17 15:10:29,795: DEBUG/MainProcess] Channel open
[2017-03-17 15:10:29,874: WARNING/MainProcess] consumer: Connection to broker lost. Trying to re-establish the connection...
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/celery/worker/consumer/consumer.py", line 318, in start
blueprint.start(self)
File "/usr/local/lib/python2.7/site-packages/celery/bootsteps.py", line 119, in start
step.start(parent)
File "/usr/local/lib/python2.7/site-packages/celery/worker/consumer/mingle.py", line 38, in start
self.sync(c)
File "/usr/local/lib/python2.7/site-packages/celery/worker/consumer/mingle.py", line 42, in sync
replies = self.send_hello(c)
File "/usr/local/lib/python2.7/site-packages/celery/worker/consumer/mingle.py", line 55, in send_hello
replies = inspect.hello(c.hostname, our_revoked._data) or {}
File "/usr/local/lib/python2.7/site-packages/celery/app/control.py", line 129, in hello
return self._request('hello', from_node=from_node, revoked=revoked)
File "/usr/local/lib/python2.7/site-packages/celery/app/control.py", line 81, in _request
timeout=self.timeout, reply=True,
File "/usr/local/lib/python2.7/site-packages/celery/app/control.py", line 436, in broadcast
limit, callback, channel=channel,
File "/usr/local/lib/python2.7/site-packages/kombu/pidbox.py", line 315, in _broadcast
serializer=serializer)
File "/usr/local/lib/python2.7/site-packages/kombu/pidbox.py", line 290, in _publish
serializer=serializer,
File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 181, in publish
exchange_name, declare,
File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 187, in _publish
channel = self.channel
File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 209, in _get_channel
channel = self._channel = channel()
File "/usr/local/lib/python2.7/site-packages/kombu/utils/functional.py", line 38, in __call__
value = self.__value__ = self.__contract__()
File "/usr/local/lib/python2.7/site-packages/kombu/messaging.py", line 224, in <lambda>
channel = ChannelPromise(lambda: connection.default_channel)
File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 819, in default_channel
self.connection
File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 802, in connection
self._connection = self._establish_connection()
File "/usr/local/lib/python2.7/site-packages/kombu/connection.py", line 757, in _establish_connection
conn = self.transport.establish_connection()
File "/usr/local/lib/python2.7/site-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
conn.connect()
File "/usr/local/lib/python2.7/site-packages/amqp/connection.py", line 294, in connect
self.transport.connect()
File "/usr/local/lib/python2.7/site-packages/amqp/transport.py", line 120, in connect
self._connect(self.host, self.port, self.connect_timeout)
File "/usr/local/lib/python2.7/site-packages/amqp/transport.py", line 161, in _connect
self.sock.connect(sa)
File "/usr/local/lib/python2.7/socket.py", line 228, in meth
return getattr(self._sock,name)(*args)
error: [Errno 111] Connection refused
[2017-03-17 15:10:29,887: DEBUG/MainProcess] Closed channel #1
[2017-03-17 15:10:29,907: DEBUG/MainProcess] | Consumer: Restarting event loop...
[2017-03-17 15:10:29,908: DEBUG/MainProcess] | Consumer: Restarting Gossip...
[2017-03-17 15:10:29,908: DEBUG/MainProcess] | Consumer: Restarting Heart...
[2017-03-17 15:10:29,908: DEBUG/MainProcess] | Consumer: Restarting Control...
[2017-03-17 15:10:29,909: DEBUG/MainProcess] | Consumer: Restarting Tasks...
[2017-03-17 15:10:29,910: DEBUG/MainProcess] Canceling task consumer...
[2017-03-17 15:10:29,911: DEBUG/MainProcess] | Consumer: Restarting Mingle...
[2017-03-17 15:10:29,912: DEBUG/MainProcess] | Consumer: Restarting Events...
[2017-03-17 15:10:29,953: DEBUG/MainProcess] | Consumer: Restarting Connection...
[2017-03-17 15:10:29,954: DEBUG/MainProcess] | Consumer: Starting Connection
[2017-03-17 15:10:30,036: ERROR/MainProcess] consumer: Cannot connect to amqp://guest:**#rabbitmq1:5672//: [Errno 111] Connection refused.
Will retry using next failover.
Unfortunately, Celery documentation doesn't say much about failover topic.
Its definitely bug, I have created issue on GitHub: https://github.com/celery/celery/issues/3921
Thanks to George Psarakis I have managed to avoid bug using --without-mingle flag for Celery workers, eg:
celery worker -A app.tasks -l debug --without-mingle

Ambari 2.2.0 error installing hadoop services

I am installing a complete new cluster using HDP 2.3
When Ambari tries to install the services (HDFS Client or Namenode) it errors out. below is the stderr
stderr: Traceback (most recent call last): File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py", line 35, in <module> BeforeAnyHook().execute() File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 220, in execute method(env) File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py", line 26, in hook import params File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/params.py", line 187, in <module> hadoop_conf_dir = conf_select.get_hadoop_conf_dir(force_latest_on_upgrade=True) File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/conf_select.py", line 374, in get_hadoop_conf_dir select(stack_name, "hadoop", version) File "/usr/lib/python2.6/site-packages/resource_management/libraries/functions/conf_select.py", line 252, in select shell.checked_call(get_cmd("set-conf-dir", package, version), logoutput=False, quiet=False, sudo=True) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 70, in inner result = function(command, **kwargs) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 92, in checked_call tries=tries, try_sleep=try_sleep) File "/usr/lib/python2.6/site-packages**/resource_management/core/shell.py", line 140, in _call_wrapper result = _call(command, **kwargs_copy) File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 291, in _call raise Fail(err_msg) resource_management.core.exceptions.Fail: Execution of 'conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0' returned 1. /usr/hdp/2.3.4.0-3485/hadoop/conf does not exist Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-302.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-302.json', 'INFO', '/var/lib/ambari-agent/tmp']**
stdout: 2016-02-26 23:04:51,884 - The hadoop conf dir /usr/hdp/current/hadoop-client/conf exists, will call conf-select on it for version 2.3.4.0-3485 2016-02-26 23:04:51,885 - Checking if need to create versioned conf dir /etc/hadoop/2.3.4.0-3485/0 2016-02-26 23:04:51,885 - call['conf-select create-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False, 'stderr': -1} 2016-02-26 23:04:51,928 - call returned (1, '/etc/hadoop/2.3.4.0-3485/0 exist already', '') 2016-02-26 23:04:51,929 - checked_call['conf-select set-conf-dir --package hadoop --stack-version 2.3.4.0-3485 --conf-version 0'] {'logoutput': False, 'sudo': True, 'quiet': False} Error: Error: Unable to run the custom hook script ['/usr/bin/python2', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-302.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-302.json', 'INFO', '/var/lib/ambari-agent/tmp']
The answer is in the error log, highlighting it for you:
Execution of 'conf-select set-conf-dir --package hadoop
--stack-version 2.3.4.0-3485 --conf-version 0' returned 1
/usr/hdp/2.3.4.0-3485/hadoop/conf does not exist
Fix your configuration and it will start to work.