Laravel application with Apache gives "use of undefined constant MCRYPT_RIJNDAEL_128 - assumed 'MCRYPT_RIJNDAEL_128'" - apache

I am very new to Laravel and my problems start at the running-my-application stage. For some reason I get this error when I try to access my application at http://127.0.0.1/test/public/ (I have Apache server):
ErrorException in EncryptionServiceProvider.php line 16:
Use of undefined constant MCRYPT_RIJNDAEL_128 - assumed 'MCRYPT_RIJNDAEL_128'
in EncryptionServiceProvider.php line 16
at HandleExceptions->handleError('8', 'Use of undefined constant MCRYPT_RIJNDAEL_128 - assumed 'MCRYPT_RIJNDAEL_128'', '/srv/http/test/vendor/laravel/framework/src/Illuminate/Encryption/EncryptionServiceProvider.php', '16', array('app' => object(Application))) in EncryptionServiceProvider.php line 16
at EncryptionServiceProvider->Illuminate\Encryption\{closure}(object(Application), array()) in Container.php line 773
at Container->build(object(Closure), array()) in Container.php line 656
at Container->make('encrypter', array()) in Application.php line 613
at Application->make('Illuminate\Contracts\Encryption\Encrypter') in Container.php line 887
at Container->resolveClass(object(ReflectionParameter)) in Container.php line 848
at Container->getDependencies(array(object(ReflectionParameter)), array()) in Container.php line 813
at Container->build('Illuminate\Cookie\Middleware\EncryptCookies', array()) in Container.php line 656
at Container->make('Illuminate\Cookie\Middleware\EncryptCookies', array()) in Application.php line 613
at Application->make('Illuminate\Cookie\Middleware\EncryptCookies') in Pipeline.php line 124
at Pipeline->Illuminate\Pipeline\{closure}(object(Request)) in CheckForMaintenanceMode.php line 42
at CheckForMaintenanceMode->handle(object(Request), object(Closure)) in Pipeline.php line 125
at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
at call_user_func(object(Closure), object(Request)) in Pipeline.php line 101
at Pipeline->then(object(Closure)) in Kernel.php line 111
at Kernel->sendRequestThroughRouter(object(Request)) in Kernel.php line 84
at Kernel->handle(object(Request)) in index.php line 53
I have mcrypt installed on my system:
[steelrat#archlinux test]$ php -m | grep mcrypt
mcrypt
[steelrat#archlinux test]$ php -i | grep mcrypt
mcrypt
mcrypt support => enabled
mcrypt_filter support => enabled
mcrypt.algorithms_dir => no value => no value
mcrypt.modes_dir => no value => no value
Moreover, when I start the artisan web server with php artisan serve and access it with http://localhost:8000, everything works fine. What can be the problem here?
I am aware of this topic: Use of undefined constant MCRYPT_RIJNDAEL_128 - assumed 'MCRYPT_RIJNDAEL_128' . But it doesn't look like my case. The problem with me is that it works with artisan serve and doesn't work with apache.

Depends on your system but there's usually separate php.ini for PHP CLI and PHP Apache.
Add the following to the top of public/index.php
<?php phpinfo(); exit;
That will tell you what extensions mod_php knows about and the location of your loaded php.ini

Related

How to compile apache module with included sds library using apxs2

I get example apache2 module here:
https://httpd.apache.org/docs/2.4/developer/modguide.html
and I want to use there this sds library https://github.com/antirez/sds
So I will create new module and goto module dir:
apxs -g -n mymodule
cd mod_mymodule
add on line 44 of mod_mymodule.c:
#include "sds.h"
and into the mymodule_handler on line 49 I will add
sds a = sdsnew("a");
sds b = sdsnew("b");
if (sdscmp(a, b) == 0)
return (DECLINED);
I also put library files (sds.c, sds.h, sdsalloc.h) into the same directory as my module source code file and then I will call:
sudo apxs -i -a -c mod_mymodule.c
Then I will restart apache, but it fails to start because of:
apache2: Syntax error on line 146 of /etc/apache2/apache2.conf: Syntax error on line 1 of /etc/apache2/mods-enabled/mymodule.load: Cannot load /usr/lib/apache2/modules/mod_mymodule.so into server: /usr/lib/apache2/modules/mod_mymodule.so: undefined symbol: sdscmp
Question: How can I modify apxs command to make my new module work with included library?
If you want to link them together into your module, you'd pass in all the .c files. Keep the one that's a module first otherwise you have to also pass -n for the module name.
If "sds" were instead an installed library, you'd pass -lsds to apxs just as you would to compiple without apxs.

Apache 2.4 start/stop throws "undefined symbol: ber_sockbuf_io_udp" error after configuring it with Shibboleth SP 3.2.0

Operating System: Red Hat Enterprise Linux Server 7.9 (Maipo)
Apache version: Apache/2.4.46 (Unix)
Shibboleth version: 3.2.0
Error when trying to stop Apache (apachectl stop):
httpd: Syntax error on line 528 of <Apache>/conf/httpd.conf: Syntax error on line 13 of <Apache>/conf/myshib.conf: Cannot load /usr/lib64/shibboleth/mod_shib_24.so into server: /usr/lib64/libldap_r-2.4.so.2: undefined symbol: ber_sockbuf_io_udp
Line 528 of <Apache>/conf/httpd.conf:
Include conf/myshib.conf
Line 13 of <Apache>/conf/myshib.conf:
LoadModule mod_shib /usr/lib64/shibboleth/mod_shib_24.so
Attempted - https://superuser.com/questions/1283252/slappasswd-symbol-lookup-error-undefined-symbol-ber-sockbuf-io-udp
But running "export LD_LIBRARY_PATH=/lib64:$LD_LIBRARY_PATH" did not resolve the issue.
Output of "ldd /usr/lib64/libldap_r-2.4.so.2" command:
linux-vdso.so.1 => (0x00007ffd1c76e000)
liblber-2.4.so.2 => /usr/lib64/liblber-2.4.so.2 (0x00007f5b3e87d000)
libresolv.so.2 => /usr/lib64/libresolv.so.2 (0x00007f5b3e663000)
libsasl2.so.3 => /usr/lib64/libsasl2.so.3 (0x00007f5b3e446000)
libssl.so.10 => /usr/lib64/libssl.so.10 (0x00007f5b3e1d4000)
libcrypto.so.10 => /usr/lib64/libcrypto.so.10 (0x00007f5b3dd71000)
libssl3.so => /usr/lib64/libssl3.so (0x00007f5b3db14000)
libsmime3.so => /usr/lib64/libsmime3.so (0x00007f5b3d8ec000)
libnss3.so => /usr/lib64/libnss3.so (0x00007f5b3d5b8000)
libnssutil3.so => /usr/lib64/libnssutil3.so (0x00007f5b3d388000)
libplds4.so => /usr/lib64/libplds4.so (0x00007f5b3d184000)
libplc4.so => /usr/lib64/libplc4.so (0x00007f5b3cf7f000)
libnspr4.so => /usr/lib64/libnspr4.so (0x00007f5b3cd41000)
libpthread.so.0 => /usr/lib64/libpthread.so.0 (0x00007f5b3cb25000)
libdl.so.2 => /usr/lib64/libdl.so.2 (0x00007f5b3c921000)
libc.so.6 => /usr/lib64/libc.so.6 (0x00007f5b3c553000)
libcrypt.so.1 => /usr/lib64/libcrypt.so.1 (0x00007f5b3c31c000)
libgssapi_krb5.so.2 => /usr/lib64/libgssapi_krb5.so.2 (0x00007f5b3c0cf000)
libkrb5.so.3 => /usr/lib64/libkrb5.so.3 (0x00007f5b3bde6000)
libk5crypto.so.3 => /usr/lib64/libk5crypto.so.3 (0x00007f5b3bbb3000)
libcom_err.so.2 => /usr/lib64/libcom_err.so.2 (0x00007f5b3b9af000)
libkrb5support.so.0 => /usr/lib64/libkrb5support.so.0 (0x00007f5b3b79f000)
libz.so.1 => /usr/lib64/libz.so.1 (0x00007f5b3b589000)
librt.so.1 => /usr/lib64/librt.so.1 (0x00007f5b3b381000)
/lib64/ld-linux-x86-64.so.2 (0x00007f5b3eceb000)
libfreebl3.so => /usr/lib64/libfreebl3.so (0x00007f5b3b17e000)
libkeyutils.so.1 => /usr/lib64/libkeyutils.so.1 (0x00007f5b3af7a000)
libselinux.so.1 => /usr/lib64/libselinux.so.1 (0x00007f5b3ad53000)
libpcre.so.1 => /usr/lib64/libpcre.so.1 (0x00007f5b3aaf1000)
Then attempted - https://unix.stackexchange.com/questions/193320/yum-corrupted-on-rhel-6
Output of "for lib in /lib64/.so.; do if nm -D $lib|grep ber_sockbuf_io_udp; then echo $lib; fi; done" command:
000000000020e020 D ber_sockbuf_io_udp
/lib64/liblber-2.4.so.2
000000000020e020 D ber_sockbuf_io_udp
/lib64/liblber-2.4.so.2.10.7
U ber_sockbuf_io_udp
/lib64/libldap-2.4.so.2
U ber_sockbuf_io_udp
/lib64/libldap-2.4.so.2.10.7
U ber_sockbuf_io_udp
/lib64/libldap_r-2.4.so.2
U ber_sockbuf_io_udp
/lib64/libldap_r-2.4.so.2.10.7
Output of "rpm -qf /lib64/liblber-2.4.so.2" command:
openldap-2.4.44-22.el7.x86_64
Tried to install 'openldap' but it already existed
yum install openldap-2.4.44-22.el7.x86_64
Loaded plugins: langpacks, product-id, search-disabled-repos
Package openldap-2.4.44-22.el7.x86_64 already installed and latest version
Nothing to do
Restarted entire server and re-installed 'openldap' using "yum reinstall openldap.x86_64" command but no luck.
Apache starts and returns a response when accessing https://hostname.domain.com but downloading the Shib Metadata XML file via Apache using URL https://hostname.domain.com/Shibboleth.sso/Metadata fails.
Apologies if I missed adding anything obvious.
Update:
Works when using older Shibboleth version 3.1.0
similar thread here: https://unix.stackexchange.com/questions/193320/yum-corrupted-on-rhel-6
In my case (RHEL7.8 + Apache 2.4 + Shibboleth 3.2) I was able to resolve the issue by replacing the /usr/lib64/libldap_r-2.4.so.2 library with the one from the Apache directory: <APACHE_ROOT>/HTTPServer/openldap/lib/libldap_r-2.4.so.2
Run:
locate libldap_r-2.4.so.2
to find the location of the library.
In my case I got:
/app/ptc/Windchill_12.0/HTTPServer/openldap/lib/libldap_r-2.4.so.2
/app/ptc/Windchill_12.0/HTTPServer/openldap/lib/libldap_r-2.4.so.2.10.12
/usr/lib/libldap_r-2.4.so.2
/usr/lib/libldap_r-2.4.so.2.10.7
/usr/lib64/libldap_r-2.4.so.2
/usr/lib64/libldap_r-2.4.so.2.10.7
I noticed that the lib used in the error was in /usr/lib64 dir.
I replaced it and now ./apachectl -t reports
"Syntax OK"
I don't know enough about Linux to explain what is going on here or what the correct fix is.
This was my observation and resolved the issue, I believe my steps are a hack though.
Anyone with a more elegant, upgrade proof solution?
Reference: https://groups.google.com/g/repmgr/c/TS7QfYEoNoY
cd /usr/lib64/
ll | grep libldap
lrwxrwxrwx. 1 root root 21 Feb 11 16:42 libldap-2.4.so.2 -> libldap-2.4.so.2.10.7
-rwxr-xr-x. 1 root root 352512 Jun 6 2020 libldap-2.4.so.2.10.7
lrwxrwxrwx. 1 root root 23 Feb 11 16:42 libldap_r-2.4.so.2 -> libldap_r-2.4.so.2.10.7
-rwxr-xr-x. 1 root root 381328 Jun 6 2020 libldap_r-2.4.so.2.10.7
It does seem like /usr/lib64/libldap_r2.4.so.2 is just a symlink to libldap_r-2.4.so.2.10.7.
I wonder if there are missing or deprecated symbols in 2.10.7...
Is there any way to tell the diff in the 2 versions?
UPDATE
I noticed that you can make use of the LoadFile command in the Apache conf.
Adding LoadFile <APACHE_ROOT>/HTTPServer/openldap/lib/libldap_r-2.4.so.2 in my 00-shib.conf file before the LoadModule mod_shib /usr/lib64/shibboleth/mod_shib_24.so entry resolved the issue.
This still seems like a workaround / hack to me and there may be an underlying issue with the libraries / different versions.

Ambari unable to run custom hook for modifying user hive

Attempting to add a client node to cluster via Ambari (v2.7.3.0) (HDP 3.1.0.0-78) and seeing odd error
stderr:
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py", line 38, in <module>
BeforeAnyHook().execute()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 352, in execute
method(env)
File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py", line 31, in hook
setup_users()
File "/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/shared_initialization.py", line 51, in setup_users
fetch_nonlocal_groups = params.fetch_nonlocal_groups,
File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __init__
self.env.run()
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 160, in run
self.run_action(resource, action)
File "/usr/lib/ambari-agent/lib/resource_management/core/environment.py", line 124, in run_action
provider_action()
File "/usr/lib/ambari-agent/lib/resource_management/core/providers/accounts.py", line 90, in action_create
shell.checked_call(command, sudo=True)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 72, in inner
result = function(command, **kwargs)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 102, in checked_call
tries=tries, try_sleep=try_sleep, timeout_kill_strategy=timeout_kill_strategy, returns=returns)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 150, in _call_wrapper
result = _call(command, **kwargs_copy)
File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 314, in _call
raise ExecutionFailed(err_msg, code, out, err)
resource_management.core.exceptions.ExecutionFailed: Execution of 'usermod -G hadoop -g hadoop hive' returned 6. usermod: user 'hive' does not exist in /etc/passwd
Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-632.json', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-632.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1_2', '']2019-11-25 13:07:58,000 - Reporting component version failed
Traceback (most recent call last):
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 363, in execute
self.save_component_version_to_structured_out(self.command_name)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 223, in save_component_version_to_structured_out
stack_select_package_name = stack_select.get_package_name()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 109, in get_package_name
package = get_packages(PACKAGE_SCOPE_STACK_SELECT, service_name, component_name)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 223, in get_packages
supported_packages = get_supported_packages()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 147, in get_supported_packages
raise Fail("Unable to query for supported packages using {0}".format(stack_selector_path))
Fail: Unable to query for supported packages using /usr/bin/hdp-select
stdout:
2019-11-25 13:07:57,644 - Stack Feature Version Info: Cluster Stack=3.1, Command Stack=None, Command Version=None -> 3.1
2019-11-25 13:07:57,651 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2019-11-25 13:07:57,652 - Group['livy'] {}
2019-11-25 13:07:57,654 - Group['spark'] {}
2019-11-25 13:07:57,654 - Group['ranger'] {}
2019-11-25 13:07:57,654 - Group['hdfs'] {}
2019-11-25 13:07:57,654 - Group['zeppelin'] {}
2019-11-25 13:07:57,655 - Group['hadoop'] {}
2019-11-25 13:07:57,655 - Group['users'] {}
2019-11-25 13:07:57,656 - User['yarn-ats'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-11-25 13:07:57,658 - User['hive'] {'gid': 'hadoop', 'fetch_nonlocal_groups': True, 'groups': ['hadoop'], 'uid': None}
2019-11-25 13:07:57,659 - Modifying user hive
Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-632.json', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-632.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1_2', '']
2019-11-25 13:07:57,971 - The repository with version 3.1.0.0-78 for this command has been marked as resolved. It will be used to report the version of the component which was installed
2019-11-25 13:07:58,000 - Reporting component version failed
Traceback (most recent call last):
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 363, in execute
self.save_component_version_to_structured_out(self.command_name)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/script/script.py", line 223, in save_component_version_to_structured_out
stack_select_package_name = stack_select.get_package_name()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 109, in get_package_name
package = get_packages(PACKAGE_SCOPE_STACK_SELECT, service_name, component_name)
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 223, in get_packages
supported_packages = get_supported_packages()
File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 147, in get_supported_packages
raise Fail("Unable to query for supported packages using {0}".format(stack_selector_path))
Fail: Unable to query for supported packages using /usr/bin/hdp-select
Command failed after 1 tries
The problem appears to be
resource_management.core.exceptions.ExecutionFailed: Execution of 'usermod -G hadoop -g hadoop hive' returned 6. usermod: user 'hive' does not exist in /etc/passwd
caused by
2019-11-25 13:07:57,659 - Modifying user hive
Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-632.json', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-632.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1_2', '']
This is further reinforced by the fact that manually adding the ambari-hdp-1.repo and yum-installing hdp-select before adding the host to the cluster shows the same error messages, just truncated up to the parts of stdout/err shown here.
When running
[root#HW001 .ssh]# /usr/bin/hdp-select versions
3.1.0.0-78
from the ambari server node, I can see the command runs.
Looking at what the hook script is trying to run/access, I see
[root#client001~]# ls -lha /var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py
-rw-r--r-- 1 root root 1.2K Nov 25 10:51 /var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py
[root#client001~]# ls -lha /var/lib/ambari-agent/data/command-632.json
-rw------- 1 root root 545K Nov 25 13:07 /var/lib/ambari-agent/data/command-632.json
[root#client001~]# ls -lha /var/lib/ambari-agent/cache/stack-hooks/before-ANY
total 0
drwxr-xr-x 4 root root 34 Nov 25 10:51 .
drwxr-xr-x 8 root root 147 Nov 25 10:51 ..
drwxr-xr-x 2 root root 34 Nov 25 10:51 files
drwxr-xr-x 2 root root 188 Nov 25 10:51 scripts
[root#client001~]# ls -lha /var/lib/ambari-agent/data/structured-out-632.json
ls: cannot access /var/lib/ambari-agent/data/structured-out-632.json: No such file or directory
[root#client001~]# ls -lha /var/lib/ambari-agent/tmp
total 96K
drwxrwxrwt 3 root root 4.0K Nov 25 13:06 .
drwxr-xr-x 10 root root 267 Nov 25 10:50 ..
drwxr-xr-x 6 root root 4.0K Nov 25 13:06 ambari_commons
-rwx------ 1 root root 1.4K Nov 25 13:06 ambari-sudo.sh
-rwxr-xr-x 1 root root 1.6K Nov 25 13:06 create-python-wrap.sh
-rwxr-xr-x 1 root root 1.6K Nov 25 10:50 os_check_type1574715018.py
-rwxr-xr-x 1 root root 1.6K Nov 25 11:12 os_check_type1574716360.py
-rwxr-xr-x 1 root root 1.6K Nov 25 11:29 os_check_type1574717391.py
-rwxr-xr-x 1 root root 1.6K Nov 25 13:06 os_check_type1574723161.py
-rwxr-xr-x 1 root root 16K Nov 25 10:50 setupAgent1574715020.py
-rwxr-xr-x 1 root root 16K Nov 25 11:12 setupAgent1574716361.py
-rwxr-xr-x 1 root root 16K Nov 25 11:29 setupAgent1574717392.py
-rwxr-xr-x 1 root root 16K Nov 25 13:06 setupAgent1574723163.py
notice there is ls: cannot access /var/lib/ambari-agent/data/structured-out-632.json: No such file or directory. Not sure if this is normal, though.
Anyone know what could be causing this or any debugging hints from this point?
UPDATE 01:
Adding some log printing lines near the offending final line in the error trace, ie. File "/usr/lib/ambari-agent/lib/resource_management/libraries/functions/stack_select.py", line 147, in get_supported_packages, I print the code and stdout:
2
ambari-python-wrap: can't open file '/usr/bin/hdp-select': [Errno 2] No such file or directory
So what the heck? It wants hdp-select to already be there, but ambari add-host UI complains if I manually install that binary myself beforehand. When I do manually install it (using the same repo file as in the rest of the existing cluster nodes) all I see is...
0
Packages:
accumulo-client
accumulo-gc
accumulo-master
accumulo-monitor
accumulo-tablet
accumulo-tracer
atlas-client
atlas-server
beacon
beacon-client
beacon-server
druid-broker
druid-coordinator
druid-historical
druid-middlemanager
druid-overlord
druid-router
druid-superset
falcon-client
falcon-server
flume-server
hadoop-client
hadoop-hdfs-client
hadoop-hdfs-datanode
hadoop-hdfs-journalnode
hadoop-hdfs-namenode
hadoop-hdfs-nfs3
hadoop-hdfs-portmap
hadoop-hdfs-secondarynamenode
hadoop-hdfs-zkfc
hadoop-httpfs
hadoop-mapreduce-client
hadoop-mapreduce-historyserver
hadoop-yarn-client
hadoop-yarn-nodemanager
hadoop-yarn-registrydns
hadoop-yarn-resourcemanager
hadoop-yarn-timelinereader
hadoop-yarn-timelineserver
hbase-client
hbase-master
hbase-regionserver
hive-client
hive-metastore
hive-server2
hive-server2-hive
hive-server2-hive2
hive-webhcat
hive_warehouse_connector
kafka-broker
knox-server
livy-client
livy-server
livy2-client
livy2-server
mahout-client
oozie-client
oozie-server
phoenix-client
phoenix-server
pig-client
ranger-admin
ranger-kms
ranger-tagsync
ranger-usersync
shc
slider-client
spark-atlas-connector
spark-client
spark-historyserver
spark-schema-registry
spark-thriftserver
spark2-client
spark2-historyserver
spark2-thriftserver
spark_llap
sqoop-client
sqoop-server
storm-client
storm-nimbus
storm-slider-client
storm-supervisor
superset
tez-client
zeppelin-server
zookeeper-client
zookeeper-server
Aliases:
accumulo-server
all
client
hadoop-hdfs-server
hadoop-mapreduce-server
hadoop-yarn-server
hive-server
Command failed after 1 tries
UPDATE 02:
Printing some custom logging from File "/usr/lib/ambari-agent/lib/resource_management/core/shell.py", line 322 (printing the values of err_msg, code, out, err), ie.
....
312 if throw_on_failure and not code in returns:
313 err_msg = Logger.filter_text("Execution of '{0}' returned {1}. {2}".format(command_alias, c ode, all_output))
314
315 #TODO remove
316 print("\n----------\nMY LOGS\n----------\n")
317 print(err_msg)
318 print(code)
319 print(out)
320 print(err)
321
322 raise ExecutionFailed(err_msg, code, out, err)
323
324 # if separate stderr is enabled (by default it's redirected to out)
325 if stderr == subprocess32.PIPE:
326 return code, out, err
327
328 return code, out
....
I see
Execution of 'usermod -G hadoop -g hadoop hive' returned 6. usermod: user 'hive' does not exist in /etc/passwd
6
usermod: user 'hive' does not exist in /etc/passwd
Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-816.json', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-816.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1_2', '']
2019-11-26 10:25:46,928 - The repository with version 3.1.0.0-78 for this command has been marked as resolved. It will be used to report the version of the component which was installed
So it seems like it is failing to create the hive user (even though it seems to have no problem creating the yarn-ats user before that)
After just giving in and trying to manually create the hive user myself, I see
[root#airflowetl ~]# useradd -g hadoop -s /bin/bash hive
useradd: user 'hive' already exists
[root#airflowetl ~]# cat /etc/passwd | grep hive
<nothing>
[root#airflowetl ~]# id hive
uid=379022825(hive) gid=379000513(domain users) groups=379000513(domain users)
The fact that this existing user's uid looks like this and is not in the /etc/passwd file made me think that there is some existing Active Directory user (which this client node syncs with via installed SSSD) that already has the name hive. Checking our AD users, this turned out to be true.
Temporarily stopping the SSSD service to stop sync with AD (service sssd stop) (since, not sure if you can get a server to ignore AD syncs on an individual user basis) before rerunning the client host add in Ambari fixed the problem for me.

The system failed to determine the “mysqld” version while creating new cpanel account via WHM

I have installed WHM on centOS7 and now I am trying to create new account but getting the database error as follows-
Account Creation Status: failed
Cpanel::Exception/(XID rt23j6) The system failed to determine the “mysqld” version. at /usr/local/cpanel/Cpanel/MysqlUtils/Version.pm line 193. Cpanel::MysqlUtils::Version::current_mysql_version() called at /usr/local/cpanel/Cpanel/Mysql/Version.pm line 27 Cpanel::Mysql::Version::get_mysql_version() called at /usr/local/cpanel/Cpanel/Validate/DB/User.pm line 100 Cpanel::Validate::DB::User::get_max_mysql_dbuser_length() called at /usr/local/cpanel/Cpanel/Validate/DB/User.pm line 157 Cpanel::Validate::DB::User::_mysql_dbuser_name_length_check("demo") called at /usr/local/cpanel/Whostmgr/Accounts/Create.pm line 1246 Whostmgr::Accounts::Create::_validate_database_config(HASH(0x3670408)) called at /usr/local/cpanel/Whostmgr/Accounts/Create.pm line 352 eval {...} called at /usr/local/cpanel/Whostmgr/Accounts/Create.pm line 352 Whostmgr::Accounts::Create::_wwwacct("homedir", "", "maxpark", "unlimited", "useregns", 0, "uid", "", ...) called at /usr/local/cpanel/Whostmgr/Accounts/Create.pm line 2971 Whostmgr::Accounts::Create::__createaccount("max_emailacct_quota", "unlimited", "maxsub", "unlimited", "dbuser", "demo", "mxcheck", "auto", ...) called at /usr/local/cpanel/Whostmgr/Accounts/Create.pm line 3025 Whostmgr::Accounts::Create::_createaccount("max_emailacct_quota", "unlimited", "maxsub", "unlimited", "dbuser", "demo", "mxcheck", "auto", ...) called at whostmgr/bin/whostmgr5.pl line 781 main::wwwacct("wwwacct") called at /usr/local/cpanel/Whostmgr/Dispatch.pm line 259 Whostmgr::Dispatch::_do_call("wwwacct", HASH(0x3210150), HASH(0x3212f80)) called at /usr/local/cpanel/Whostmgr/Dispatch.pm line 157 Whostmgr::Dispatch::dispatch("wwwacct", 1, ARRAY(0x3212f38), HASH(0x3212f80)) called at whostmgr/bin/whostmgr5.pl line 257
This might be a very basic problem but I am a beginner at WHM-Cpanel and I am stuck here and genuinely need help if anyone can do.
Thanks in advance.
Whats your country ?
Maybe your country is under sanctions
There is no problem installing a WHM unless your provider has problems.
Mysql does not provide sanctions to countries but mariadb has no problem.
Clear the server and reinstall OS
Before installing WHM write this code :
mkdir /root/cpanel_profile
echo 'mysql-version=10.3' > /root/cpanel_profile/cpanel.config
cd /home && curl -o latest -L https://securedownloads.cpanel.net/latest && sh latest
with this code you are installing mariadb instead of the mysql
Next method :
You can proxy the yum file in /etc/yum.conf
[main]
………………
proxy=http://<Proxy-Server-IP-Address>:<Proxy_Port>
proxy_username=<Proxy-User-Name>
proxy_password=<Proxy-Password>
Or
proxy=http://<Proxy-User-Name>:<Proxy-Password>#<Proxy-Server-IP-Address>:<Proxy_Port>
………………
Next method :
Edit your /etc/resolv.conf and change your nameserver to change request
#nameserver 8.8.8.8
nameserver <your best dns server to change requests>

Scrapy always running same command from command prompt

I'm trying to learn Scrapy on BashOnUbunty on Windows 10. I created a spider (yelprest) using the genspider command, and then directly created another spider (quotes_spider) by creating the spider file (followed the official tutorial https://doc.scrapy.org/en/latest/intro/tutorial.html).
The first spider is not yet tested, but I tried to go through the tutorial with second spider, and when I try to run, I'm getting an error which points to the first spider. Also, when I try to run any other scrapy command like version, I'm getting the same error as above. Below is the error:
(BashEnv) root > scrapy version
Traceback (most recent call last):
File "/mnt/s/BashEnv/bin/scrapy", line 11, in <module>
sys.exit(execute())
File "/mnt/s/BashEnv/local/lib/python2.7/site-packages/scrapy/cmdline.py", line 148, in execute
cmd.crawler_process = CrawlerProcess(settings)
File "/mnt/s/BashEnv/local/lib/python2.7/site-packages/scrapy/crawler.py", line 243, in __init__
super(CrawlerProcess, self).__init__(settings)
File "/mnt/s/BashEnv/local/lib/python2.7/site-packages/scrapy/crawler.py", line 134, in __init__
self.spider_loader = _get_spider_loader(settings)
File "/mnt/s/BashEnv/local/lib/python2.7/site-packages/scrapy/crawler.py", line 330, in _get_spider_loader
return loader_cls.from_settings(settings.frozencopy())
File "/mnt/s/BashEnv/local/lib/python2.7/site-packages/scrapy/spiderloader.py", line 61, in from_settings
return cls(settings)
File "/mnt/s/BashEnv/local/lib/python2.7/site-packages/scrapy/spiderloader.py", line 25, in __init__
self._load_all_spiders()
File "/mnt/s/BashEnv/local/lib/python2.7/site-packages/scrapy/spiderloader.py", line 47, in _load_all_spiders
for module in walk_modules(name):
File "/mnt/s/BashEnv/local/lib/python2.7/site-packages/scrapy/utils/misc.py", line 71, in walk_modules
submod = import_module(fullpath)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/mnt/s/BashEnv/Scrapy/Scrapy/spiders/yelprest.py", line 14
rules = (
^
IndentationError: unexpected indent
(BashEnv) root >
I'm not understanding why I am getting the same error for any command I give.
There is some error in your yelprest.py file (at line 14 or before): it is not valid Python. Fix this error and everything will work. Make sure your file is correctly indented and do not mix spaces and tabs.
Edit:
To make sure the error is in this file, just delete it. If everything works without this file, the error must be there!
Update:
Your question does not state it clearly, but by your comment your question is "why does Scrapy load my spider code for every command?". And the answer is: because Scrapy was made to do it. Some commands can be run only inside a project, like check or crawl. And some commands may be run anywhere, like startproject. But inside a Scrapy project, ANY command will load ALL your code. Scrapy was made this way.
For example, I have a project named crawler (I know, very descriptive!):
$ cd ~
$ scrapy version
Scrapy 1.4.0
$ cd crawler/
$ scrapy version
2017-10-31 14:47:42 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: crawler)
2017-10-31 14:47:42 [scrapy.utils.log] INFO: Overridden settings: {...}
Scrapy 1.4.0