"service hadoop-yarn-resourcemanager start" command not working? - hadoop-yarn

Redirecting to /bin/systemctl start hadoop-yarn-resourcemanager.service
Failed to start hadoop-yarn-resourcemanager.service: Unit not found.
cluster: Cloudera 5.14

may be Due to permission issue it is causing this issue.
sudo chown hdfs:hadoop -R /hdfs/data
sudo chmod 777 -R /hdfs/data
hadoop namenode -format
Run these commands from CLI and try
it again.

Related

Docker entrypoint initdb PERMISSION DENIED

I am getting the following error when I run docker-compose up:
Thanks a lot for your help
I resolved this problem by adding this to the Dockerfile after it copies the scripts to docker-entrypoint-initdb.d
RUN chown -R mysql:mysql /docker-entrypoint-initdb.d/
Example Dockerfile:
FROM mysql:latest
ENV MYSQL_DATABASE NAME_DATABASE
ENV MYSQL_ROOT_PASSWORD ***********
COPY ./sql-scripts/ /docker-entrypoint-initdb.d/
RUN chown -R mysql:mysql /docker-entrypoint-initdb.d/
EXPOSE 3306
CMD ["mysqld", "--character-set-server=utf8mb4", "--collation-server=utf8mb4_unicode_ci"]
The next step is to build the image:
docker build -t image-db:latest .
The next step is to create the container
docker run -d -p 3306:3306 --name container-db image-db:latest
You should not override the postgres image entrypoint. It is designed to look for .sql files in /docker-entrypoint-initdb.d/ directory (See line in script).
You should just mount your .sql files into /docker-entrypoint-initdb.d/ and it should be processed on startup (only if database does not already exist)
I had the same issue, however, my problem occurred due to Linux user. I am using root as a runner so the problem happened because the mounting volume in the local machine did not have permissions. in this regard, I used chmod -R 777 scripts and it worked fine. Technically, you need to set permissions for both local machine and your container.

Laravel Continuous Integration with Gitlab-runner in offline environment (CentOS 7)

I'm developing a website on a totally offline environment. also, I use gitlab runner for CI and the host is CentOS 7.
the problem is that gitlab runner uses gitlab-runner user on centos for deploying laravel application and apache uses apache user for running laravel.
I got Permission denied error on apache til I changed ownership of files. after that I get this error on apache log:
Uncaught UnexpectedValueException: The stream or file "storage/logs/laravel.log" could not be opened: failed to open stream: Permission denied
it seems that some vendor libraries like monolog want to write error or debug logs onto storage/logs/laravel.log but it gets permission denied. :(
.gitlab-ci.yml
stages:
- build
- test
- deploy
buildBash:
stage: build
script:
- bash build.sh
testBash:
stage: test
script:
- bash test.sh
deployBash:
stage: deploy
script:
- sudo bash deploy.sh
build.sh
#!/bin/bash
set -xe
# creating env file from production file
cp .env.production .env
# initializing laravel
php artisan key:generate
php artisan config:cache
# database migration
php artisan migrate --force
deploy.sh
#!/bin/bash
PWD=$(pwd)'/public'
STG=$(pwd)'/storage'
ln -s $PWD /var/www/html/public
chown apache.apache -R /var/www/html/public
chmod -R 755 /var/www/html/public
chmod -R 775 $STG
Am I using gitlab runner correct? how can I fix the permission denied error?
SELinux
I found the problem and it was selinux, like always it was selinux and I ignored it at the begining
What's the problem:
you can see selinux context on files with ls -lZ command, by default all files on www are httpd_sys_content_t, the problem is that selinux just allow apache to read these files. you should change storage and bootstrap/cache context so it can be writable.
there are 4 apache context type:
httpd_sys_content_t: read-only directories and files
httpd_sys_rw_content_t: readable and writable directories and files used by Apache
httpd_log_t: used by Apache for log files and directories
httpd_cache_t: used by Apache for cache files and directories
What to do:
first of all install policycoreutils-python for better commands
yum install -y policycoreutils-python
after installing policycoreutils-python the semanage command is available, so you can change file context like this:
semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/laravel/storage(/.*)?"
semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/laravel/bootstrap/cache(/.*)?"
don't forget to commit the changes by this command:
restorecon -Rv /var/www/html/laravel/storage
restorecon -Rv /var/www/html/laravel/bootstrap/cache
the problem is solved :)
ref: http://www.serverlab.ca/tutorials/linux/web-servers-linux/configuring-selinux-policies-for-apache-web-servers/

Apache Tomcat 8 not starting within a docker container

I am experimenting with Docker and am very new to it. I am struck at a point for a long time and am not getting a way through and hence came up with this question here...
Problem Statement:
I am trying to create an image from a docker file containing Apache and lynx installation. Once done I am trying to access tomcat on 8080 of the container which is in turn forwarded to the 8082 of the host. But when running the image I never get tomcat started in the container.
The Docker file
FROM ubuntu:16.10
#Install Lynx
Run apt-get update
Run apt-get install -y lynx
#Install Curl
Run apt-get install -y curl
#Install tools: jdk
Run apt-get update
Run apt-get install -y openjdk-8-jdk wget
#Install apache tomcat
Run groupadd tomcat
Run useradd -s /bin/false -g tomcat -d /opt/tomcat tomcat
Run cd /tmp
Run curl -O http://apache.mirrors.ionfish.org/tomcat/tomcat- 8/v8.5.12/bin/apache-tomcat-8.5.12.tar.gz
Run mkdir /opt/tomcat
Run tar xzvf apache-tomcat-8*tar.gz -C /opt/tomcat --strip-components=1
Run cd /opt/tomcat
Run chgrp -R tomcat /opt/tomcat
Run chmod -R g+r /opt/tomcat/conf
Run chmod g+x /opt/tomcat/conf
Run chown -R tomcat /opt/tomcat/webapps /opt/tomcat/work /opt/tomcat/temp opt/tomcat/logs
Run cd /opt/tomcat/bin
Expose 8080
CMD /opt/tomcat/bin/catalina.sh run && tail -f /opt/tomcat/logs/catalina.out
When the image is built I tried running the container by the two below methods
docker run -d -p 8082:8080 imageid tail -f /dev/null
While using the above, container is running but tomcat is not started inside the container and hence not accessible from localhost:8082. Also I do not see anything if I perform docker logs longcontainerid
docker run -d -p 8082:8080 imageid /path/to/catalina.sh start tail -f /dev/null
I see tomcat started when I do docker logs longconatainrid
While using the above the container is started and stopped immediately and is not running as I can see from docker ps and hence again not accessible from localhost:8082.
Can anyone please tell me where I am going wrong?
P.s. I searched a lot on the internet but could not get the thing right. Might be there is some concept that i am not getting clearly.
Looking at the docker run command documentation, the doc states that any command passed to the run will override the original CMD in your Dockerfile:
As the operator (the person running a container from the image), you can override that CMD instruction just by specifying a new COMMAND
1/ Then when you run:
docker run -d -p 8082:8080 imageid tail -f /dev/null
The container is run with COMMAND tail -f /dev/null, the original command starting tomcat is overridden.
To resolve your problem, try to run:
docker run -d -p 8082:8080 imageid
and
docker log -f containerId
To see if tomcat is correctly started.
2/ You should not use the start argument with catalina.sh. Have a look at this official tomcat Dokerfile, the team uses :
CMD ["catalina.sh", "run"]
to start tomcat (when you use start, docker ends container at the end of the shell script and tomcat will start but not maintain a running process).
3/ Finally, why don't you use tomcat official image to build your container? You could just use the :
FROM tomcat:latest
directive at the beginning of your Dockerfile, and add you required elements (new files, webapps war, settings) to the docker image.

Failed to start redis.service: Unit redis-server.service is masked

I Installed Redis Server on ubuntu 16.04. but when I try to start the redis service using
$ sudo systemctl start redis
I receive message:
Failed to start redis.service: Unit redis-server.service is masked.
I don't have any idea about this error.
I found the solution. I think it will help for others
| systemctl unmask servicename
$ sudo systemctl unmask redis-server.service
if redis.service file is missed in directory path /etc/systemd/system/redis.service so we have to create file into this directory using command like:
sudo nano /etc/systemd/system/redis.service and write following things
[Unit]
Description=Redis In-Memory Data Store
After=network.target
[Service]
User=redis
Group=redis
ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf
ExecStop=/usr/local/bin/redis-cli shutdown
Restart=always
[Install]
WantedBy=multi-user.target
Here
Unit is description and defining the network behavior,
Service is to specify the service's behavior and
Install define the systemd target that the service should attach to if enabled.
Then create redis user group and directories using
sudo adduser --system --group --no-create-home redis
sudo mkdir /var/lib/redis (Create directory)
sudo chown redis:redis /var/lib/redis (Change owner permission)
sudo chmod 770 /var/lib/redis
run the service sudo systemctl start redis
It will be work fine and check the status using
sudo systemctl status redis
1.sudo systemctl unmask redis-server.service
if redis.service file is missed in directory path
/etc/systemd/system/redis.service ,so we have to create file into this directory using command like:
sudo nano /etc/systemd/system/redis.service and write following things
[Unit]
Description=Redis In-Memory Data Store
After=network.target
[Service]
User=redis
Group=redis
ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf
ExecStop=/usr/local/bin/redis-cli shutdown
Restart=always
[Install]
WantedBy=multi-user.target
Then create redis user group and directories using
sudo adduser --system --group --no-create-home redis
sudo mkdir /var/lib/redis (Create directory)
sudo chown redis:redis /var/lib/redis (Change owner permission)
sudo chmod 770 /var/lib/redis
run the service sudo systemctl start redis It will be work fine and check the status using
sudo systemctl status redis
For Ubuntu users run
sudo systemctl restart redis-server
you must run code
sudo systemctl daemon-reload

Error with rabbit-mq server

I am trying to setup OpenStack on Ubuntu 12.04 using devstack. Now, the error I am getting is:
Setting up rabbitmq-server (2.7.1-0ubuntu4) ...
Starting rabbitmq-server: FAILED - check /var/log/rabbitmq/startup_{log, _err}
rabbitmq-server.
invoke-rc.d: initscript rabbitmq-server, action "start" failed.
dpkg: error processing rabbitmq-server (--configure):
subprocess installed post-installation script returned error exit status 1
No apport report written because MaxReports is reached already
Errors were encountered while processing:
rabbitmq-server
E: Sub-process /usr/bin/dpkg returned an error code (1)
++ err_trap
++ local r=100
++ set +o xtrace
stack.sh failed
Any idea why am I getting this error?
I had this issue twice, when either hostname or ip address in the hosts file didn't match.
Therefore, check that you provide the correct ip address and hostname in the /etc/hosts file
Run sudo cat /etc/hostname to see your hostname
Output:
yoursite
Run sudo nano /etc/hosts
File contains:
127.0.0.1 yoursite
As you see from cat /etc/hostname, hostname is the same as in the /etc/hosts:
Run sudo rabbitmq-server start to start the rabbitmq-server
Try deleting the folder /var/lib/rabbitmq and re-running ./stack.sh
If that doesn't work either, run the following after stach.sh fails:
chown -R rabbitmq:rabbitmq /var/lib/rabbitmq
chown -R rabbitmq:rabbitmq /var/log/rabbitmq
service rabbitmq-server restart
and check the status of rabbitmq using "rabbitmqctl status"
Similar thing happen to me. Rabbit depends on being able to resolve a hostname, run this:
echo "127.0.0.1 $(hostname -s)" | sudo tee -a /etc/hosts
This way works for me.
First go to
sudo vim /etc/hosts
and set
127.0.0.1 <hostname>
then open firewall
sudo rabbitmq-plugins enable rabbitmq_management
sudo service rabbitmq-server restart
For a clean environment, this will not happen. You must run devstack for several times, and one of them failed but you didn't get it cleaned.
run command pf -ef | grep rabbitmq, kill all rabbitmq processes. then it would be fine to run ./stack.sh
it is highly recommended to run ./unstack.sh && ./clean.sh before ./stack.sh
Just to be sure, take a look to your local network
ip add
If there's no lo network, then you should enable it:
ifconfig lo up
Then restart the server again and let's see if it works again now
systemctl start rabbitmq-server
I had the same problem though my /etc/hosts and DNS were OK. I suspect that SystemV init script was started too early when the network was not ready yet. I rewrote the startup script to systemd on CentOS 7.8 and it seems to work well now.
[Unit]
Description=RabbitMQ
Wants=network-online.target
After=network-online.target
[Service]
Type=simple
RuntimeDirectory=rabbitmq
PrivateTmp=true
Restart=on-failure
RestartSec=10
WorkingDirectory=/opt/data/rabbitmq/
User=rabbitmq
Group=rabbitmq
ExecStart=/opt/app/rabbitmq/default/sbin/rabbitmq-server
ExecStop=/opt/app/rabbitmq/default/sbin/rabbitmqctl stop
ExecStop=/bin/sh -c "while ps -p $MAINPID >/dev/null 2>&1; do sleep 1; done"
StandardOutput=journal
StandardError=inherit
[Install]
WantedBy=multi-user.target