I need help.
I've downloaded Apache Flume and installed outside Hadoop, just wanna try netcat logging through console.
I used 1.6.0 version.
Here's my conf https://gist.github.com/ans-4175/297e2b4fc0a67d826b4b
Here's how I started it
bin/flume-ng agent -c conf -f conf/netcat.conf Dflume.root.logger=DEBUG,console -n Agent1
But it's stuck after only printed these output
Info: Sourcing environment configuration script /root/apache-flume/conf/flume-env.sh
Info: Including Hive libraries found via () for Hive access
+ exec /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -Xms100m -Xmx2000m -cp '/root/apache-flume/conf:/root/apache-flume/lib/*:/root/apache-flume/lib/*:/lib/*' -Djava.library.path= org.apache.flume.node.Application -f conf/netcat.conf Dflume.root.logger=DEBUG,console -n Agent1
Any suggestions for simple start and install?
Thanks
dumb me, it should be
bin/flume-ng agent -c conf -f conf/netcat.conf -Dflume.root.logger=DEBUG,console -n Agent1
Related
Good day,
I have just uploaded Symfony 3.4 project (PHP 7.2) to Centos server and my application needs to be connected to RabbitMQ. I want to do that in Centos server rabbitmq is constantly consuming messages. I know how to consume those messages temporarily by running this command:
bin/console rabbitmq:consumer messaging . But how could permanently I consume the messages on server? I tried to google but didn't find any useful information
In my application I've installed:
"php-amqplib/php-amqplib": "*",
"php-amqplib/rabbitmq-bundle": "*"
UPDATE:
I achieved my desired situation with the following command:
nohup bin/console rabbitmq:consumer <your-consumer> &
idk if there's an "official" way of doing it, but as with anything in Linux, you could just write a little daemon to do it, a minimum example would be to add this to your crontab -e
#reboot /bin/bash /project/folder/cronjob_starter.sh
with cronjob_starter.sh containing
#!/bin/bash
if [[ $(screen -ls | grep rabbitmq_daemon) ]]
then
echo "rabbitmq_daemon already running!"
/bin/true
else
# echo " rabbitmq_daemon not running!"
screen -S rabbitmq_daemon -dm
# workaround for https://savannah.gnu.org/bugs/index.php?54164
sleep 1
screen -S rabbitmq_daemon -X stuff "cd /project/folder; bin/console rabbitmq:consumer messaging^M"
fi
then you can inspect your daemon with screen -xS rabbitmq_daemon , or with the Screenie application (honestly idk how to "properly" install Screenie on CentOS, i just run curl https://gist.githubusercontent.com/divinity76/1a583968c997869b27a5ee2c1ed24259/raw/76453e61a92676386589fbb3f4ef0225ac98fb19/screenie.b64 | base64 -d | sudo tee /usr/local/bin/screenie ; sudo chmod 0555 /usr/local/bin/screenie; )
if there's an "official" way of doing it tho, you should probably do it the official way instead, i don't know anything about that unfortunately.
I'm developing a website on a totally offline environment. also, I use gitlab runner for CI and the host is CentOS 7.
the problem is that gitlab runner uses gitlab-runner user on centos for deploying laravel application and apache uses apache user for running laravel.
I got Permission denied error on apache til I changed ownership of files. after that I get this error on apache log:
Uncaught UnexpectedValueException: The stream or file "storage/logs/laravel.log" could not be opened: failed to open stream: Permission denied
it seems that some vendor libraries like monolog want to write error or debug logs onto storage/logs/laravel.log but it gets permission denied. :(
.gitlab-ci.yml
stages:
- build
- test
- deploy
buildBash:
stage: build
script:
- bash build.sh
testBash:
stage: test
script:
- bash test.sh
deployBash:
stage: deploy
script:
- sudo bash deploy.sh
build.sh
#!/bin/bash
set -xe
# creating env file from production file
cp .env.production .env
# initializing laravel
php artisan key:generate
php artisan config:cache
# database migration
php artisan migrate --force
deploy.sh
#!/bin/bash
PWD=$(pwd)'/public'
STG=$(pwd)'/storage'
ln -s $PWD /var/www/html/public
chown apache.apache -R /var/www/html/public
chmod -R 755 /var/www/html/public
chmod -R 775 $STG
Am I using gitlab runner correct? how can I fix the permission denied error?
SELinux
I found the problem and it was selinux, like always it was selinux and I ignored it at the begining
What's the problem:
you can see selinux context on files with ls -lZ command, by default all files on www are httpd_sys_content_t, the problem is that selinux just allow apache to read these files. you should change storage and bootstrap/cache context so it can be writable.
there are 4 apache context type:
httpd_sys_content_t: read-only directories and files
httpd_sys_rw_content_t: readable and writable directories and files used by Apache
httpd_log_t: used by Apache for log files and directories
httpd_cache_t: used by Apache for cache files and directories
What to do:
first of all install policycoreutils-python for better commands
yum install -y policycoreutils-python
after installing policycoreutils-python the semanage command is available, so you can change file context like this:
semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/laravel/storage(/.*)?"
semanage fcontext -a -t httpd_sys_rw_content_t "/var/www/html/laravel/bootstrap/cache(/.*)?"
don't forget to commit the changes by this command:
restorecon -Rv /var/www/html/laravel/storage
restorecon -Rv /var/www/html/laravel/bootstrap/cache
the problem is solved :)
ref: http://www.serverlab.ca/tutorials/linux/web-servers-linux/configuring-selinux-policies-for-apache-web-servers/
I have tried to work with mahout using the link:
http://girlincomputerscience.blogspot.in/2010/11/apache-mahout.html
When i execute the command
anand#ubuntu:~/Downloads/mahout-distribution-0.9$ bin/mahout recommenditembased --input mydata.dat --usersFile user.dat --numRecommendations 2 --output output/ --similarityClassname SIMILARITY_PEARSON_CORRELATION
it shows:
hadoop binary is not in PATH,HADOOP_HOME/bin,HADOOP_PREFIX/bin, running locally
Error occurred during initialization of VM
Could not reserve enough space for object heap
Could not create the Java virtual machine.
What may be the possible reasons for the errors?
You need to setup Apache Hadoop first ( also described here ):
$ http://mirror.metrocast.net/apache/hadoop/common/hadoop-1.2.1/hadoop-1.2.1-bin.tar.gz
$ tar zxf hadoop-1.2.1-bin.tar.gz
$ cd hadoop-1.2.1
$ export PATH=`pwd`/bin:$PATH
Then try to setup Apache Mahout
Apache Mahout setup
$ wget -c http://archive.apache.org/dist/mahout/0.9/mahout-distribution-0.9.tar.gz
$ tar zxf mahout-distribution-0.9.tar.gz
$ cd mahout-distribution-0.9
Setup env variables:
export HADOOP_HOME=/path/to/hadoop-1.2.1
export PATH=$HADOOP_HOME/bin:$PATH
export MAHOUT_HOME=/path/to/mahout-distribution-0.9
export PATH=$MAHOUT_HOME/bin:$PATH
Now your Mahout installation should work fine ( read here for more ).
I'm trying to run activemq on my ubuntu virtual machine but have constantly been running into issues getting it to start up. I've tried downloading the binary and source with no luck. Currently I have downloaded the source, run "mvn clean install -Dmaven.test.skip=true" and mvn reported successful installation. I then hunted around in my .m2 folder found apache-activemq-5.5.1-bin.tar.gz and extracted it to my home/USERNAME dir and attempted to run "bash bin/activemq start" only to receive the following error.
INFO: Loading '/etc/default/activemq'
INFO: Using java '/usr/bin/java'
INFO: Starting - inspect logfiles specified in logging.properties
and log4j.properties
to get details
bin/activemq: line 370: /usr/bin/java -Xms256M -Xmx256M -Dorg.apache.activemq.UseDedicatedTaskRunner=true
-Djava.util.logging.config.file=logging.properties
-Dcom.sun.management.jmxremote
-Dactivemq.classpath="/home/jacob/activeMq1/apache-activemq-5.5.1/conf;"
-Dactivemq.home="/home/jacob/activeMq1/apache-activemq-5.5.1"
-Dactivemq.base="/home/jacob/activeMq1/apache-activemq-5.5.1"
-jar "/home/jacob/activeMq1/apache-activemq-5.5.1/bin/run.jar" start >/dev/null 2>&1 &
RET="$?"; APID="$!";
echo $APID > /home/jacob/activeMq1/apache-activemq-5.5.1/data/activemq.pid;
echo "INFO: pidfile created : '/home/jacob/activeMq1/apache-activemq-5.5.1/data/activemq.pid' (pid '$APID')";
exit $RET: No such file or directory
Has any one run into this type of error before?
Looks like I'm answering one of my questions again, but maybe this will help someone in the future.
steps.
I ended up getting activemq to work by creating a configuration file via running the command "./bin/activemq setup newConfig" (exclude the quotes)
I then replaced the current config file "activemq" which was located at etc/default/. (I made a backup of the original activemq file before overwriting it with newConfig).
Run "./bin/activemq start" which will create a PID file.
After the file is created re-run "./bin/activemq start" to finally start up the broker.
You can then test the install by navigating to "http://localhost:8161/admin/" or by doing a "netstat -an | grep 61616" if you kept the default ports etc.
I install activemq 5.13 on Debian, download and unzip in /opt, then I go to /opt/apache-activemq-5.13.1/run "./bin/activemq start", then appear this error:
xx#debian:/opt/apache-activemq-5.13.1$ ./bin/activemq start
INFO: Loading '/etc/default/activemq'
INFO: Using java '/usr/bin/java'
INFO: Starting - inspect logfiles specified in logging.properties and log4j.properties to get details
./bin/activemq: 330: ./bin/activemq: "/usr/bin/java" -Djava.util.logging.config.file=logging.properties -Djava.security.auth.login.config=/opt/apache-activemq-5.13.1//conf/login.config -Djava.awt.headless=true -Djava.io.tmpdir="/opt/apache-activemq-5.13.1//tmp" -Dactivemq.classpath="/opt/apache-activemq-5.13.1//conf:/opt/apache-activemq-5.13.1//../lib/:" -Dactivemq.home="/opt/apache-activemq-5.13.1/" -Dactivemq.base="/opt/apache-activemq-5.13.1/" -Dactivemq.conf="/opt/apache-activemq-5.13.1//conf" -Dactivemq.data="/opt/apache-activemq-5.13.1//data" -jar "/opt/apache-activemq-5.13.1//bin/activemq.jar" start >/dev/null 2>&1 &
RET="$?"; APID="$!";
echo $APID > /opt/apache-activemq-5.13.1//data/activemq.pid;
echo "INFO: pidfile created : '/opt/apache-activemq-5.13.1//data/activemq.pid' (pid '$APID')";exit $RET: not found
What I did is to check the Debian version using "uname -a":
Linux debian 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u3 (2016-01-17) x86_64 GNU/Linux
I found my debian is 64 bit system. so I run
./bin/linux-x86-64/activemq start
It shows:
Starting ActiveMQ Broker...
Then I can access the site:http://localhost:8161/admin/ with username "admin" and password "admin"
with Ubuntu 14.04, I had to create a link in /etc/activemq/instances-enabled
sudo ln -s ../instances-available/main/
similar to apache2 setup
then started the server with /etc/init.d/activemq start
sudo is necessary.
bin$ sudo ./activemq start
bin$ sudo ./activemq status
INFO: Loading '/opt/runtime/apache-activemq-5.11.1/bin/env' INFO: Using java '/usr/bin/java' ActiveMQ is running (pid '29887')
I need to rebuild an apache server, but the original source is no longer available. Is there any way ( command line switch to httpd? ) to get the build options which were originally used?
Try -V which "Print the version and build parameters of httpd, and then exit."
httpd -V
Also, you can see the options for httpd via:
httpd -h
I found previous configure options in the build directory of apache root.
I'm a Centos 5/6 user.
Apache ver. is 2.2.27.
apachedir/build/config.nice
#! /bin/sh
#
# Created by configure
"./configure" \
"--prefix=/usr/local/apache2" \
"--enable-so" \
"--enable-mods-shared=most" \
"--enable-ssl" \
"--with-mpm=worker" \
"--enable-cgi" \
"$#"
I re-compiled apache 2.4.3 recently and change the MPM from worker to prefork, what you have to do if you still keep your original compiled directory without ran "make distclean" (if you ran "make clean" it still OK). You can use the SAME configure option to re-configure by exec ./config.status or you can find and copy './configure' from ./config.status (yes, all the original options that you used to run configure still there).
Here is part of my config.status...
if $ac_cs_silent; then
exec 6>/dev/null
ac_configure_extra_args="$ac_configure_extra_args --silent"
fi
if $ac_cs_recheck; then
set X /bin/sh **'./configure' '--enable-file-cache' '--enable-cache' '--enable-disk-cache' '--enable-mem-cache' '--enable-deflate' '--enable-expires' '--enable-headers' '--enable-usertrack' '--enable-cgi' '--enable-vhost-alias' '--enable-rewrite' '--enable-so' '--with-apr=/usr/local/apache/' '--with-apr-util=/usr/local/apache/' '--prefix=/usr/local/apache' '--with-mpm=worker' '--with-mysql=/var/lib/mysql' '--with-mysql-sock=/var/run/mysqld/mysqld.sock' '--enable-mods-shared=most' '--enable-ssl' 'CFLAGS=-Wall -O3 -ffast-math -frename-registers -mtune=corei7-avx' '--enable-modules=all' '--enable-proxy' '--enable-proxy-fcgi'** $ac_configure_extra_args --no-create --no-recursion
shift
$as_echo "running CONFIG_SHELL=/bin/sh $*" >&6
CONFIG_SHELL='/bin/sh'
export CONFIG_SHELL
exec "$#"
fi