I have installed the fluentd agent in my VM where postgresql is installed. Now I am looking for the td-agent.conf file which could help me to ship the pg logs and /var/log/messages to kibana. I am very new to ELF and hence looking for a sample file which I could use and add more logs to forwarding. Please help.
Thank you .
Related
I'm trying to install HortanWorks Ambari-Server in a local setup and I'm running into a problem with the Kafka client installation. Following error pops up when I try to setup the cluster.
resource_management.core.exceptions.ExecutionFailed: Execution of
'ambari-python-wrap /usr/bin/hdp-select set zookeeper-client 2.6.3.0-235'
returned 1. symlink target /usr/hdp/current/zookeeper-client for zookeeper
already exists and it is not a symlink.
How to correct this issue? I've looked into the HDP problem discussion as well [1] but the answers that are given there seems to not work for me. Can someone please elaborate what needs to be done to correct this issue ?
HDP discussion [1]
Error is self explanatory and it indicates directory or file /usr/hdp/current/zookeeper-client already exist. Hence to resolve issue you need to delete directory or file /usr/hdp/current/zookeeper-client and then create correct soft link.
mv /usr/hdp/current/zookeeper-client /usr/hdp/current/zookeeper-client_back
ln -s /usr/hdp/2.6.3.0-235/zookeeper /usr/hdp/current/zookeeper-client
Problem is the plain vanila apache is working, the moment I say sslEngine ON
it gives error, I am not able to find where any ssl setup is?
it does not create any log as well..
How this ssl setup is done? will copying a mod_ssl.so file from outside will work?
Apache version : 2.4.29
Os is RHEL 6.7
If this Red Hat documentation is not wrong the RHEL 6.x releases include Apache/2.2. Since you have Apache/2.4 I'd say you probably have an unofficial binary. Such binaries could of course have been installed in any conceivable way so this answer can easily be wrong. Yet making the hopefully safe assumption that you're still using the package manager and you just switched to a third-party repository you can try this:
yum install mod_ssl
Then tweak /etc/httpd/conf.d/ssl.conf to your liking and restart the service:
service httpd graceful
After a lot of research, I found below command which can help you show the actual issue while restarting. when I ran this it told me that .csr which I was using were wrong. I could not get this when I normally started the Apache but this command helped.
replace below with location where apache is installed and conf file is located, this will print the error ( if any).
/xxx/app/xxx/apache22/0/bin/apachectl_worker -DSSL -k start -f /xxx/app/xxx/apache22/0/conf/xxx.conf
I just tried to install Apache Zookeeper standalone in Ubuntu OS. I have installed Java environment and install Zookeeper 3.4.6.
However, when I typed JPS here are only I got.
following are the configuration for bashrc and zoo.cfg file:
[terminal~] vim .bashrc
[terminal~]vim /usr/local/zookeeper/conf/zoo.cfg
Please anyone help me. I wasted 2 days for only Zookeeper. It was really frustrated.
I have a fresh installed Ubuntu machine, and this is what I did to get ZooKeeper working as a standalone program (I assume you mean that you didn't install it with the package manager.)
Download ZooKeeper tar. (I used 3.4.8)
Extract the folder zookeeper-3.4.8 somewhere. (I placed it on my desktop for now)
Copy .../zookeeper-3.4.8/conf/zoo_sample.cfg to /zookeeper-3.4.8/conf/zoo.cfg
And change the dataDir=... line to whatever you want. (I made a data dir inside the zookeeper-3.4.8 folder)
Now you can run zookeeper by executing the script ../zookeeper-3.4.8/bin/zkServer.sh start
foo#bar:~$ /home/foo/Desktop/zookeeper-3.4.8/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/foo/Desktop/zookeeper-3.4.8/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
foo#bar:~$ /home/foo/Desktop/zookeeper-3.4.8/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /home/foo/Desktop/zookeeper-3.4.8/bin/../conf/zoo.cfg
Mode: standalone
Relatively new to Apache OOZIE and did an installation on Ubuntu 14.04, Hadoop 2.6.0, JDK 1.8. I was able to install oozie and the web console is visible at the 11000 port of my server.
Now while i copied the examples bundled with oozie and tried to run them i am running into an error which says no sharedlib exists.
Installed the sharedlib as below-
bin/oozie-setup.sh sharelib create -fs hdfs://localhost:54310
(my namenode is running on localhost 54310 and JT on localhost 54311)
hadoop fs -ls /user/hduser/share/lib is showing shared library created as per the oozie-site.xml file. However when i check the shared library using the command -
oozie admin -oozie http://localhost:11000/oozie -shareliblist the list is blank and also jobs are failing for the same reason.
Any clues on how should i approach this problem?
Thanks.
The sharelib create command looks fine.
If you havent done so already copy the core-site.xml from your hadoop installation folder into $OOZIE_HOME/conf/hadoop-conf/.
There might already be a "placeholder" core-site.xml in the hadoop-conf folder, delete or rename that one. Oozie doesnt get its hadoop configuration directly from your hadoop install (like hive for example) but from the core-site.xml you place in that hadoop-conf folder.
Okay i got a solution for this.
So when i was trying to create the sharedlib directory it was doing on HDFS but while running the job local path was being refereed. So i extracted the oozie-sharedlib tar.gz file in my local /user/hduser/share/lib directory and its working now.
But did not get the reason so its still an open question.
I have encountered the same issue and it turned out that
oozie was not able to communicate with hdfs, as it was not able to find the location for core-site.xml or any other hadoop configuration which has to be declared inside oozie-site.xml.
Corresponding property in oozie-site.xml is oozie.service.HadoopAccessorService.hadoop.configurations
this property was defined wrongly in my case.
changed it to point to where my Hadoop configuration xmls are present and then it started communicating with hdfs and hence was able to locate the sharelib on hdfs
I want to clear my Apache log without stopping apache or anything like that, but I have a problem while apache's running I can not edit the active log file.
I attempted some solutions but all is not good, or it just does not work - for example:
cat /dev/null > /path/to/log.log
It works but it kills the CPU.
Now I want to know has anybody got a good solution for clearing log files or creating a new log file with a new name every N hours which doesn't kill the CPU,RAM,etc ...
What can I do? Please give me a solution :((
Kind regards.
Install and set up logrotate, assuming you are on a debian or ubuntu distribution do this :
$ sudu apt-get install logrotate
This is an "industry standard" solution for what you want to achieve.
Recommended reading:
http://articles.slicehost.com/2010/6/30/understanding-logrotate-on-ubuntu-part-1
http://articles.slicehost.com/2010/6/30/understanding-logrotate-on-centos-part-1
https://unix.stackexchange.com/questions/66292/learning-about-general-logging-logrotation-on-linux