I'd like to know how to have td-agent read apache log files on Amazon linux2.
It seems td-agent doesn't have permission to read log file but I cannot fix this.
I installed amazon linux2 into virtualbox using vagrant.
Ref.
https://app.vagrantup.com/aibax/boxes/amazonlinux2
https://docs.fluentd.org/v1.0/articles/install-by-rpm#systemd
versions
virtual box: 5.2.16 r123759
vagrant: Vagrant 2.1.2
1 install apache and td-agent using ansible.
# ansible/playbook.yml
# install apache
- name: install httpd (apache)
become: yes
yum:
name: httpd
state: present
# install td-agent
- name: install td-agent
raw: "curl -L https://toolbelt.treasuredata.com/sh/install-amazon2- td-agent3.sh | sh"
- name: upload etc/td-agent/td-agent.conf
become: yes
copy: src=../etc/td-agent/td-agent.conf
dest=/etc/td-agent/td-agent.conf
directory_mode=yes
--
# /etc/td-agent/td-agent.conf
<source>
type tail
format apache
path /var/log/httpd/access_log
tag apache.access_log
pos_file /var/log/td-agent/access_log.pos
</source>
<source>
type tail
format apache
path /var/log/httpd/error_log
tag apache.error_log
pos_file /var/log/td-agent/error_log.pos
</source>
<match apache.**>
type forward
<server>
name jobq01
host 52.68.22.100
port 24224
</server>
</match>
run apache
$ sudo systemctl start httpd
confirm apache works from chrome.
run td-agent and confirm
--
$ sudo systemctl start td-agent.service
$ sudo systemctl status td-agent.service
● td-agent.service - td-agent: Fluentd based data collector for Treasure Data
Loaded: loaded (/usr/lib/systemd/system/td-agent.service; disabled; vendor preset: disabled)
Active: active (running) since 月 2018-08-06 05:24:22 UTC; 1min 1s ago
Docs: https://docs.treasuredata.com/articles/td-agent
Process: 5532 ExecStart=/opt/td-agent/embedded/bin/fluentd --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td-agent.pid $TD_AGENT_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 5537 (fluentd)
CGroup: /system.slice/td-agent.service
└─5537 /opt/td-agent/embedded/bin/ruby /opt/td-agent/embedded/bin/fluentd --log /var/log/td-agent/td-agent.log --daemon /var/run/td-agent/td-agent.pid
8月 06 05:24:21 localhost systemd[1]: Starting td-agent: Fluentd based data collector for Treasure Data...
8月 06 05:24:22 localhost systemd[1]: Started td-agent: Fluentd based data collector for Treasure Data.
I fixed this problem myself
User and Group in the file below should be modified to root.
/lib/systemd/system/td-agent.service
Related
I am trying to create a web server on my ubuntu 18.04 so i installed Apache2
but i can't start it.
Here's what appeared when i run the systemctl status apache2.service command
apache2.service - The Apache HTTP Server
Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
Drop-In: /lib/systemd/system/apache2.service.d
└─apache2-systemd.conf
Active: failed (Result: exit-code) since Sat 2020-02-22 13:58:09 CET; 34s ago
Process: 2791 ExecStart=/usr/sbin/apachectl start (code=exited, status=1/FAILURE)
Feb 22 13:58:09 moemen apachectl[2791]: AH00558: apache2: Could not reliably determine the server's
Feb 22 13:58:09 moemen apachectl[2791]: (98)Address already in use: AH00072: make_sock: could not b
Feb 22 13:58:09 moemen apachectl[2791]: (98)Address already in use: AH00072: make_sock: could not b
Feb 22 13:58:09 moemen apachectl[2791]: no listening sockets available, shutting down
Feb 22 13:58:09 moemen apachectl[2791]: AH00015: Unable to open logs
Feb 22 13:58:09 moemen apachectl[2791]: Action 'start' failed.
Feb 22 13:58:09 moemen apachectl[2791]: The Apache error log may have more information.
Feb 22 13:58:09 moemen systemd[1]: apache2.service: Control process exited, code=exited status=1
Feb 22 13:58:09 moemen systemd[1]: apache2.service: Failed with result 'exit-code'.
Feb 22 13:58:09 moemen systemd[1]: Failed to start The Apache HTTP Server.
I'm new at this can you please help me
I also faced same problem.
First check
$ sudo systemctl status nginx
If nginx is active then stop this with
$ sudo systemctl stop nginx
then again try to start apache2 server in different terminal.
first remove apache2
sudo apt-get --purge remove apache2
sudo apt-get autoremove
after that if there files (.conf) /etc/sites-available remove them using
rm example.com.conf
then install again
sudo apt-get install apache2
now it will fixed
check it now
sudo ufw allow 'Apache'
sudo systemctl status apache2
Let me give a more general answer than the first 2. One possible problem with Apache is, when we try to run it, it may fail because port 80 is used by another software:
a common case is nginx which is covered by Devashish Mishra
in my case it was a server app that I deployed (in node.js, I had to tell pm2 to stop it)
in general, you may want to find what uses port 80. This may be done like Chi.C.J.Rajeeva Lochana has suggested: install netstat if you don't have it (sudo apt install net-tools), use it: sudo netstat -antup | grep 80. It will show some lines which may include :::80 or <your IP>:80 which will tell what is listening to the port
Once you've found what listens to the 80 port, you have to decide what to do with it. For instance, if that's nginx and you don't use it, you may go like Devashish Mishra has suggested: just stop it (sudo systemctl stop nginx). Likewise, you can stop or kill (sudo killall -9 program-name) other programs. However, if you need them, you'll also need to further configure Apache and rerun them (the exact steps highly depend on the case).
Please read this carefully.
Perform the following command, and if you see it is apache, then do the following below the command.
Note: You need to install the net-tools package before you could run netstat. Run sudo apt install net-tools to install it.
sudo netstat -antup | grep 80
You should check the line with something like <Your IP>:80.
Please note that this might also happen when you uninstall Apache when it is running.
The command could be:
sudo killall -9 program-name
Replace program-name with the program's name if the program running on port 80 is not stoppable. Let me know it it doesn't work.
Thanks.
I found this problem and was able to solve it by creating a folder /var/log/apache2, I checked in the /var/log/ folder, it turns out that there is no apache2 folder, just like in the case of mysql that won't start.
seen from your log that
Feb 22 13:58:09 moment apachectl[2791]: AH00015: Unable to open logs
maybe this will help
On your terminal.
Type: sudo stop /etc/init.d/apache2
The response will be:
Stopping apache2 (via systemctl): apache2.service.
Now start the server:
sudo /opt/lampp/lampp start
If you installed lamp correctly this should work
In Ubuntu 18.04.02 Server Edition with postgreSQL-11 I want to add the possibility of SSL communication between the postgreSQL server and a client.
I turned the ssl on in postgresql.conf :
sudo nano /etc/postgresql/11/main/postgresql.conf
ssl = on
and in pg_hba.conf I added a line for ssl :
sudo nano /etc/postgresql/11/main/pg_hba.conf
local all postgres peer
# TYPE DATABASE USER ADDRESS
METHOD
# "local" is for Unix domain socket connections only
local all all peer
# IPv4 local connections:
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 md5
# Allow replication connections from localhost, by a user with the
# replication privilege.
local replication all peer
host replication all 127.0.0.1/32 md5
host replication all ::1/128 md5
host all all 192.168.1.1/24 md5
# Only for SSL connections :
hostssl all all cert
But when reloading the new configuration, and restarting postgreSQL :
(base) marco#pc:~$ sudo /etc/init.d/postgresql reload
[ ok ] Reloading postgresql configuration (via systemctl):
postgresql.service.
(base) marco#pc:~$ sudo service postgresql restart
(base) marco#pc:~$ sudo service postgresql status
● postgresql.service - PostgreSQL RDBMS
Loaded: loaded (/lib/systemd/system/postgresql.service; enabled;
vendor preset: enabled)
Active: active (exited) since Mon 2019-09-16 18:48:13 CEST; 6s
ago
Process: 3349 ExecReload=/bin/true (code=exited, status=0/SUCCESS)
Process: 3399 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
Main PID: 3399 (code=exited, status=0/SUCCESS)
Sep 16 18:48:13 pc systemd[1]: Starting PostgreSQL RDBMS...
Sep 16 18:48:13 pc systemd[1]: Started PostgreSQL RDBMS.
I get this error:
(base) marco#pc:~$ sudo su -l postgres
postgres#pc:~$ psql
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I already tried the suggestions found here: https://askubuntu.com/questions/50621/cannot-connect-to-postgresql-on-port-5432
withour any success
When commenting the hostssl line in pg_hba.conf:
# Only for SSL connections :
#hostssl all all cert
The error disappears:
(base) marco#pc:~$ sudo /etc/init.d/postgresql reload
[ ok ] Reloading postgresql configuration (via systemctl):
postgresql.service.
(base) marco#pc:~$ sudo service postgresql restart
(base) marco#pc:~$ sudo service postgresql status
● postgresql.service - PostgreSQL RDBMS
Loaded: loaded (/lib/systemd/system/postgresql.service; enabled;
vendor preset: enabled)
Active: active (exited) since Mon 2019-09-16 18:52:37 CEST; 3s
ago
Process: 3455 ExecReload=/bin/true (code=exited, status=0/SUCCESS)
Process: 3511 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
Main PID: 3511 (code=exited, status=0/SUCCESS)
Sep 16 18:52:37 pc systemd[1]: Starting PostgreSQL RDBMS...
Sep 16 18:52:37 pc systemd[1]: Started PostgreSQL RDBMS.
(base) marco#pc:~$ sudo su -l postgres
postgres#pc:~$ psql
psql (11.5 (Ubuntu 11.5-1.pgdg18.04+1))
Type "help" for help.
postgres=# help
You are using psql, the command-line interface to PostgreSQL.
Type: \copyright for distribution terms
\h for help with SQL commands
\? for help with psql commands
\g or terminate with semicolon to execute query
\q to quit
postgres=#
So, my question is : how to correctly set ssl in /etc/postgresql/11/main/pg_hba.conf ?
SOLVED: adding the IP addresses range in pg_hba.conf:
hostssl all all 192.168.1.0/24 cert
I migrated from Cloud9 to AWS Cloud9. I'm unable to see the preview of the Apache server. The server is running :
ubuntu#ip-172-31-22-83:~/environment$ sudo service apache2 status
● apache2.service - The Apache HTTP Server
Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
Drop-In: /lib/systemd/system/apache2.service.d
└─apache2-systemd.conf
Active: active (running) since Thu 2019-04-11 09:17:31 UTC; 17min ago
Process: 791 ExecStart=/usr/sbin/apachectl start (code=exited, status=0/SUCCESS)
Main PID: 997 (apache2)
Tasks: 6 (limit: 1152)
CGroup: /system.slice/apache2.service
├─ 997 /usr/sbin/apache2 -k start
├─1086 /usr/sbin/apache2 -k start
├─1087 /usr/sbin/apache2 -k start
├─1088 /usr/sbin/apache2 -k start
├─1089 /usr/sbin/apache2 -k start
└─1090 /usr/sbin/apache2 -k start
Apr 11 09:17:29 ip-172-31-22-83 systemd[1]: Starting The Apache HTTP Server...
Apr 11 09:17:31 ip-172-31-22-83 systemd[1]: Started The Apache HTTP Server.
But when I try previewing the server, nothing works:
localhost/
Client Closed Request 499 - AWSCloud9VFSService VFS connection does
not exist
localhost:8080
Cannot GET /vfs/80990xxxxxxxxxxxxxxxxxxxxxxxxxxx/previewlocalhost:8080
I already tryed this :
https://docs.aws.amazon.com/cloud9/latest/user-guide/app-preview.html#app-preview-app-reload
But it's a fake server and don't load the root file index.php. It displays list of files in folders and do not consider .htaccess files.
I also try changing port for 8080 according to Configure apache to listen on port other than 80
Apache is natively running on AWS Cloud9. To make Apache working with the Cloud9 IDE preview system, you need to :
Make Apache running on port 8080, 8081 or 8082.
Configure Apache Root directory to ~/environment
Enable .htaccess
A step-by-step guide is detailed on this page
I m trying to deploy my 1st ASP.NET Core 2.2 API on AWS RHEL 7.5
my /etc/systemd/system/kestrel-mytest.service
[Unit]
Description=.NET Prototypes Application on Linux
[Service]
WorkingDirectory=/home/ec2-user/webapi
ExecStart=/usr/bin/dotnet /home/ec2-user/webapi/prototypes.dll
Restart=always
# Restart service after 10 seconds if the dotnet service crashes:
RestartSec=10
KillSignal=SIGINT
SyslogIdentifier=dotnet-example
User=apache
Environment=ASPNETCORE_ENVIRONMENT=Production
Environment=DOTNET_PRINT_TELEMETRY_MESSAGE=false
TimeoutStopSec=90
[Install]
WantedBy=multi-user.target
now I am facing with:
[ec2-user#ip-172-31-6-33 dotnet]$ sudo systemctl status kestrel-mytest.service
â kestrel-mytest.service - .NET Prototypes Application on Linux
Loaded: loaded (/etc/systemd/system/kestrel-mytest.service; disabled; vendor preset: disabled)
Active: activating (auto-restart) (Result: exit-code) since Tue 2019-02-19 14:06:39 UTC; 6s ago
Process: 3902 ExecStart=/usr/bin/dotnet /home/ec2-user/webapi/prototypes.dll (code=exited, status=145)
Main PID: 3902 (code=exited, status=145)
Feb 19 14:06:39 ip-172-31-6-33.ap-southeast-1.compute.internal systemd[1]: kestrel-mytest.service: main process exited, code=exited, status=145/n/a
Feb 19 14:06:39 ip-172-31-6-33.ap-southeast-1.compute.internal systemd[1]: Unit kestrel-mytest.service entered failed state.
Feb 19 14:06:39 ip-172-31-6-33.ap-southeast-1.compute.internal systemd[1]: kestrel-mytest.service failed.
[ec2-user#ip-172-31-6-33 dotnet]$
what I missed?..
thanks a lot in advance
Don
this would resolved:
working directory MUST be the same as DocumentRoot of Apache (/etc/httpd/conf/httpd.conf) (in my case, DocumentRoot is /var/www/html/ so, it should be:
WorkingDirectory=/var/www/html/webapi, do does the ExecStart as below:
ExecStart=/usr/bin/dotnet /var/www/html/webapi/prototypes.dll
things to consider:
chown -R apache:your_group /var/www/html/webapi
don't forget to stop and start kestrel to take effect.
systemctl stop kestrel-xxx
systemctl start kestrel-xxx
systemctl enable kestrel-xxx to automate start after machine rebooted
to check dotnet listener port status
sudo lsof -i -P -n | grep LISTEN
I try to restart the apache service on ubuntu 16.04 but i can not because the port '80' is listened to by nginx.
But when I try to stop nginx, the service is not found.
netstat -ltnp | grep ':80'
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 6069/nginx
systemctl status nginx
● nginx.service
Loaded: not-found (Reason: No such file or directory)
Active: inactive (dead)
service nginx stop
Failed to stop nginx.service: Unit nginx.service not loaded.
So how can I stop nginx in order to restart apache ?
Thanks for your help
nginx on your machine was started in other way then systemd standard nginx service.
alex#openresty:~$ ps aux | grep nginx
root 2668 0.0 0.0 41040 928 ? Ss 12:33 0:00 nginx: master process /usr/local/openresty/bin/openresty -p /usr/local/openresty/nginx/
nobody 2669 0.0 0.0 41040 3316 ? S 12:33 0:00 nginx: worker process
Will show you the full path of running nginx with command line arguments. Here is example from my PC (I use Openresty bundle so your paths may vary).
Now take a look at nginx master process command line parameters. If -p is present you should use exactly the same to stop nginx. For my case it should be:
/usr/local/openresty/bin/openresty -p /usr/local/openresty/nginx/ -s stop
If -p is missed you may just
/usr/local/openresty/bin/openresty -s stop
It would stop nginx, but it may be possible that on your PC nginx is configured to run as not standard service, for example hand written systemd unit.
So on every reboot you will have nginx running again.