"Failed to start The Apache HTTP Server" after misconfiguring Froxlor - apache

This all started yesterday after I added a second IP address for port 443 to the "ips and ports" list in Froxlor. As soon as Froxlor's cron job ran, Apache failed to restart. Ever since then, nothing I try will get Apache to stay running with SSL enabled in Froxlor.
System Config:
Ubuntu 20.04.2 LTS (focal)
Apache 2.4.41
Froxlor 0.10.27
Output from sudo systemctl start apache2:
Job for apache2.service failed because the control process exited with error code.
See "systemctl status apache2.service" and "journalctl -xe" for details.
Output from systemctl status apache2.service:
● apache2.service - The Apache HTTP Server
Loaded: loaded (/lib/systemd/system/apache2.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2021-07-19 13:33:31 UTC; 41s ago
Docs: https://httpd.apache.org/docs/2.4/
Process: 17629 ExecStart=/usr/sbin/apachectl start (code=exited, status=1/FAILURE)
systemd[1]: Starting The Apache HTTP Server...
apachectl[17641]: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName'>
apachectl[17629]: Action 'start' failed.
apachectl[17629]: The Apache error log may have more information.
systemd[1]: apache2.service: Control process exited, code=exited, status=1/FAILURE
systemd[1]: apache2.service: Failed with result 'exit-code'.
systemd[1]: Failed to start The Apache HTTP Server.
Output from sudo journalctl -u apache2.service --since today --no-pager:
systemd[1]: Starting The Apache HTTP Server...
apachectl[17169]: AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message
apachectl[17165]: Action 'start' failed.
apachectl[17165]: The Apache error log may have more information.
systemd[1]: apache2.service: Control process exited, code=exited, status=1/FAILURE
systemd[1]: apache2.service: Failed with result 'exit-code'.
systemd[1]: Failed to start The Apache HTTP Server.
systemd[1]: apache2.service: Unit cannot be reloaded because it is inactive.
"Address already in use" error
Initially I was also getting an error that said apachectl[16500]: (98)Address already in use: AH00072: make_sock: could not bind to address on port 443. Running netstat -anp | grep 443 did not reveal any other processes hogging that port, so I suspected that Apache was trying to use port 443 twice (which tracks with my configuration goof). I managed to get into the database and delete the ip/port record (which had not been assigned to any sites yet) and this particular error went away because Froxlor stopped creating an extra conf file containing Listen 443.
If I comment out both instances of Listen 443 within /etc/apache2/ports.conf, then this particular error goes away but Apache still fails to load.
# If you just change the port or add more ports here, you will likely also
# have to change the VirtualHost statement in
# /etc/apache2/sites-enabled/000-default.conf
Listen 80
#<IfModule ssl_module>
# Listen 443
#</IfModule>
#<IfModule mod_gnutls.c>
# Listen 443
#</IfModule>
# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
(This post with a similar issue offered some insight on this bit)
Output from sudo grep "443" /etc/apache2/*
grep: /etc/apache2/conf-available: Is a directory
grep: /etc/apache2/conf-enabled: Is a directory
grep: /etc/apache2/htpasswd: Is a directory
grep: /etc/apache2/mods-available: Is a directory
grep: /etc/apache2/mods-enabled: Is a directory
/etc/apache2/ports.conf:# Listen 443
/etc/apache2/ports.conf:# Listen 443
grep: /etc/apache2/sites-available: Is a directory
grep: /etc/apache2/sites-enabled: Is a directory
Misc. remarks about Froxlor:
If I comment out \Froxlor\Cron\MasterCron::run(); inside of /var/www/froxlor/scripts/froxlor_master_cronjob.php, then the Froxlor cron job is effectively disabled. Can be useful for troubleshooting, but doesn't fix anything.
Running sudo /usr/bin/php /var/www/froxlor/scripts/froxlor_master_cronjob.php --force will trigger Froxlor to execute its cron job immediately
Current Status:
After many hours of troubleshooting, here is what I know:
when no ip is configured with port 443/SSL, Apache will start.
deleting /etc/apache2/sites-enabled/ directory allows Apache to start, until Froxlor's cron job regenerates it.
likewise, just deleting the *.443.conf files and any ssl.conf files from /etc/apache2/sites-enabled/ also temporarily allows Apache to start (until the Froxlor cron job runs)
removing Froxlor from the server allows Apache to start, but the problem comes back immediately after configuring port 443 within Froxlor.
TLDR: Something broke when I opened Froxlor and added a second IP with a port that was already in use (port 443). Now Apache won't start unless I delete any .conf file involving SSH. Removing Froxlor (including deleting the database) and deleting sites-enabled before reinstalling Froxlor did not resolve the issue.
EDIT: Regenerated my security certificates and now all is good.

Ugh... multitasking bit me again.
Apparently there was something off with the local security certificate. I regenerated it and Apache started working again.
Related post here

Related

monit: monitoring certificates

I am currently trying to keep watch of my systems certificates to make sure I get alerting 30 days before expiring. As an example I took google.com and monitoring its certificates. Looking at monitrc file I have added this:
check host google.com with address google.com
if failed
port 443
protocol https
with ssl options {verify: enable}
certificate valid > 1095 days
then alert
When I restart monit I get the error there is no service called google.com
When setting up new monit config or change/edit current config, should reload first to make sure monit reinitializing the daemon
Reproduce issue
# Edit monit config
root#home:~# vim /etc/monit/conf-enabled/test
# Restart right way will fail
root#home:~# monit restart all
There is no service named "google.com"
# Logs
[CEST Sep 12 19:16:29] info : 'home' trying to restart
[CEST Sep 12 19:16:29] info : 'home' restart action done
[CEST Sep 12 19:17:09] info : 'home' restart on user request
[CEST Sep 12 19:17:09] error : HttpRequest: error -- client [::1]: HTTP/1.0 400 There is no service named "google.com"
[CEST Sep 12 19:17:09] error : There is no service named "google.com"
Solution
# Edit monit config
root#home:~# vim /etc/monit/conf-enabled/test
# Reload to reinitialize monit
root#home:~# monit reload
Reinitializing monit daemon
root#home:~# monit status
Monit 5.25.2 uptime: 24m
Remote Host 'google.com'
status OK
monitoring status Monitored
monitoring mode active
on reboot start
port response time 878.069 ms to google.com:443 type TCP/IP using TLS (certificate valid for 69 days) protocol HTTP
data collected Thu, 12 Sep 2019 19:24:06
you can also run periodic tests with a software like Overseer, and receive notifications using a Notify17 notification template (see the sample recipe).
You could use a test rule like:
https://myurl.com/path must run https
Or
https://myurl.com/path must run ssl
These rules evaluate if a website is reachable over SSL and if the certificate will expire soon (you can see more options in the source code).
P.S. To have an easy start with Overseer, you can check out the Kubernetes deployment example.

Cannot install apache2 on Google Compute Engine

I deploy a Google Cloud Compute Engine small Instance. But I want to run an Apache2 Web Server on that instance. When it install by apt-get installer, at the end it says,
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Processing triggers for libc-bin (2.24-11+deb9u3) ..
When it start the service it says,
Starting Apache httpd web server: apache2AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2. Set the 'ServerName' directive globally to suppress this message
(98)Address already in use: AH00072: make_sock: could not bind to address [::]:80
(98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
AH00015: Unable to open logs
Action 'start' failed.
The Apache error log may have more information.
I allowed HTTP and HTTPS Firewall rules
What is missing there? Please help..

404 Not Found on previously working website

I previously had a website that I was hosting personally on a ubuntu server 16.04 LTS, sort of just a project to show that I knew what I was doing. (Turns out I don't have a clue!)
The site was/is structured such that I have a landing page located in /var/www/html. a second directory located in /var/www/agv-guru and a third directory located in /var/www/owncloud/.
I have an alias set up for both owncloud and the agv-guru folder and everything was working properly for months.
For whatever reason I decided that instead of having a separate server (raspberry-pi) running pi-hole, I would go ahead and install it on my existing server. I quickly realized that my website was down and I un-installed Pi-Hole, which may or may not have made matters worse.
I'm looking for help troubleshooting this one as I am not entirely sure whats going on.
Here is what i know so far.
The folders are still located in the same directory just like before.
The folders are owned by www-data.
The Document Root is /var/www/html.
The Alias is still set up for both agv-guru and owncloud.
/var/www/html/index.html still runs as normal with the exception of one photo not loading that was being pulled from the agv-guru folder.
Webserver is Apache2
Edit 1.
I just cleared the logs hoping to just reload the service but I get an error.
usr#Server:/var/log/apache2# service apache2 reload
apache2.service is not active, cannot reload.
So I ran the status as shown below.
usr#Server:/var/log/apache2# service apache2 status
● apache2.service - LSB: Apache2 web server
Loaded: loaded (/etc/init.d/apache2; bad; vendor preset: enabled)
Drop-In: /lib/systemd/system/apache2.service.d
└─apache2-systemd.conf
Active: inactive (dead) since Tue 2017-10-17 22:16:29 CDT; 58s ago
Docs: man:systemd-sysv-generator(8)
Process: 14594 ExecStop=/etc/init.d/apache2 stop (code=exited, status=0/SUCCESS)
Process: 14577 ExecStart=/etc/init.d/apache2 start (code=exited, status=0/SUCCESS)
Oct 17 22:16:29 Server apache2[14577]: (98)Address already in use: AH00072: make_sock: could not bind to address 0.0.0.0:80
Oct 17 22:16:29 Server apache2[14577]: no listening sockets available, shutting down
Oct 17 22:16:29 Server apache2[14577]: AH00015: Unable to open logs
Oct 17 22:16:29 Server apache2[14577]: Action 'start' failed.
Oct 17 22:16:29 Server apache2[14577]: The Apache error log may have more information.
Oct 17 22:16:29 Server apache2[14577]: *
Oct 17 22:16:29 Server apache2[14594]: * Stopping Apache httpd web server apache2
Oct 17 22:16:29 Server apache2[14594]: *
Oct 17 22:16:29 Server systemd[1]: Started LSB: Apache2 web server.
Oct 17 22:16:36 Server systemd[1]: apache2.service: Unit cannot be reloaded because it is inactive.
Edit 2
sudo netstat -natp | grep ':80.*LISTEN'
Yields
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN
1636/lighttpd
Solution
With sborsky's help the solution was found.
When installing pi-hole lighttpd was also installed. This webserver was occupying the port preventing apache from running.

Mod_jk workers - ipv4 / ipv6 /fqdn

I just set up our first machine with Ubuntu 16 LTS and Tomcat 8.5.11 + Apache/2.4.18, mod_jk/1.2.41.
I was quiet familiar with Ubuntu 14 LTS, Tomcat 7.0.70 and Apache/2.4.7 mod_jk/1.2.37.
Deploying my servlet seemed fine, no errors in tomcat or app log, but still not avail.
Using fqdn/app showed an error 503.
Using ip:port/app worked fine..
I saw these entries in mod_jk.log:
[Fri Feb 24 11:17:49.149 2017] [9219:139689407260416] [info] ajp_connect_to_endpoint::jk_ajp_common.c (1068): (worker1) Failed opening socket to (::1:8009) (errno=111)
[Fri Feb 24 11:17:49.149 2017] [9219:139689407260416] [error] ajp_send_request::jk_ajp_common.c (1728): (worker1) connecting to backend failed. Tomcat is probably not started or is listening on the wrong port (errno=111)
-> ::1:8009
Why is my worker.host=localhost resolved to ::1 (IPv6) instead of 127.0.0.1 (IPv4)?
I also saw the "newer" parameter "prefer_ipv6" and set it to "worker.prefer_ipv6=0" but without luck..
Workaround:
When I set the worker.host=127.0.0.1 everything works fine/as I am used to.
Downside:
I know some collegue of mine has changed the 127.0.0.1 entry to "localhost" in the past for some reason (different ip stack in processing?), so I am not 100% confident leaving it with IPv4-IP.
Some advices on how I could fix that?
It's a bug in the JK connector, where it always prefers IPv6 when resolving the hostname in the "worker.*.host=" setting. The only way to force the latest JK connector to connect via IPv4 is to use an IPv4 address (rather than a DNS name).
The other alternative is to configure tomcat AJP/1.3 listener to listen on IPv6.
Until the folks at Apache fix the bug in the JK connector, these are the only options right now.

Apache2 won't start, ulimit error?

Here's the log from trying to start apache2:
* Starting web server apache2
/usr/sbin/apache2ctl: 87: ulimit: error setting limit (Operation not permitted)
(13)Permission denied: make_sock: could not bind to address [::]:80
(13)Permission denied: make_sock: could not bind to address 0.0.0.0:80
no listening sockets available, shutting down
Unable to open logs
Action 'start' failed.
The Apache error log may have more information.
I'm new to running my own Apache. Any idea what's causing this failure?
Use sudo for starting apache server, the issue if due to permissions :
sudo service apache2 start