Gitlab CE 14.10.0 - 404 not found and lacking permissions after Installation - apache

I've recently updated my setup from Debian 10 (Buster) to Debian 11 (Bullseye) and have Gitlab-CE installed.
I'm using Apache instead of Nginx as the system is being used as a website development server which is making use of port 80 and 443.
On my Debian 10 setup I was able to access Gitlab through a proxy / reverse proxy on a subdomain e.g. "git.example.com"
settings can be found below.
However in Debian 11 with the same configuration and setup, I am faced with an error 404 response.
Issue | Error Message
"404 : The page could not be found or you don't have permission to view it."
Image of Error Message:
https://aws1.discourse-cdn.com/business4/uploads/gitlab/original/3X/c/2/c2879ede19d693d0eb4032fd9a6a1f192bcddb97.png
Information
Notes:
System is running in Oracle Virtual Machine: Virtual Box
Example.com is used in place of my DNS / domain
Setup uses Subdomain access
GitLab has been configured with the use of Apache2 instead of Nginx
Docker is NOT being used.
Port Forwarding has been configured (Router, VM and UFW Firewall)
Image of UFW settings: https://aws1.discourse-cdn.com/business4/uploads/gitlab/original/3X/d/a/daac2a6b30f519272d7b358a4089675aab905588.png
Apache2 Modules
headers
rewrite
ssl
proxy
proxy_http
Installation
Gitlab-CE Packages
curl -s https://packages.gitlab.com/install/repositories/gitlab/gitlab-ce/script.deb.sh | sudo bash
sudo apt-get install gitlab-ce
Gitlab.rb
nano /etc/gitlab/gitlab.rb
## Set Url
external_url "https://gitlab.example.com"
## Give apache user privileges to listen to gitLab
web_server['external_users'] = ['www-data']
web_server['username'] = 'apache' # updated: apache, default: gitlab-www
web_server['group'] = 'apache' # updated: apache, default: gitlab-www
## Disable nginx
nginx['enable'] = false
## Config network
gitlab_workhorse['enable'] = true
gitlab_workhorse['listen_network'] = "tcp"
gitlab_workhorse['listen_umask'] = 0
gitlab_workhorse['listen_addr'] = "127.0.0.1:8181"
gitlab_workhorse['auth_backend'] = "http://127.0.0.1:8080"
# Puma
puma['listen'] = '127.0.0.1'
puma['port'] = 8080
Apache Http.conf
nano /etc/apache2/sites-available/http.conf
<VirtualHost *:80>
ServerName git.example.com
ServerSignature Off
RewriteEngine on
RewriteCond %{HTTPS} !=on
RewriteRule .* https://%{SERVER_NAME}%{REQUEST_URI} [NE,R,L]
</VirtualHost>
Apache Https.conf
nano /etc/apache2/sites-available/https.conf
certbot --apache
<IfModule mod_ssl.c>
<VirtualHost *:443>
# Server
ServerName git.example.com
ServerSignature Off
ProxyPreserveHost On
# Ensure that encoded slashes are not decoded but left in their encoded state.
# http://doc.gitlab.com/ce/api/projects.html#get-single-project
AllowEncodedSlashes NoDecode
<Location />
# New authorization commands for apache 2.4 and up
# http://httpd.apache.org/docs/2.4/upgrading.html#access
Require all granted
# Allow forwarding to gitlab-workhorse
ProxyPassReverse http://127.0.0.1:8181
ProxyPassReverse http://git.example.com
</Location>
# Apache equivalent of nginx try files
# http://serverfault.com/questions/290784/what-is-apaches-equivalent-of-nginxs-try-files
# http://stackoverflow.com/questions/10954516/apache2-proxypass-for-rails-app-gitlab
RewriteEngine on
# Forward all requests to gitlab-workhorse except existing files like error documents
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f [OR]
RewriteCond %{REQUEST_URI} ^/uploads/.*
RewriteRule .* http://127.0.0.1:8181%{REQUEST_URI} [P,QSA,NE]
RequestHeader set X_FORWARDED_PROTO 'https'
RequestHeader set X-Forwarded-Ssl on
# Needed for downloading attachments
DocumentRoot /opt/gitlab/embedded/service/gitlab-rails/public
#Set up apache error documents, if back end goes down (i.e. 503 error) then a maintenance/deploy page is thrown up.
ErrorDocument 404 /404.html
ErrorDocument 422 /422.html
ErrorDocument 500 /500.html
ErrorDocument 502 /502.html
ErrorDocument 503 /503.html
# The default directory log is usually /var/log/httpd
# As this is an Apache2 configuration, use of ${APACHE_LOG_DIR} may be more approrpaite
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" common_forwarded
ErrorLog ${APACHE_LOG_DIR}/git.example.com_error.log
CustomLog ${APACHE_LOG_DIR}/git.example.com_forwarded.log common_forwarded
CustomLog ${APACHE_LOG_DIR}/git.example.com_access.log combined env=!dontlog
CustomLog ${APACHE_LOG_DIR}/git.example.com.log combined
# SSL
# Use strong encryption ciphers only
# See ciphers(1) http://www.openssl.org/docs/apps/ciphers.html
SSLEngine on
SSLProtocol all -SSLv2
SSLHonorCipherOrder on
SSLCipherSuite "ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS"
Header add Strict-Transport-Security: "max-age=15768000;includeSubdomains"
SSLCompression Off
SSLCertificateFile /etc/letsencrypt/live/git.example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/git.example.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>
System & Gitlab
gitlab-rake gitlab:env:info
System: Debian 11
Current User: git
Using RVM: no
Ruby Version: 2.7.5p203
Gem Version: 3.1.4
Bundler Version:2.2.33
Rake Version: 13.0.6
Redis Version: 6.2.6
Sidekiq Version:6.4.0
Go Version: unknown
GitLab information
Version: 14.10.0
Revision: 88da5554d96
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: PostgreSQL
DB Version: 12.7
URL: https://git.example.com
HTTP Clone URL: https://git.example.com/some-group/some-project.git
SSH Clone URL: git#git.example.com:some-group/some-project.git
Using LDAP: no
Using Omniauth: yes
Omniauth Providers:
GitLab Shell
Version: 13.25.1
Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories
GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell
Gitlab Status
service gitlab-runsvdir status
gitlab-runsvdir.service - GitLab Runit supervision process
Loaded: loaded (/lib/systemd/system/gitlab-runsvdir.service; enabled; vendor preset: enabled)
Active: active (running) since Sun 2022-05-01 16:48:52 BST; 40min ago
Main PID: 1786 (runsvdir)
Tasks: 304 (limit: 4915)
Memory: 2.8G
CPU: 43min 49.508s
CGroup: /system.slice/gitlab-runsvdir.service
├─1786 runsvdir -P /opt/gitlab/service log: ..............................................................>
├─1788 runsv node-exporter
├─1789 runsv grafana
├─1790 runsv gitlab-workhorse
├─1791 runsv sidekiq
├─1792 runsv gitlab-kas
├─1793 runsv puma
├─1794 runsv prometheus
├─1795 runsv redis
├─1796 runsv postgresql
├─1797 runsv postgres-exporter
├─1798 runsv logrotate
├─1799 runsv redis-exporter
├─1800 runsv alertmanager
├─1801 runsv gitlab-exporter
├─1802 runsv gitaly
├─1803 svlogd -tt /var/log/gitlab/node-exporter
├─1804 svlogd -tt /var/log/gitlab/prometheus
├─1805 svlogd /var/log/gitlab/gitlab-workhorse
├─1806 svlogd -tt /var/log/gitlab/gitlab-kas
├─1807 svlogd -tt /var/log/gitlab/puma
├─1808 svlogd -tt /var/log/gitlab/redis
├─1809 svlogd /var/log/gitlab/sidekiq
├─1810 svlogd -tt /var/log/gitlab/logrotate
├─1811 /opt/gitlab/embedded/bin/node_exporter --web.listen-address=localhost:9100 --collector.mountstats ->
├─1812 svlogd -tt /var/log/gitlab/grafana
├─1813 svlogd -tt /var/log/gitlab/gitlab-exporter
├─1814 svlogd -tt /var/log/gitlab/postgresql
├─1815 svlogd -tt /var/log/gitlab/redis-exporter
├─1816 svlogd -tt /var/log/gitlab/alertmanager
├─1817 svlogd -tt /var/log/gitlab/postgres-exporter
├─1818 svlogd /var/log/gitlab/gitaly
├─1819 /opt/gitlab/embedded/bin/prometheus --web.listen-address=localhost:9090 --storage.tsdb.path=/var/op>
├─1820 /opt/gitlab/embedded/bin/gitlab-workhorse -listenNetwork tcp -listenUmask 0 -listenAddr localhost:8>
├─1821 /opt/gitlab/embedded/bin/gitlab-kas --configuration-file /var/opt/gitlab/gitlab-kas/gitlab-kas-conf>
├─1823 /bin/sh /opt/gitlab/embedded/bin/gitlab-logrotate-wrapper
├─1824 /opt/gitlab/embedded/bin/redis-server unixsocket:/var/opt/gitlab/redis/redis.socket
├─1825 ruby /opt/gitlab/embedded/service/gitlab-rails/bin/sidekiq-cluster -e production -r /opt/gitlab/emb>
├─1833 /opt/gitlab/embedded/bin/postgres -D /var/opt/gitlab/postgresql/data
├─1834 /opt/gitlab/embedded/bin/redis_exporter --web.listen-address=localhost:9121 --redis.addr=unix:///va>
├─1835 /opt/gitlab/embedded/bin/alertmanager --web.listen-address=localhost:9093 --storage.path=/var/opt/g>
lines 1-49
Debugging
As this is system running in a VM, I've made restore points to recover the VM and rollback if necessary.
Attempts made
Change folder permissions and ownership of gitlab.rb & /opt/gitlab/embedded/service/gitlab-rails/public
Checked examples of Apache Configurations from various sources
Cleared Cache / Cookies in browser
Used other browsers to attempt loading
Tried loading Gitlab through localhost on Debian server directly "same result as accessing via git.example.com"
Updating and Upgrading packages

Related

Run Mercure on production : 404 no found

I am contacting you because I can't get mercure to work in production.
The binary prebuild runs fine, but when I try to connect to the hub, I get a 404 no found.
Here is the command I run :
sudo MERCURE_PUBLISHER_JWT_KEY='eyJhbGciOiJIUzI1NiIsInR5cCI6...' MERCURE_SUBSCRIBER_JWT_KEY='eyJhbGciOiJIUzI1NiIsInR5cCI6...' SERVER_NAME=:3000 ./mercure run
the server launches without any problem apparently:
2022/02/15 17:38:09.919 INFO using adjacent Caddyfile
2022/02/15 17:38:09.920 WARN input is not formatted with 'caddy fmt' {"adapter": "caddyfile", "file": "Caddyfile", "line": 3}
2022/02/15 17:38:09.921 INFO admin admin endpoint started {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["[::1]:2019", "127.0.0.1:2019", "localhost:2019"]}
2022/02/15 17:38:09.922 INFO tls.cache.maintenance started background certificate maintenance {"cache": "0xc0000cb7a0"}
2022/02/15 17:38:09.935 INFO tls cleaning storage unit {"description": "FileStorage:/root/.local/share/caddy"}
2022/02/15 17:38:09.935 INFO tls finished cleaning storage units
2022/02/15 17:38:09.935 INFO autosaved config (load with --resume flag) {"file": "/root/.config/caddy/autosave.json"}
2022/02/15 17:38:09.935 INFO serving initial configuration
my .env is configured as such:
###> symfony/mercure-bundle ###
MERCURE_URL=https://monsite.com/.well-known/mercure
MERCURE_PUBLIC_URL=https://monsite.com/.well-known/mercure
MERCURE_JWT_SECRET="eyJhbGciOiJIUzI1NiIsInR5cCI6..."
###< symfony/mercure-bundle ###
My CaddyFile :
# Learn how to configure the Mercure.rocks Hub on https://mercure.rocks/docs/hub/config
{
{$GLOBAL_OPTIONS}
}
{$SERVER_NAME:monsite.com}
log
route {
encode zstd gzip
mercure {
# Transport to use (default to Bolt)
transport_url {$MERCURE_TRANSPORT_URL:bolt://mercure.db}
# Publisher JWT key
publisher_jwt {env.MERCURE_PUBLISHER_JWT_KEY} {env.MERCURE_PUBLISHER_JWT_ALG}
# Subscriber JWT key
subscriber_jwt {env.MERCURE_SUBSCRIBER_JWT_KEY} {env.MERCURE_SUBSCRIBER_JWT_ALG}
# Allow Subscribers
anonymous
# Extra directives
{$MERCURE_EXTRA_DIRECTIVES}
}
respond /healthz 200
respond "Not Found" 404
}
When I try to access the hub with postman by typing the following command:
https://monsite.com/.well-known/mercure
I get a 404 no found.
I am under linux debian 10 with apache2. I don't understand what I did wrong. Thanks for your help.
EDIT 21/02/2022
Hi Mehmet, here is what I did:
In /etc/apache2/sites-available monsite.conf and monsite-le-ssl.conf :
ProxyPass /mercure-hub http://localhost:8080/
ProxyPassReverse /mercure-hub http://localhost:8080/
In Caddyfile and Caddyfile.dev :
{
{$GLOBAL_OPTIONS}
auto_https off }
{$SERVER_NAME::8080}
Apparently the hub launches well, I have no error in the console:
debian#vps-...:/var/www/monsite/mercure$ sudo MERCURE_PUBLISHER_JWT_KEY='eyJhbGciOiJIUzI1NiIsInR5cCI6I...' MERCURE_SUBSCRIBER_JWT_KEY='eyJhbGciOiJIUzI1NiIsInR5cCI6I...' ./mercure run -config Caddyfile.dev
2022/02/21 13:31:20.672 INFO using provided configuration {"config_file": "Caddyfile.dev", "config_adapter": ""}
2022/02/21 13:31:20.675 WARN input is not formatted with 'caddy fmt' {"adapter": "caddyfile", "file": "Caddyfile.dev", "line": 3}
2022/02/21 13:31:20.676 INFO admin admin endpoint started {"address": "tcp/localhost:2019", "enforce_origin": false, "origins": ["localhost:2019", "[::1]:2019", "127.0.0.1:2019"]}
2022/02/21 13:31:20.676 INFO tls.cache.maintenance started background certificate maintenance {"cache": "0xc0003fe700"}
2022/02/21 13:31:20.703 INFO tls cleaning storage unit {"description": "FileStorage:/root/.local/share/caddy"}
2022/02/21 13:31:20.703 INFO tls finished cleaning storage units
2022/02/21 13:31:20.703 INFO autosaved config (load with --resume flag) {"file": "/root/.config/caddy/autosave.json"}
2022/02/21 13:31:20.704 INFO serving initial configuration
Whether I run Caddyfile or Caddyfile.dev, when accessing https://monsite.com/mercure-hub, I get a 500 error.
That is my apache settings. Maybe help to you.
open
nano /etc/apache2/sites-available/yourdomain.com-le-ssl.conf
<IfModule mod_ssl.c>
<VirtualHost *:443>
DocumentRoot /var/www/html/yourdomain.com
DirectoryIndex /index.php
ServerName yourdomain.com
#Settings for mercure
ProxyPass /mercure-hub http://localhost:8080
ProxyPassReverse /mercure-hub http://localhost:8080
<Directory /var/www/html/yourdomain.com >
AllowOverride None
Order Allow,Deny
Allow from All
FallbackResource /index.php
Options FollowSymLinks MultiViews
</Directory>
<Directory /var/www/html/yourdomain.com >
DirectoryIndex disabled
FallbackResource disabled
</Directory>
RewriteEngine on
Include /etc/letsencrypt/options-ssl-apache.conf
#YOUR SSL PEM FİLES
SSLCertificateFile /etc/letsencrypt/live …..
SSLCertificateKeyFile /etc/letsencrypt/live …..
</VirtualHost>
</IfModule>
Your caddyfile option should like that
{
{$GLOBAL_OPTIONS}
auto_https off
}
{$SERVER_NAME::8080}#this parameter will run http://localhost:8080
Mercure command
MERCURE_PUBLISHER_JWT_KEY='YOUR_KEY' MERCURE_SUBSCRIBER_JWT_KEY='YOUR_KEY' ./mercure run -config Caddyfile
you can try with Caddyfile.dev for test.
MERCURE_PUBLISHER_JWT_KEY='YOUR_KEY' MERCURE_SUBSCRIBER_JWT_KEY='YOUR_KEY' ./mercure run -config Caddyfile.dev
after this settings your mercure will run yourdomain.com/mercure-hub
After some digging, I tried to figure out why it was returning a 500 error. I went to the apache logs, I had this error message:
"No protocol handler was valid for the URL /. If you are using a DSO
version of mod_proxy, make sure the proxy submodules are included in
the configuration using LoadModule"
So I installed the missing components:
sudo a2enmod ssl
sudo a2enmod proxy
sudo a2enmod proxy_balancer
sudo a2enmod proxy_http
I also modified the proxy urls by adding a slash at the end of mercure-hub, otherwise I would get a 404 error:
ProxyPass /mercure-hub/ http://localhost:8080/
ProxyPassReverse /mercure-hub/ http://localhost:8080/
Restarted apache, and updated the Url in my .env :
MERCURE_URL=https://monsite.com/mercure-hub/.well-known/mercure
MERCURE_PUBLIC_URL=https://monsite.com/mercure-hub/.well-known/mercure
And it works, thanks a lot !

Gitlab with Apache 2.4 gives 502: AH01102: error reading status line from remote server 127.0.0.1:8080

I am not a server admin, so I do not configure Apache Servers on a daily basis.
But I want to use Gitlab on our server with SSL and Apache 2.4.6. (httpd on CentOS 7).
So far I have added the certificates (.pem) and turned on SSL.
head -30 /opt/gitlab/embedded/service/gitlab-rails/config/gitlab.yml
gitlab:
## Web server settings (note: host is the FQDN, do not include http://)
host: gitlab.my-domain.org
port: 443
https: true
# The maximum time unicorn/puma can spend on the request. This needs to be smaller than the worker timeout.
# Default is 95% of the worker timeout
max_request_duration_seconds: 57
The Apache Configuration:
<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerName gitlab.my-domain.org
ServerSignature Off
ProxyPreserveHost On
ProxyTimeout 60
AllowEncodedSlashes NoDecode
SSLEngine on
SSLProxyEngine on
SSLCertificateFile /etc/letsencrypt/live/some-original-domain/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/some-original-domain/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateChainFile /etc/letsencrypt/live/some-original-domain/chain.pem
<Location />
# New authorization commands for apache 2.4 and up
# http://httpd.apache.org/docs/2.4/upgrading.html#access
Require all granted
ProxyPassReverse http://127.0.0.1:8080
ProxyPassReverse https://gitlab.my-domain.org/
</Location>
RewriteEngine on
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f [OR]
RewriteCond %{REQUEST_URI} ^/uploads/.*
RewriteRule .* https://127.0.0.1:8080%{REQUEST_URI} [P,QSA]
RequestHeader set X_FORWARDED_PROTO 'https'
RequestHeader set X-Forwarded-Ssl on
# needed for downloading attachments
DocumentRoot /opt/gitlab/embedded/service/gitlab-rails/public
#Set up apache error documents, if back end goes down (i.e. 503 error) then a maintenance/deploy page is thrown up.
# ErrorDocument 404 /404.html
# ErrorDocument 422 /422.html
# ErrorDocument 500 /500.html
# ErrorDocument 503 /deploy.html
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" common_forwarded
ErrorLog /var/log/httpd/gitlab.my-domain_error.log
CustomLog /var/log/httpd/gitlab.my-domain_forwarded.log common_forwarded
CustomLog /var/log/httpd/gitlab.my-domain_access.log combined env=!dontlog
CustomLog /var/log/httpd/gitlab.my-domain.log combined
</VirtualHost>
</IfModule>
There must be some misconfiguration, I only see:
[proxy_http:error] [pid 18466] (103)Software caused connection abort: [client {IP}:32906] AH01102: error reading status line from remote server 127.0.0.1:8080
[proxy:error] [pid 18466] [client {IP}:32906] AH00898: Error reading from remote server returned by /
And I do not understand the <location> part of this configuration in
Also for wget http://localhost:8080/api/v3/internal/check on the server
{....} connection refused
ERROR 410: Gone.
Finally gitlab-rake gitlab:check:
Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 13.7.0 ? ... OK (13.7.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... skipped (no tmp uploads folder yet)
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
GitLab Instance / Monitoring ... yes
sdp-dev / sdp-services ... yes
sdp-dev / co2compass-app ... yes
sdp-dev / sdp-api ... yes
sdp-dev / sdp-ops ... yes
Redis version >= 4.0.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.6)
Git version >= 2.24.0 ? ... yes (2.28.0)
Git user has default SSH configuration? ... yes
Active users: ... 3
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
I think it is not a timeout issue, because the whole request from my browser takes ~150ms. I was able to reach the Gitlab instance via http before.
Any idea?
Same answer than on serverfault:
There was a problem in the Apache configuration:
RewriteRule .* http://127.0.0.1:8080%{REQUEST_URI} [P,QSA]
instead of https://...
The other change I've made was adding
SSLCompression Off
but I am not sure at all regarding its impact.
Anyway, now it works fine.

Apache 2.4 ReverseProxy to Internal CentOS 7 GitLab Server

I have been trying to setup Apache as a ReverseProxy to an internal Gitlab server with no success.
We are running multiple virtual servers, each server having a different application. We have an existing server with Apache 2.4 installed on CentOS 6.6 and working, and I just recently stood up a new server with GitLab 8.12 on CentOS 7. I have made the following change in /etc/gitlab/gitlab.rb
external_url 'http://mypublic.domain.com:80'
I have added the following rules in iptables
-A INPUT -p tcp -m state NEW -m tcp --dport 22 -j ACCEPT
-A INPUT -p tcp -m state NEW -m tcp --dport 80 -j ACCEPT
I have verified that we can ssh in using http://mypublic.domain.com:22
I have tried every setup I could find online to set the VirtualHost in httpd.conf, but nothing is working correctly. Below are a couple of the more recent, simple examples I have found. Does anyone have any ideas?
<VirtualHost 192.168.1.000:80>
ServerName mypublic.domain.com
RewriteEngine On
ProxyPass / http://192.168.1.999/
ProxyPassReverse / http://192.168.1.999/
</VirtualHost>
####
<VirtualHost 192.168.1.000:80>
ServerName mypublic.domain.com
ProxyPreserveHost On
AllowEncodedSlashes NoDecode
<Location />
Require all granted
ProxyPass http://192.168.1.999:80
ProxyPassReverse http://192.168.1.999:80
</Location>
</VirtualHost>
####
Apache Server: 192.168.1.000
GitLab Server: 192.168.1.999
Any help would be much appreciated, thanks!

Apache Webserver ReverseProxy to serve Apache Solr Admin Panel

I'm trying to run an Apache Solr Service (on its emdedded jetty server) on a remote server. The admin has provided me following information:
DNS: my.server.com
IP: xxx.xxx.xxx
Server OS: 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux
Only Port 80 is accessible. On the server we want to deploy Apache Solr and a microservice which uses Solr as search engine. I want to use Apache Webserver to forward the HTTP-Request to the Solr Admin UI and to the microservice UI, but it doesn't seem to work, I use Apache Server version: Apache/2.4.10 (Debian)
Server built: Sep 15 2016 20:44:43.
I installed Apache and started the server, so far everything works as expected. I can access the admin view from Apache entering the DNS in my browser.
I enabled a few modules following this articel https://www.digitalocean.com/community/tutorials/how-to-use-apache-http-server-as-reverse-proxy-using-mod_proxy-extension:
a2enmod proxy
a2enmod proxy_http
a2enmod proxy_ajp
a2enmod rewrite
a2enmod deflate
a2enmod headers
a2enmod proxy_balancer
a2enmod proxy_connect
a2enmod proxy_html
Then I tried to configure a virtual host under /etc/apache2/sites-available/myconf.conf:
<VirtualHost *:80>
DocumentRoot /var/www/html
ErrorLog /var/log/apache2/error.log
CustomLog /var/log/apache2/access.log combined
ProxyPass /solr http://my.server.com:8983 retry=0 timeout=5
ProxyPassReverse /solr http://my.server.com:8983
ProxyPass /microservice http://my.server.com:6868 retry=0 timeout=5
ProxyPassReverse /microservice http://my.server.com:6868
LogLevel debug
</VirtualHost>
Solr uses its standard port 8983 and the microservice will be on port 6868. When I try to acces solr with http://my.server.com/solr I get an HTTP 503 Service unavailable.
I first tried this:
/usr/sbin/setsebool -P httpd_can_network_connect 1
But it changed nothing. I also had to install first:
apt-get install policycoreutils
to make this option available. The solr service seems to be ok:
solr status
Found 1 Solr nodes:
Solr process 14082 running on port 8983
{
"solr_home":"/etc/apache-solr/solr-6.2.0/server/solr",
"version":"6.2.0 764d0f19151dbff6f5fcd9fc4b2682cf934590c5 - mike - 2016-08-20 05:41:37",
"startTime":"2016-10-07T12:02:05.300Z",
"uptime":"0 days, 1 hours, 29 minutes, 55 seconds",
"memory":"29.7 MB (%6.1) of 490.7 MB"}
The Apache log keeps saying:
The timeout specified has expired: AH00957: HTTP: attempt to connect to xxx.xxx.xxx:8983 (my.server.com) failed
AH00959: ap_proxy_connect_backend disabling worker for (my.server.com) for 0s
AH01114: HTTP: failed to make connection to backend: my.server.com
Without my timeout setting everthing keeps the same but it takes ages before I get the 503 Error.
Any hints? After one day struggeling I'm depressed ... all I want is to finish the task.
Thanks in advance!
It turns out that I needed to append a slash to the urls:
ProxyPass /solr/ http://my.server.com:8983/ retry=0 timeout=5
ProxyPassReverse /solr/ http://my.server.com:8983/
ProxyPass /microservice/ http://my.server.com:6868/ retry=0 timeout=5
ProxyPassReverse /microservice/ http://my.server.com:6868/

ISPCONFIG 3 and GITLAB

I have my gitlab working locally, but somehow i cannot access it externally. Can't figure out the problem. I'm running Debian 8 system.
Current conf files :
/etc/gitlab/gitlab.rb
gitlab_url = "http://127.0.0.1:9999"
external_url "http://gitlab.example.ee"
gitlab_rails['gitlab_host'] = "gitlab.example.ee"
gitlab_rails['gitlab_email_from'] = "gitlab#example.ee"
gitlab_rails['internal_api_url'] = "http://localhost:9999"
web_server['external_users'] = ['www-data']
unicorn['port'] = "9999"
nginx['enable'] = false
apache vhost (/etc/apache2/sites-available/gitlab.conf)
<VirtualHost *:9999>
ServerAdmin info#example.ee
DocumentRoot /opt/gitlab/embedded/service/gitlab-rails/public
ServerName gitlab.example.ee
ServerAlias gitlab.example.ee
ProxyPreserveHost On
<Location />
Order deny,allow
Allow from all
Options FollowSymLinks
Require all granted
ProxyPassReverse http://localhost:9999/
ProxyPassReverse http://gitlab.example.ee/
</Location>
RewriteEngine on
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
RewriteRule .* http://localhost:9999%{REQUEST_URI} [P,QSA]
ErrorDocument 404 /404.html
ErrorDocument 422 /422.html
ErrorDocument 500 /500.html
ErrorDocument 503 /deploy.html
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" common_forwarded
ErrorLog /${APACHE_LOG_DIR}/gitlab.error.log
CustomLog /${APACHE_LOG_DIR}/gitlab.forwarded.log common_forwarded
CustomLog /${APACHE_LOG_DIR}/gitlab.access.log combined env=!dontlog
CustomLog /${APACHE_LOG_DIR}/gitlab.log combined
</VirtualHost>
You may need to ensure that your firewall is not blocking connections to port 9999. On Ubuntu you might need to do something like:
sudo ufw allow 9999/tcp
I realize this question is a few years old, but I've been playing with a similar setup recently (but without the ISPConfig installation). Throwing in my 2 cents for others who may run across this (this is my first time answering, so take it easy on me).
NOTE The op has not specified the version of GitLab being used. More recent versions of GitLab are probably using newer versions of gitlab.rb so I'm not sure if that will make a difference.
NOTE 2 I have found great information directly from GitLab at this site: https://docs.gitlab.com/omnibus/settings/nginx.html#using-a-non-bundled-web-server. This is basically a copy and paste, but I'm throwing in my own notes to help out where I had issues as a complete n00b.
Disable bundled Nginx
In /etc/gitlab/gitlab.rb set:
nginx['enable'] = false
Set the username of the non-bundled web-server user
By default, omnibus-gitlab has no default setting for the external
webserver user, you have to specify it in the configuration. For
Debian/Ubuntu the default user is www-data for both Apache/Nginx
whereas for RHEL/CentOS the Nginx user is nginx.
Note: Make sure you have first installed Apache/Nginx so the webserver
user is created, otherwise omnibus will fail while reconfiguring.
Let's say for example that the webserver user is www-data. In
/etc/gitlab/gitlab.rb set:
web_server['external_users'] = ['www-data']
Note: This setting is an array so you can specify more than one user
to be added to gitlab-www group. Personal Note: Please pay
close attention to the single-quotes and the array here. While
developing, I've rebuild my gitlab server multiple times and entered
in JUST a string or JUST an array and both will fail. The Chef script
will use this value to set up file permissions for its internal
directories, so Apache will not be able to write to files if this is
not correct.
Run sudo gitlab-ctl reconfigure for the change to take effect.
Note: if you are using SELinux and your web server runs under a
restricted SELinux profile you may have to loosen the restrictions on
your web server.
*Note: make sure that the webserver user has the correct permissions on all directories used by external web-server, otherwise you will
receive failed (XX: Permission denied) while reading upstream errors.
Add the non-bundled web-server to the list of trusted proxies (OPTIONAL: This is only required if your web server is on a
different machine from your gitlab instance)
Normally, omnibus-gitlab defaults the list of trusted proxies to the
what was configured in the real_ip module for the bundled NGINX.
For non-bundled web-servers the list needs to be configured directly,
and should include the IP address of your web-server if it not on the
same machine as GitLab. Otherwise users will be shown as being signed
in from your web-server's IP address.
gitlab_rails['trusted_proxies'] = [ '192.168.1.0/24', '192.168.2.1', '2001:0db8::/32' ]
(Optional) Set the right gitlab-workhorse settings if using Apache PERSONAL NOTE: I believe this is the missing config from the op's question.
Note: The values below were added in GitLab 8.2, make sure you have
the latest version installed.
Apache cannot connect to a UNIX socket but instead needs to connect to
a TCP Port. To allow gitlab-workhorse to listen on TCP (by default
port 8181) edit /etc/gitlab/gitlab.rb:
gitlab_workhorse['listen_network'] = "tcp"
gitlab_workhorse['listen_addr'] = "127.0.0.1:8181"
Run sudo gitlab-ctl reconfigure for the change to take effect.
Download the right web server configs
Go to GitLab recipes repository and look for the omnibus configs
in the webserver directory of your choice. Make sure you pick the
right configuration file depending whether you choose to serve GitLab
with SSL or not. The only thing you need to change is YOUR_SERVER_FQDN
with your own FQDN and if you use SSL, the location where your SSL
keys currently reside. You also might need to change the location of
your log files.
For the sake of completeness, here is an example of the Apache v2.4 config without the SSL configuration: READ THE COMMENTS: If you followed my above notes, in step 4 the gitlab_workhorse has already been configured to listen on tcp instead of a unix socket so that line may be ignored. DO NOT IGNORE the module dependencies! These are required for Apache to be able to proxy requests to your gitlab instance. On Ubuntu (I'm using Ubuntu Server 16.04.4, but I believe most other Ubuntu version react the same), these modules can be activated using sudo a2enmod rewrite proxy proxy_http.
# This configuration has been tested on GitLab 8.2
# Note this config assumes unicorn is listening on default port 8080 and
# gitlab-workhorse is listening on port 8181. To allow gitlab-workhorse to
# listen on port 8181, edit or create /etc/default/gitlab and change or add the following:
#
# gitlab_workhorse_options="-listenUmask 0 -listenNetwork tcp -listenAddr 127.0.0.1:8181 -authBackend http://127.0.0.1:8080"
#
#Module dependencies
# mod_rewrite
# mod_proxy
# mod_proxy_http
<VirtualHost *:80>
ServerName YOUR_SERVER_FQDN
ServerSignature Off
ProxyPreserveHost On
# Ensure that encoded slashes are not decoded but left in their encoded state.
# http://doc.gitlab.com/ce/api/projects.html#get-single-project
AllowEncodedSlashes NoDecode
<Location />
# New authorization commands for apache 2.4 and up
# http://httpd.apache.org/docs/2.4/upgrading.html#access
Require all granted
#Allow forwarding to gitlab-workhorse
ProxyPassReverse http://127.0.0.1:8181
ProxyPassReverse http://YOUR_SERVER_FQDN/
</Location>
# Apache equivalent of nginx try files
# http://serverfault.com/questions/290784/what-is-apaches-equivalent-of-nginxs-try-files
# http://stackoverflow.com/questions/10954516/apache2-proxypass-for-rails-app-gitlab
RewriteEngine on
#Forward all requests to gitlab-workhorse except existing files like error documents
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f [OR]
RewriteCond %{REQUEST_URI} ^/uploads/.*
RewriteRule .* http://127.0.0.1:8181%{REQUEST_URI} [P,QSA,NE]
# needed for downloading attachments
DocumentRoot /home/git/gitlab/public
#Set up apache error documents, if back end goes down (i.e. 503 error) then a maintenance/deploy page is thrown up.
ErrorDocument 404 /404.html
ErrorDocument 422 /422.html
ErrorDocument 500 /500.html
ErrorDocument 502 /502.html
ErrorDocument 503 /503.html
# It is assumed that the log directory is in /var/log/httpd.
# For Debian distributions you might want to change this to
# /var/log/apache2.
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" common_forwarded
ErrorLog /var/log/httpd/logs/YOUR_SERVER_FQDN_error.log
CustomLog /var/log/httpd/logs/YOUR_SERVER_FQDN_forwarded.log common_forwarded
CustomLog /var/log/httpd/logs/YOUR_SERVER_FQDN_access.log combined env=!dontlog
CustomLog /var/log/httpd/logs/YOUR_SERVER_FQDN.log combined
</VirtualHost>
This config file will not work with a simple copy and paste!
Find and replace "YOUR_SERVER_FQDN" with the Fully Qualified Domain Name of your gitlab instance. Per the op's question, this would be http://gitlab.example.ee, but should basically match the value of external_url from your gitlab.rb file.
Find and replace "httpd" with "apache". The config was not designed with a Ubuntu server in mind and the appropriate directory is called "apache". I would assume you could also use ${APACHE_LOG_DIR}, but I have not tested this myself.
For a basic setup, this should work fine. I would highly recommend looking into using the SSL setup (documentation can be found at the links provided). Even if you don't need a secure setup (maybe this is an internal server), other features in the omnibus, like Mattermost, are prone to throw errors without SSL enabled (and not with a self-signed certificate).