Jenkins behind apache proxy - apache

My Jenkins is running in Kubernetes with Service type: LoadBalancer, and added below azure annotations to take internal subnet private ip address to expose service internally.
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
service.beta.kubernetes.io/azure-load-balancer-internal-subnet: subnetName
I've one ubuntu VM where Apache is installed. Created self signed certificated and terminated in apache configurations, and I'm able to access apache home page using HTTPS.
Then I've created proxy rule to Jenkins service IP address. Basically I want to access Jenkins from Apache HTTPS --> to internally HTTP traffic towards kubernetes service.
Here is Apache configurations:
xxxx#xxxx:/etc/apache2/sites-available$ ls -ltrh
total 28K
-rw-r--r-- 1 root root 1332 Jul 16 18:14 000-default.conf
-rw-r--r-- 1 root root 6338 Jul 16 18:14 default-ssl.conf
drwxr-xr-x 2 root root 4096 Dec 12 17:24 abc
-rw-r--r-- 1 root root 680 Dec 12 13:04 abc.conf
drwxr-xr-x 2 root root 4096 Dec 12 14:29 xyz
-rw-r--r-- 1 root root 1151 Dec 12 13:08 xyz.conf
cat abc/00-redirect-to-https.conf
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule ^jenkins$ login [L,R=302]
cat abc.conf
<VirtualHost *:80>
ServerAdmin webmaster#localhost
LogLevel warn
ErrorLog ${APACHE_LOG_DIR}/abc_error.log
CustomLog ${APACHE_LOG_DIR}/abc_access.log combined
<IfModule mod_headers.c>
RequestHeader unset X-Forwarded-For
RequestHeader unset X-Forwarded-Host
RequestHeader unset X-Forwarded-Server
RequestHeader set X-Forwarded-Proto "http"
RequestHeader set X-Forwarded-Port "80"
</IfModule>
# Apache will try to set application/json based on mime type
# This behaviour casing problems with empty json responses from spring
RemoveType json
Include sites-available/abc/*.conf
</VirtualHost>
cat xyz/00-jenkins.conf
ProxyPass /jenkins balancer://jenkins/jenkins
ProxyPassReverse /jenkins balancer://jenkins/jenkins
<Proxy balancer://jenkins>
BalancerMember http://x.x.x.x:8080 loadfactor=1 keepalive=On retry=0
ProxySet lbmethod=bytraffic
</Proxy>
cat xyz.conf
<VirtualHost *:443>
ServerAdmin webmaster#localhost
ServerName FQDN
LogLevel warn
ErrorLog ${APACHE_LOG_DIR}/xyz_error.log
CustomLog ${APACHE_LOG_DIR}/xyz_access.log combined
<IfModule mod_headers.c>
RequestHeader unset X-Forwarded-For
RequestHeader unset X-Forwarded-Host
RequestHeader unset X-Forwarded-Server
RequestHeader set X-Forwarded-Proto "https"
RequestHeader set X-Forwarded-Port "443"
</IfModule>
SSLEngine on
SSLProtocol -ALL +TLSv1 +TLSv1.1 +TLSv1.2
SSLHonorCipherOrder On
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
SSLCertificateFile /etc/apache2/certs/ca.cert
SSLCertificateKeyFile /etc/apache2/certs/ca.key
# Apache will try to set application/json based on mime type
# This behaviour casing problems with empty json responses from spring
RemoveType json
Include sites-available/xyz/*.conf
</VirtualHost>
If I do curl -k https://localhost/jenkins from local ubuntu VM then response shows that authentication required which is fine as below, but redirecting url becomes window.location.replace('/login?from=%2F')
<html><head><meta http-equiv='refresh' content='1;url=/login?from=%2Fjenkins'/><script>window.location.replace('/login?from=%2Fjenkins');</script></head><body style='background-color:white; color:white;'>
Authentication required
<!--
You are authenticated as: anonymous
Groups that you are in:
Permission you need to have (but didn't): hudson.model.Hudson.Read
... which is implied by: hudson.security.Permission.GenericRead
... which is implied by: hudson.model.Hudson.Administer
-->
</body></html>
But with the same case, when I request from browser https://FQDN/jenkins again URL becomes https://FQDN/login?from=%2F But there browser throws URL Not found error
Not Found
The requested URL was not found on this server.
Please assist here to correct the configurations..
Thanks..
More observation from logs:
when I did curl -k https://localhost/jenkins apache access logs shows 403 which is ok because I've not passed credentials
127.0.0.1 - - [13/Dec/2019:13:37:40 +0000] "GET /jenkins HTTP/1.1" 403 3297 "-" "curl/7.58.0"
and when same tried from internet browser https://FQDN/jenkins apache logs first shows 403 which is wanted but soon after apache tries to find changed url in same VM instead of redirecting, due to which i'm not getting jenkins page.
165.225.106.137 - - [13/Dec/2019:13:38:19 +0000] "GET /jenkins HTTP/1.1" 403 3446 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
165.225.106.137 - - [13/Dec/2019:13:38:19 +0000] "GET /jenkins HTTP/1.1" 403 1564 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
165.225.106.137 - - [13/Dec/2019:13:38:20 +0000] "GET /login?from=%2F HTTP/1.1" 404 541 "https://DNSname/jenkins" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"

It's working now after changing below configurations --
(1)
Set the context path by modifying the jenkins.xml configuration file and adding --prefix=/jenkins (or similar) to the <arguments> entry.
https://wiki.jenkins.io/display/JENKINS/Running+Jenkins+behind+Apache
(2)
Initially -- RewriteRule ^jenkins$ login [L,R=302]
Now -- RewriteRule ^/jenkins(.*)$ /
(3)
Initially --
ProxyPass /jenkins balancer://jenkins/jenkins
ProxyPassReverse /jenkins balancer://jenkins/jenkins
<Proxy balancer://jenkins>
BalancerMember http://x.x.x.x:8080 loadfactor=1 keepalive=On retry=0
ProxySet lbmethod=bytraffic
</Proxy>
Now --
ProxyPass /jenkins balancer://jenkins
ProxyPassReverse /jenkins balancer://jenkins
ProxyRequests Off
AllowEncodedSlashes NoDecode
<Proxy balancer://jenkins>
BalancerMember http://x.x.x.x:8080/jenkins loadfactor=1 keepalive=On retry=0
ProxySet lbmethod=bytraffic
</Proxy>

Related

SAML with mod_auth_mellon

I've configured apache to make SAML auth for Grafana but the "X-WEBAUTH-USER" is not transfered to the header :
nc -l -p 9119
POST /grafana/ HTTP/1.1
Host: 127.0.0.1:9119
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Referer: https://samlidp.example.com/
Content-Type: application/x-www-form-urlencoded
Origin: https://samlidp.example.ch
DNT: 1
Cookie: mellon-cookie=cookietest
Upgrade-Insecure-Requests: 1
X-WEBAUTH-USER: (null)
Here is my config :
ServerName servername.com
ServerAdmin webmaster#servername.com
ServerAlias servername.com
DocumentRoot "/var/www/html"
# Logs and diagnotic
LogLevel debug
SSLEngine on
SSLProxyEngine On
SSLCertificateFile /etc/apache2/ssl/server.crt
SSLCertificateKeyFile /etc/apache2/ssl/server.key
#SSLv2 and v3 are bad
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite HIGH:!SSLv2:!ADH:!aNULL:!eNULL:!NULL
ProxyPass / http://127.0.0.1:9119/
ProxyPassReverse / http://127.0.0.1:9119/
<Location />
Require valid-user
AuthType "Mellon"
MellonEnable "auth"
MellonDecoder "none"
MellonVariable "cookie"
MellonSecureCookie On
MellonUser "NAME_ID"
MellonSetEnv REMOTE_USER MELLON_NAME_ID
MellonSetEnv "REMOTE_MAIL" "email"
MellonEndpointPath "/endpoint"
MellonDefaultLoginPath "/"
MellonSessionLength 300
# Mellon requires a cert, regardless if it's actually being used.
MellonSPPrivateKeyFile /etc/apache2/mellon/urn_grafana.key
MellonSPCertFile /etc/apache2/mellon/urn_grafana.cert
MellonSPMetadataFile /etc/apache2/mellon/urn_grafana.xml
#MellonSPPrivateKeyFile /etc/apache2/mellon/urn_keycloak.key
#MellonSPCertFile /etc/apache2/mellon/urn_keycloak.cert
#MellonSPMetadataFile /etc/apache2/mellon/urn_keycloak.xml
# Make sure to copy your IdP metadata here
MellonIdPMetadataFile /etc/apache2/mellon/idp-persistent.xml
#MellonIdPMetadataFile /etc/apache2/mellon/idp-keycloak.xml
MellonSamlResponseDump On
MellonSessionDump On
RequestHeader set X-WEBAUTH-USER "%{REMOTE_USER}e"
RequestHeader set X-MAIL "%{REMOTE_MAIL}e"
</Location>
<Location /grafana/>
MellonEnable "off"
Order Deny,Allow
Allow from all
Satisfy Any
</Location>
Any ideas ?
I've tried this : Federate grafana with apache2 + mod_auth_mellon to have SSO with SAML but in that case the X-WEBAUTH-USER is not even in the header.

Spinnaker HTTPS configuration through Apache

My Spinnaker is running in Kubernetes with service type: LoadBalancer and added below azure annotations to take internal subnet private ip address to expose service internally.
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
service.beta.kubernetes.io/azure-load-balancer-internal-subnet: subnetName
I've one ubuntu VM where Apache is installed. Created self signed certificated and terminated in apache configurations, and I'm able to access apache home page using HTTPS.
Then I've created proxy rule to Spinnaker service IP address. Basically I want to access Spinnaker from Apache HTTPS --> to internally HTTP traffic towards kubernetes service.
Here is Apache configurations:
xxxx#xxxx:/etc/apache2/sites-available$ ls -ltrh
total 28K
-rw-r--r-- 1 root root 1332 Jul 16 18:14 000-default.conf
-rw-r--r-- 1 root root 6338 Jul 16 18:14 default-ssl.conf
drwxr-xr-x 2 root root 4096 Dec 12 17:24 abc
-rw-r--r-- 1 root root 680 Dec 12 13:04 abc.conf
drwxr-xr-x 2 root root 4096 Dec 12 14:29 xyz
-rw-r--r-- 1 root root 1151 Dec 12 13:08 xyz.conf
cat abc/00-redirect-to-https.conf
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule ^spinnaker$ / [L,R=302]
cat abc.conf
<VirtualHost *:80>
ServerAdmin webmaster#localhost
LogLevel warn
ErrorLog ${APACHE_LOG_DIR}/abc_error.log
CustomLog ${APACHE_LOG_DIR}/abc_access.log combined
<IfModule mod_headers.c>
RequestHeader unset X-Forwarded-For
RequestHeader unset X-Forwarded-Host
RequestHeader unset X-Forwarded-Server
RequestHeader set X-Forwarded-Proto "http"
RequestHeader set X-Forwarded-Port "80"
</IfModule>
# Apache will try to set application/json based on mime type
# This behaviour casing problems with empty json responses from spring
RemoveType json
Include sites-available/abc/*.conf
</VirtualHost>
cat xyz/00-spinnaker.conf
ProxyPass /spinnaker balancer://spinnaker
ProxyPassReverse /spinnaker balancer://spinnaker
ProxyRequests Off
AllowEncodedSlashes NoDecode
<Proxy balancer://spinnaker>
BalancerMember http://172.18.1.99:9000/spinnaker loadfactor=1 keepalive=On retry=0
ProxySet lbmethod=bytraffic
</Proxy>
cat xyz.conf
<VirtualHost *:443>
ServerAdmin webmaster#localhost
ServerName FQDN
LogLevel warn
ErrorLog ${APACHE_LOG_DIR}/xyz_error.log
CustomLog ${APACHE_LOG_DIR}/xyz_access.log combined
<IfModule mod_headers.c>
RequestHeader unset X-Forwarded-For
RequestHeader unset X-Forwarded-Host
RequestHeader unset X-Forwarded-Server
RequestHeader set X-Forwarded-Proto "https"
RequestHeader set X-Forwarded-Port "443"
</IfModule>
SSLEngine on
SSLProtocol -ALL +TLSv1 +TLSv1.1 +TLSv1.2
SSLHonorCipherOrder On
SSLCipherSuite ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
SSLCertificateFile /etc/apache2/certs/ca.cert
SSLCertificateKeyFile /etc/apache2/certs/ca.key
# Apache will try to set application/json based on mime type
# This behaviour casing problems with empty json responses from spring
RemoveType json
Include sites-available/xyz/*.conf
</VirtualHost>
if I request this url in browser https://apacheServerDomainName/spinnaker then it redirects to spinnaker internally,
But then if I want to go any other page in spinnaker say click on projects, applications etc. then it won't work because url will change to https://apacheServerDomainName/applications and this will give 404 because it assumes to get the page from local ubuntu apache server,
whereas that request should also redirect and response from spinnaker.
please advise what kind of apache rewrite rule could help to achieve this requirement or any other suggestion..
Follow these steps
deploy nginx ingress controller
Define Ingress rule for Spinnaker including TLS certificate in a secret
Nginx controller would do TLS termination allowing external connections over HTTPS

403 Forbidden on CORS request from local

I am working with Gluu Server and trying to get the OpenID Connect configuration from the /.well-known/openid-configuration endpoint through a CORS/AJAX request (for use with an Angular app). However, when I try to request the endpoint from a locally hosted app/HTML file with XHR requesting the endpoint, I receive a 403 Forbidden error.
This only seems to happen when the request stems from a local context, i.e. Angular's development server or a local HTML file requesting the endpoint. If I open the same HTML file that performs the AJAX request, hosted on a server, it works.
The testing HTML file looks like the following
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
</head>
<body>
<div id="content"></div>
<script type="text/javascript">
var url = 'https://example.com/.well-known/openid-configuration';
var req = new XMLHttpRequest();
req.open('GET', url, true);
req.setRequestHeader('Content-Type', 'application/json');
req.onload = () => {
if (req.status >= 200 && req.status < 400) {
console.log('[XHR SUCCESS]');
var el = document.getElementById('content');
el.innerHTML = req.responseText;
} else {
console.log('[XHR ERROR]', req);
}
}
req.onerror = () => {
console.log('[XHR CONNECTION ERROR]');
}
req.send();
</script>
</body>
</html>
Requesting from local file
As mentioned above, when requesting from a local HTML file, I receive the 403 Forbidden error.
In the browser console (Chrome), two errors are output:
Failed to load resource: the server responded with a status of 403 (Forbidden)
Access to XMLHttpRequest at 'https://example.com/.well-known/openid-configuration' from origin 'null' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status.
The only output on the server, that I have found, relating to this, is in the file /var/log/apache2/other_vhosts_access.log:
example.com:443 <IP> - - [11/Mar/2019:10:45:20 +0000] "OPTIONS /.well-known/openid-configuration HTTP/1.1" 403 3763 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36"
The server receives the following (from the log_forensic module for Apache) when requested from local:
OPTIONS /.well-known/openid-configuration HTTP/1.1|Host:example.com|Connection:keep-alive|Pragma:no-cache|Cache-Control:no-cache|Access-Control-Request-Method:GET|Origin:null|User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36|Access-Control-Request-Headers:content-type|Accept:*/*|Accept-Encoding:gzip, deflate, br|Accept-Language:en-US,en;q=0.9
Requesting from server-hosted file
When doing the exact same thing as above, but with the HTML file hosted on a server, the request completes successfully.
Output in the access log:
example.com:443 <IP> - - [11/Mar/2019:11:06:46 +0000] "OPTIONS /.well-known/openid-configuration HTTP/1.1" 200 779 "http://example.org/xhr-cors.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36"
example.com:443 <IP> - - [11/Mar/2019:11:06:46 +0000] "GET /.well-known/openid-configuration HTTP/1.1" 200 6629 "http://example.org/xhr-cors.html" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36"
From log_forensic:
OPTIONS /.well-known/openid-configuration HTTP/1.1|Host:example.com|Connection:keep-alive|Access-Control-Request-Method:GET|Origin:http%3a//example.org|User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36|Access-Control-Request-Headers:content-type|Accept:*/*|Referer:http%3a//example.org/xhr-cors.html|Accept-Encoding:gzip, deflate, br|Accept-Language:en-US,en;q=0.9
GET /.well-known/openid-configuration HTTP/1.1|Host:example.com|Connection:keep-alive|Origin:http%3a//example.org|User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.121 Safari/537.36|Content-Type:application/json|Accept:*/*|Referer:http%3a//example.org/xhr-cors.html|Accept-Encoding:gzip, deflate, br|Accept-Language:en-US,en;q=0.9
Apache configuration
The configuration for Apache on the server is
<VirtualHost *:80>
ServerName example.com
Redirect / https://example.com/
DocumentRoot "/var/www/html/"
RewriteEngine on
RewriteCond %{SERVER_NAME} =example.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
<VirtualHost *:443>
DocumentRoot "/var/www/html/"
ServerName example.com:443
LogLevel warn
SSLEngine on
SSLProtocol -all +TLSv1.1 +TLSv1.2
SSLHonorCipherOrder On
SSLCipherSuite ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:DHE-RSA-AES128-SHA256:DHE-DSS-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:DHE-RSA-AES128-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK
# SetEnv proxy-nokeepalive 1
SetEnv proxy-initial-not-pooled 1
Timeout 60
ProxyTimeout 60
# Security headers
# Header always append X-Frame-Options SAMEORIGIN
Header always set X-Xss-Protection "1; mode=block"
Header always set X-Content-Type-Options nosniff
# Header always set Content-Security-Policy "default-src 'self' 'unsafe-inline' https://example.com"
Header always set Strict-Transport-Security "max-age=31536000; includeSubDomains"
Header edit Set-Cookie ^((?!session_state).*)$ $1;HttpOnly
SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
# Unset X-ClientCert to make sure that we not get certificate in request
RequestHeader unset X-ClientCert
# Turn off support for true Proxy behaviour as we are acting as a transparent proxy
ProxyRequests Off
# Turn off VIA header as we know where the requests are proxied
ProxyVia Off
# Turn on Host header preservation so that the servlet container
# can write links with the correct host and rewriting can be avoided.
ProxyPreserveHost On
# Preserve the scheme when proxying the request to Jetty
RequestHeader set X-Forwarded-Proto "https" env=HTTPS
Header unset ETag
FileETag None
RedirectMatch ^(/)$ /identity/
# Set the permissions for the proxy
<Proxy *>
AddDefaultCharset off
Order deny,allow
Allow from all
</Proxy>
<Location /oxauth>
ProxyPass http://localhost:8081/oxauth retry=5 connectiontimeout=60 timeout=60
# Header set Access-Control-Allow-Origin "*"
Order deny,allow
Allow from all
</Location>
<LocationMatch /oxauth/auth/cert/cert-login>
SSLVerifyClient optional_no_ca
SSLVerifyDepth 10
SSLOptions -StdEnvVars +ExportCertData
# Forward certificate to destination server
RequestHeader set X-ClientCert %{SSL_CLIENT_CERT}s
</LocationMatch>
<Location /idp>
ProxyPass http://localhost:8086/idp retry=5 connectiontimeout=60 timeout=60
Order deny,allow
Allow from all
</Location>
<Location /identity>
ProxyPass http://localhost:8082/identity retry=5 connectiontimeout=60 timeout=60
Order deny,allow
Allow from all
</Location>
<Location /cas>
ProxyPass http://localhost:8083/cas retry=5 connectiontimeout=60 timeout=60
Order deny,allow
Allow from all
</Location>
<Location /oxauth-rp>
ProxyPass http://localhost:8085/oxauth-rp retry=5 connectiontimeout=60 timeout=60
Order deny,allow
Allow from all
</Location>
<Location /asimba>
ProxyPass http://localhost:8084/asimba retry=5 connectiontimeout=60 timeout=60
Order deny,allow
Allow from all
</Location>
<Location /passport>
ProxyPass http://localhost:8090/passport retry=5 connectiontimeout=60 timeout=60
Order deny,allow
Allow from all
</Location>
<Location /casa>
ProxyPass http://localhost:8091/casa retry=5 connectiontimeout=60 timeout=60
Order deny,allow
Allow from all
</Location>
<LocationMatch "/.well-known/openid-configuration">
ProxyPass http://localhost:8081/oxauth/.well-known/openid-configuration
Header set Access-Control-Allow-Origin "*"
</LocationMatch>
# ProxyPass /.well-known/openid-configuration http://localhost:8081/oxauth/.well-known/openid-configuration
ProxyPass /.well-known/simple-web-discovery http://localhost:8081/oxauth/.well-known/simple-web-discovery
ProxyPass /.well-known/webfinger http://localhost:8081/oxauth/.well-known/webfinger
ProxyPass /.well-known/uma2-configuration http://localhost:8081/oxauth/restv1/uma2-configuration
ProxyPass /.well-known/fido-configuration http://localhost:8081/oxauth/restv1/fido-configuration
ProxyPass /.well-known/fido-u2f-configuration http://localhost:8081/oxauth/restv1/fido-configuration
ProxyPass /.well-known/scim-configuration http://localhost:8082/identity/restv1/scim-configuration
ServerAlias example.com
SSLCertificateFile /etc/letsencrypt/live/example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
I have commented out the
ProxyPass /.well-known/openid-configuration http://localhost:8081/oxauth/.well-known/openid-configuration
directive and introduced the
<LocationMatch "/.well-known/openid-configuration">
ProxyPass http://localhost:8081/oxauth/.well-known/openid-configuration
Header set Access-Control-Allow-Origin "*"
</LocationMatch>
directive to add CORS header(s).
Other
Other things I've tried to figure out what the issue is:
Performed a GET request to the endpoint through Postman, which completed successfully.
Performed an OPTIONS request to the endpoint through Postman, which completed successfully.
I'd very much appreciate some input on this as it has me pretty stumped and being unable to work from local when developing is cumbersome. If any clarification is needed, please let me know.
Turns out this problem was an amalgamation of two unrelated things.
First, and this is mostly conjecture, it seems that Chrome blocks requests from a local file (the HTML file) and simply provides output that is, to me, very confusing. I.e. the 403 error might be because Chrome blocks the CORS request somehow. I tried running Chrome with various flags, e.g. --disable-web-security and --allow-file-access-from-files, but this did not change the output from the local HTML file. So, the local file request still fails and I don't really know the exact reason. But, since this was just for testing it is not that relevant, for me, currently.
Secondly, an erroneous implementation in an interceptor in the Angular project overwrote all headers for requests. After fixing this, the local server was able to request the endpoint.
It just so happened that the output from the two different issues looked pretty much identical which threw me off.

Apache load balancing with proxy_balancer

I am having difficulties configuring apache 2.4 to use it's proxy_balancer mod. My use case is fairly straight forward. Requests are reaching the apache node which should be load balanced to the cluster. This is my config:
<VirtualHost *:80>
ProxyRequests off
ServerName localhost
<Proxy balancer://geocode>
BalancerMember "http://192.168.2.11:8080/ors/status"
BalancerMember "http://192.168.2.35:8080/ors/status"
Require all granted
ProxySet lbmethod=byrequests
</Proxy>
<Location /balancer-manager>
SetHandler balancer-manager
Require all granted
</Location>
ProxyPass /balancer-manager !
ProxyPass "/geocodelb" "balancer://geocode"
ProxyPass "/geocode" "http://192.168.2.35:8080/ors/status"
</VirtualHost>
When navigating to localhost:80/geocodelb I receive 404 Not Found:
129.206.205.50 - - [01/Oct/2017:19:39:55 +0000] "GET /geocodelb
HTTP/1.1" 404 164 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12;
rv:56.0) Gecko/20100101 Firefox/56.0"
but localhost:80/geocode works:
129.206.205.50 - - [01/Oct/2017:19:40:07 +0000] "GET /geocode HTTP/1.1"
200 757 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:56.0)
Gecko/20100101 Firefox/56.0"
(The ../status points to a tomcat instance and returns a json object)
If I replace the url's of the BalancerMembers with something arbitrary - like google.com - it works.
I can also reach the balancer-manager at localhost:80/balancer-manager and obtain the correct information. If I am not mistaken it is telling me that everything looks ok (see screenshot). What am I missing here?
The reason for this not working is that the balancermember must be a url to a server, without a path. The path is to be added in the ProxyPass directive, like this balancer://orsbackend/ors/geocode. Here the working config:
<VirtualHost *:80>
ProxyRequests off
ServerName localhost
<Proxy balancer://orsbackend>
BalancerMember "http://192.168.2.11:8080"
BalancerMember "http://192.168.2.35:8080"
ProxySet lbmethod=byrequests
</Proxy>
<Location /balancer-manager>
SetHandler balancer-manager
Require all granted
</Location>
ProxyPass /balancer-manager !
ProxyPass "/geocode" "balancer://orsbackend/ors/geocode"
</VirtualHost>

Jenkins / Apache Reverse Proxy Error

I am running into an issue that seems to be fairly common based off of my searches, however I've followed all the instructions and/or fixes I've run into but none have worked for me so I'm asking this hoping someone can guide me in the right direction.
I have Jenkins 1.644 installed on OS X 10.11.2 from Homebrew. I followed these instructions on how to install and get it setup inside OS X Server 5.0.15 Websites (I believe this version of OS X server is running Apache 2.4.16.
The problem: When I connect to the manage console in Jenkins, I get the error message "It appears that your reverse proxy set up is broken." and a link to this jenkins doc.
Hitting http://127.0.0.1:8080/manage does not produce the error.
I have added the proxy settings to my virtual host file like so:
ProxyRequests Off
ProxyPreserveHost On
AllowEncodedSlashes NoDecode
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass / http://localhost:8080/ nocanon
ProxyPassReverse / http://localhost:8080/
ProxyPassReverse / http://jenkins.exampledomain.com/
RequestHeader set X-Forwarded-Proto "https"
RequestHeader set X-Forwarded-Port "443"
When I do the test curl:
curl -iLk -e https://jenkins.exampledomain.com/manage \
https://jenkins.exampledomain.com/administrativeMonitor/hudson.diagnosis.ReverseProxySetupMonitor/test
I get the following results:
HTTP/1.1 302 Found
Date: Fri, 22 Jan 2016 06:30:57 GMT
Server: Jetty(winstone-2.9)
X-Content-Type-Options: nosniff
Location: https://jenkins.exampledomain.com/administrativeMonitor/hudson.diagnosis.ReverseProxySetupMonitor/testForReverseProxySetup/https%3A%2F%2Fjenkins.exampledomain.com%2Fmanage/
Content-Length: 0
MS-Author-Via: DAV
HTTP/1.1 404 Not Found
Date: Fri, 22 Jan 2016 06:30:57 GMT
Server: Apache/2.4.16 (Unix) OpenSSL/0.9.8zg
Content-Length: 325
Content-Type: text/html; charset=iso-8859-1
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL /administrativeMonitor/hudson.diagnosis.ReverseProxySetupMonitor/testForReverseProxySetup/https://jenkins.exampledomain.com/manage/ was not found on this server.</p>
</body></html>
Clearly that address is on this server because I can enter the management console by going to the correct address.
I'm stuck... Apache configuration is not my strong point. I'm looking for any help.
--EDIT More Info--
Adding the full virtual host file from the /Library/Server/Web/Config/apache2/sites directory for further detail.
<VirtualHost 127.0.0.1:34543>
ServerName https://jenkins.exampledomain.com:443
ServerAdmin admin#example.com
DocumentRoot "/Library/Server/Web/Data/Sites/jenkins.exampledomain.com"
DirectoryIndex index.html index.php default.html
CustomLog /var/log/apache2/access_log combinedvhost
ErrorLog /var/log/apache2/error_log
<IfModule mod_ssl.c>
SSLEngine On
SSLCipherSuite "ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM"
SSLProtocol -ALL +TLSv1
SSLProxyEngine On
SSLCertificateFile "/etc/certificates/machine.local.certCA1FileLocation.pem"
SSLCertificateKeyFile "/etc/certificates/machine.local.certCA2FileLocation.key.pem"
SSLCertificateChainFile "/etc/certificates/machine.local.certCA3FileLocation.chain.pem"
SSLProxyProtocol -ALL +TLSv1
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
</IfModule>
<Directory "/Library/Server/Web/Data/Sites/jenkins.exampledomain.com">
Options All -Indexes -ExecCGI -Includes +MultiViews
AllowOverride None
<IfModule mod_dav.c>
DAV Off
</IfModule>
<IfDefine !WEBSERVICE_ON>
Require all denied
ErrorDocument 403 /customerror/websitesoff403.html
</IfDefine>
</Directory>
ProxyRequests Off
ProxyPreserveHost On
AllowEncodedSlashes NoDecode
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPass / http://localhost:8080/ nocanon
ProxyPassReverse / http://localhost:8080/
ProxyPassReverse / http://jenkins.exampledomain.com/
RequestHeader set X-Forwarded-Proto "https"
RequestHeader set X-Forwarded-Port "443"
</VirtualHost>
--EDIT 2 Another Finding--
I have noticed by attempting to curl to the 'not found' url above that indeed the server is reporting it not found. If I hit https://jenkins.exampledomain.com/manage/ I will get a 404. However, if I leave off the trailing /, it works. https://jenkins.exampledomain.com/manage is successful. Hopefully this means something to someone!
Thanks
I know this is an old question, but I was having the same problem with the error:
HTTP ERROR 404
Problem accessing /administrativeMonitor/hudson.diagnosis.ReverseProxySetupMonitor/testForReverseProxySetup/https%3A%2F%2Fbuild.scopeitconsulting.com%2Fmanage/. Reason:
http://build.domain.com/manage vs. https://build.domain.com/manage
I was able to solve my problem by including the two lines from the author's question:
RequestHeader set X-Forwarded-Proto "https"
RequestHeader set X-Forwarded-Port "443"
So here is my relevant section from a working ssl.conf configuration in case it helps anybody. I am running Jenkins on port 8080 at the root context with http but reverse proxying it behind Apache enforcing https.
ProxyPass / http://localhost:8080/ nocanon
ProxyPassReverse / http://localhost:8080/
ProxyPassReverse / http://build.domain.com/
ProxyPassReverse / https://build.domain.com/
ProxyRequests Off
ProxyPreserveHost On
AllowEncodedSlashes NoDecode
RequestHeader set X-Forwarded-Proto "https"
RequestHeader set X-Forwarded-Port "443"
<Proxy http://localhost:8080/>
Order deny,allow
Allow from all
</Proxy>
I hope this helps somebody who like me has spent way too much time trying to find a working configuration to resolve the error.
You need to add below to catalina.properties file. Updating Apache configuration itself is not sufficient.
org.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true