yesterday a make a new gitlab 8.17.0 installation on ubuntu 16.04 installation and confgure it like the old one which I have documented. I am running a apache2 which letsencrypt as a proxy. It looks as if everything runs well, but a few JS files are not loaded. I get "422 Unprocessable Entity" https at
domain/assets/webpack/application-XXX.js
domain/assets/webpack/lib_dX.js
domain/assets/webpack/users-X.js
domain/assets/webpack/lib_d3-X.js
domain/assets/webpack/users-X.js
/etc/gitlab/gitlab.rb
external_url 'https://gitlab.example.com'
nginx['listen_address'] = 'localhost'
nginx['listen_port'] = 8080
nginx['listen_https'] = false
web_server['external_users'] = ['www-data']
nginx['enable'] = false
/etc/apache2/sites-available/gitlab.example.com.conf
<VirtualHost *:80>
ServerName gitlab.example.com
ServerAdmin mail#gitlab.example.com
ErrorLog /var/log/apache2/gitlab.example.com/error.log
CustomLog /var/log/apache2/gitlab.example.com/access.log combined
Redirect 301 / https://gitlab.example.com/
</VirtualHost>
<VirtualHost *:443>
ServerName gitlab.example.com
ServerAdmin mail#gitlab.example.com
ErrorLog /var/log/apache2/gitlab.example.com/error.log
CustomLog /var/log/apache2/gitlab.example.com/access.log combined
RequestHeader set X-Forwarded-Proto "https"
RequestHeader set X-Forwarded-Port "443"
ProxyPreserveHost On
ProxyPass / http://localhost:8080/
ProxyPassReverse / http://localhost:8080/
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/gitlab.example.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/gitlab.example.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/gitlab.example.com/chain.pem
<Proxy http://localhost:8080/>
Order deny,allow
Allow from all
</Proxy>
</VirtualHost>
Does anyone have ideas to solve the problem?
Thank you! I wish you a nice Sunday.
I had same problem with serving mentioned static files but behind nginx proxy. In /var/log/gitlab/gitlab-rails/production.log file there were errors:
Started GET "/assets/webpack/application-0b895f7016d93748393a-v2.js" for 127.0.0.1 at 2017-03-05 11:14:21 +0100
Processing by ApplicationController#route_not_found as JS
Parameters: {"unmatched_route"=>"assets/webpack/application-0b895f7016d93748393a-v2"}
Security warning: an embedded <script> tag on another site requested protected JavaScript.
If you know what you're doing, go ahead and disable forgery protection on this action to
permit cross-origin JavaScript embedding.
Completed 422 Unprocessable Entity in 28ms (ActiveRecord: 1.8ms)
ActionController::InvalidCrossOriginRequest (Security warning: an embedded
<script> tag on another site requested protected JavaScript. If you know
what you're doing, go ahead and disable forgery protection on this action to
permit cross-origin JavaScript embedding.):
lib/gitlab/middleware/multipart.rb:93:in `call'
lib/gitlab/request_profiler/middleware.rb:14:in `call'
lib/gitlab/middleware/go.rb:16:in `call'
So I've made changes in /opt/gitlab/embedded/service/gitlab-rails/config/environments/production.rb:
config.action_controller.allow_forgery_protection = false
# Disable Rails's static asset server (Apache or nginx will already do this)
config.serve_static_files = true
and restarted gitlab (gitlab-ctl restart). Now it works, however I suppose that security of my server is weakened.
This is an issue in 8.17 thas hasn't been fixed yet.
Just downgrade to 8.16 sudo apt-get install gitlab-ce=8.16.6-ce.0
Related
I have a problem. On my local laptop I can build my docker image and run it. A Ruby-On-Rails application is then running on localhost:3000. Going to that page gives me a login screen and I can successfully login. Now when I publish this code to my docker repository and run it in a kubernetes cluster, I get the following error:
With status code:
Status Code: 422 Unprocessable Entity
After some googling and debugging, I read some stuff about having the correct headers in Nginx-Ingress and Apache to handle the CSRF requests. Using this I tried many ways of allowing headers and manually adding RequestHeaders, but all resulted in the same error.
In both pages I can see in the network tab the cookie in the page. The cookie in local environment contains:
csrftoken=7tzlvuigfaUatDBMTEJUBE53OREXHxOJ1ZzJ8h4cuHvnvUza0xLH2lglIFZ1gS4S; __profilin=p=t; _app_session=BBr2bnKi6qYJwW4yvQUU0J3Q2K8JW2i9DZ5zJS0qQmCc48uRNbPrThJhbzhnGXrxLDwD/Y8ZAW5w/x2VhXeotq3TWjWcjVucUzNPA6u6ePMcN8O7XVfcIwV2HR/aGpJ1oxB1B/TbsAcfB4LaSqckVWMZ70anWDOD+xFz0xkKPiOkd++NwnMuWix2TyaicqXmQ+JsK1yWacS2dnTWvC70xyX38JjuC/VUqu2hUFIQHeNuK7Gfp7lMCh0MEJeob43lFhgFZLOJXTC+rPYIUxMTN3aIwD0=--LsM3EqEzNEglvj6H--RHcx/5CRIYG/CWJMUER1eQ==
In my production environment, I can see that the entire csrftoken part is missing:
__profilin=p=t; _app_session=FWpoSIJ8zzB8nkRsF2A30f552HjyAVlwbMva/HN4vnKaTECnPWzfWoc/+P9k/xO+fpmd2j9ntxLFOnflB5vttwxpLL1RtjZWcPmpCWfdR0nqG3EOLPLK9yR0xrvmfKMbFsYeJ2F/hZrJIs+4iOO92CioXo1s51sZX5E3R6c9XIKn5NjHUKWKuEArVV/4MibTg/CSqPWyH3lg1bjxKoZvixSZUBdtxTq1sW5NKsJnnwPgC0ZSkQzsSquCwcxKEf9L1wPxgZQTLGRREY7JeD/+mUc0WL4=--n+JNO4hoF17EGI5K--vY2DcanQEF7ICpZwDxq9tg==
Why is the CSRF token not comming through in a production environment? I already tried setting RAILS_ENV to development in the production environment, so it uses a different config, but that also gives the Status Code: 422 Unprocessable Entity
IMPORTANT
All trafic goes to my portal proxy server, which runs Apache2 and handles the SSL certificates for my domain. Apache forwards the traffic in my internal network using HTTP. Nginx-Ingress-Controller only handles http traffic because of this setup. I am not really sure if Nginx is the problem then, or Apache2?
How can I fix this CSRF issue ASAP?
UPDATE
I looked into the rails logs and found the following which might cause the problem:
I, [2022-12-12T08:16:40.202890 #1] INFO -- : [f785b7f7f797c67c0ab0f008e8a0eb83] Started POST "/login" for 192.168.1.10 at 2022-12-12 08:16:40 +0000
I, [2022-12-12T08:16:40.204343 #1] INFO -- : [f785b7f7f797c67c0ab0f008e8a0eb83] Processing by AuthController#login as HTML
I, [2022-12-12T08:16:40.204460 #1] INFO -- : [f785b7f7f797c67c0ab0f008e8a0eb83] Parameters: {"authenticity_token"=>"[FILTERED]", "login"=>{"email"=>"myemail#gmail.com", "password"=>"[F
W, [2022-12-12T08:16:40.204795 #1] WARN -- : [f785b7f7f797c67c0ab0f008e8a0eb83] HTTP Origin header (https://my.site.com) didn't match request.base_url (http://my.site.com)
I, [2022-12-12T08:16:40.205536 #1] INFO -- : [f785b7f7f797c67c0ab0f008e8a0eb83] Completed 422 Unprocessable Entity in 1ms (ActiveRecord: 0.0ms | Allocations: 192)
F, [2022-12-12T08:16:40.207591 #1] FATAL -- : [f785b7f7f797c67c0ab0f008e8a0eb83]
In my Ingress resource I now have the following:
annotations:
nginx.org/proxy-connect-timeout: 3600s
nginx.org/proxy-read-timeout: 3600s
nginx.org/proxy-send-timeout: 3600s
nginx.org/proxy-pass-headers: "Set-Cookie"
nginx.ingress.kubernetes.io/proxy-body-size: "8g"
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Host $http_host";
more_set_headers "X-Real-IP $remote_addr";
more_set_headers "X-Forwarded-Proto $scheme";
more_set_headers "X-Forwarded-For $proxy_add_x_forwarded_for";
And my apache conf looks like this:
<VirtualHost *:80>
ServerName my.site.com
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
ProxyPreserveHost on
ProxyPass / http://192.168.1.8/
ProxyPassReverse / http://192.168.1.8
RewriteEngine on
RewriteCond %{SERVER_NAME} =my.site.com
RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]
</VirtualHost>
<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerName my.site.com
ProxyRequests Off
<Proxy *>
Order deny,allow
Allow from all
</Proxy>
SSLProxyEngine On
SSLProxyCheckPeerCN on
SSLProxyCheckPeerExpire on
ProxyPreserveHost on
ProxyPass / http://192.168.1.8/
ProxyPassReverse / http://192.168.1.8
Include /etc/letsencrypt/options-ssl-apache.conf
ErrorLog /var/log/apache2/sites/my.site.com/error.log
CustomLog /var/log/apache2/sites/my.site.com/access.log combined
SSLCertificateFile /etc/letsencrypt/live/my.site.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/my.site.com/privkey.pem
</VirtualHost>
</IfModule>
I already tried adding to my Apache conf:
RequestHeader set X-Forwarded-Proto "https"
but that doesn't change anything...
Can someone tell me what I should change here to get this to work?
I would propose adding the origin header in nginx. You probably also have to set it in your apache configuration to the external URL.
annotations:
nginx.org/proxy-connect-timeout: 3600s
nginx.org/proxy-read-timeout: 3600s
nginx.org/proxy-send-timeout: 3600s
nginx.org/proxy-pass-headers: "Set-Cookie"
nginx.ingress.kubernetes.io/proxy-body-size: "8g"
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Host $http_host";
more_set_headers "X-Real-IP $remote_addr";
more_set_headers "X-Forwarded-Proto $scheme";
more_set_headers "X-Forwarded-For $proxy_add_x_forwarded_for";
more_set_headers "origin $remote_addr";
I know, I know, Apache is not the best tool to use as HTTP proxy, however I need it on my server.
Here's my virtual host configuration:
<VirtualHost *:*>
RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}
</VirtualHost>
<VirtualHost *:80>
ServerName my.public.domain
Redirect / https://my.public.domain/
</VirtualHost>
<VirtualHost *:443>
ProxyPreserveHost On
ProxyPass / http://127.0.0.1:50001/
ProxyPassReverse / http://127.0.0.1:50001/
ServerName my.public.domain
ErrorLog ${APACHE_LOG_DIR}my-app-error.log
CustomLog ${APACHE_LOG_DIR}my-app-access.log common
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateFile /etc/letsencrypt/live/my-cert/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/my-cert/privkey.pem
</VirtualHost>
In UseUrls method i have http://localhost:50001 configured as main URL, and this is redirected by Apache to HTTPS #443.
It works as charm, however I see this in logs:
warn: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[3]
Failed to determine the https port for redirect.
In my configuration Apache handles https traffic, BTW, I can't communicate my app with Apache locally over HTTPS, it just doesn't work. I also think it's pointless to encrypt local internal traffic.
Unfortunately my solution requires some hacking to work 100% properly - I need to provide my public site URL in my app configuration - otherwise the app doesn't know what it's external address is. I mean - I build some links manually, because this is the core of my question - I don't know where the framework would keep such information. For example NavigationManager thinks my site URL is "http://localhost:50001", so if I need absolute URL in my app I can't use NavigationManager directly, I need to "manually" create the URL in app.
Links generated by Identity have "http" instead of "https", but it works because apache redirects everything to https.
Is there a way (and HOW) to do it more properly - a mean to officially tell the AspNET.Core it has specific external URL?
Where you have ServerName my.public.domain, use the following:
For port 80:
ServerName http://my.public.domain:80
For port 443:
ServerName https://my.public.domain:443
Since integrating CsrfPreventionRequestCycleListener into our Apache Wicket (7.6.0) application, we have a problem operating the application behind an Apache reverse proxy.
Our configuration terminates SSL at Apache, and the reverse proxy passes the requests via http to our Wildfly 10 application server. This allows us to offload TLS/SSL among other things.
But since adding the CsrfPreventionRequestCycleListener, we are seeing the following in the server.log file and connections are aborted:
[org.apache.wicket.protocol.http.CsrfPreventionRequestCycleListener]
(default task-12) Possible CSRF attack, request URL:
http://example.com/example/portal/wicket/page,
Origin: https://example.com, action: aborted with error
400 Origin does not correspond to request
The problematic Apache config:
<VirtualHost example.com:443>
ServerName example.com
LogLevel debug
SSLEngine On
SSLCertificateFile /var/example/example.com/signed.crt
SSLCertificateKeyFile /var/example/example.com/domain.key
SSLCACertificateFile /var/example/example.com/intermediate.pem
SSLProtocol +TLSv1.2 +TLSv1.1 +TLSv1
SSLOpenSSLConfCmd DHParameters "/usr/local/apache2/1024dhparams.pem"
SSLProxyEngine on
ProxyPass / http://localhost:8390/ timeout=600
ProxyPassReverse / http://localhost:8390/ timeout=600
ProxyPreserveHost On
Header set Access-Control-Allow-Origin "*"
Header set Access-Control-Allow-Credentials "true"
Header edit Location ^http(\:\/\/.*)$ https$1
We found a solution using http2 but would prefer one without http2 (for reasons, see in https://http2.pro/doc/Apache).
The working Apache configuration using http2
<VirtualHost example.com:443>
ServerName example.com
LogLevel debug
SSLEngine On
SSLCertificateFile /var/example/example.com/signed.crt
SSLCertificateKeyFile /var/example/example.com/domain.key
SSLCACertificateFile /var/example/example.com/intermediate.pem
SSLProtocol +TLSv1.2 +TLSv1.1 +TLSv1
SSLOpenSSLConfCmd DHParameters "/usr/local/apache2/1024dhparams.pem"
SSLProxyEngine On
ProxyPreserveHost On
# Settings for http2 communication
Protocols h2 http/1.1
ProxyPass / https://localhost:8754/ timeout=600
ProxyPassReverse / https://localhost:8754/ timeout=600
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off
SSLProxyCheckPeerExpire off
Header set Access-Control-Allow-Origin "*"
Header set Access-Control-Allow-Credentials "true"
# Header edit Location ^http(\:\/\/.*)$ https$1
</VirtualHost>
Can anyone help us create a valid apache reverse proxy configuration that works with the CsrfPreventionRequestCycleListener without the http2 module?
I think I had a similar issue here and solved it just now. I think you might have solved your issue by now but maybe anyone else is falling over this topic.
You might want to look into a network dump to verify the problem. For me, apache sent requests to the locally running service (your case to http://localhost:8754) while some request headers were still referring the public name https://example.com. This was detected as a security risk and the connection refused by the underlying service.
I made sure, that mod_headers was enabled in apache and added the line
RequestHeader edit Referer "https://example.com" "http://localhost:8754"
After that no more references to anything related to example.com was in my tcp dumps and the connection was opened successfully.
You might need to adopt according to your setup (maybe you need multiple headers corrected or something similar). I cannot try it out at the moment.
I have an httpd server configured and working. The server returns "It works" and I can see the SSL is installed correctly.
The next step I undertook was configuring the reverse proxy, so that the users requests are redirected and I can have more customers' apps under one subdomain. The httpd configuration (shown below) I use is not mine, I am just attempting to reconfigure it to work for me. But with no big success up to now. There are directives that may be incorrect, but I have not tried commenting anything out.
#Apache is listening on port 443
Listen 443
SSLSessionCache shmcb:c:/Apache24/logs/shmcb_cache(512000)
SSLSessionCacheTimeout 300
Mutex default ssl-cache
SSLRandomSeed connect builtin
SSLCryptoDevice builtin
<VirtualHost *:443>
#ProxyPreserveHost On
SSLProxyEngine On
ServerName XXXX.sk
ServerAdmin admin
# Logs
ErrorLog /var/log/rsk_error_log
TransferLog /var/log/rsk_access_log
# Server Certificate and Private Key:
SSLCertificateFile /ssl/certificate.crt
SSLCertificateKeyFile /ssl/private.key
SSLCertificateChainFile /ssl/chain.crt
#Include conf/extra/proxy-443-to-8890.conf
ProxyPass /customer http://172.17.0.4:8080
ProxyPassReverse /customer http://172.17.0.4:8080
</VirtualHost>
Now when I type XXXX.sk/customer I receive a response which is a login screen, but it is incorrectly rendered, the CSS is not used at all. There are many errors appearing. When I log in, no response is returned and the URL is corrupted.
Can any of you, using httpd in a reverse proxy mode, please share your configs, at least a part of them?
The default webroot in base-component is mapping to / in url. All of resources like css, js etc is using "/" to build url, So Although the proxying is
ProxyPass /customer http://172.17.0.4:8080
The actual js location is still
/lib/jquery/jquery-ui.min.css
not
/customer/lib/jquery/jquery-ui.min.css
To make it work, the reverse proxy would need more messy location proxy_pass configurations.
So using additional path to proxy the webroot is not suggested.
I have the interactive slider example, sliders.py, running locally on my server with no issues. However, when I try to use Apache 2.4.7 to set up a reverse proxy between bokeh.example.blah and http://127.0.0.1:5006/sliders, the page on a remote computer loads without the interactive element, although HTML elements like the page title do load.
I run sliders.py with this command
bokeh serve sliders.py --host bokeh.example.blah
A tornado 404 warning pops up when I try to access bokeh.example.blah:
INFO:bokeh.command.subcommands.serve:Starting Bokeh server on port 5006 with applications at paths ['/sliders']
DEBUG:bokeh.server.tornado:[pid 7611] 0 clients connected
DEBUG:bokeh.server.tornado:[pid 7611] /sliders has 0 sessions with 0 unused
INFO:tornado.access:200 GET /sliders (127.0.0.1) 43.49ms
WARNING:tornado.access:404 GET /slidersstatic/css/bokeh.min.css?v=25ce17349a0082e9eeac845e42a822ec (127.0.0.1) 0.84ms
WARNING:tornado.access:404 GET /slidersstatic/css/bokeh-widgets.min.css?v=3b73ec63df304369ffc9c7e5132463cd (127.0.0.1) 0.78ms
WARNING:tornado.access:404 GET /slidersstatic/js/bokeh.min.js?v=50ffae624cbe67773b5995270f946406 (127.0.0.1) 0.69ms
WARNING:tornado.access:404 GET /slidersstatic/js/bokeh-widgets.min.js?v=7cfd458197bc395bb09e9dc3b8ac7975 (127.0.0.1) 0.71ms
WARNING:tornado.access:404 GET /slidersstatic/js/bokeh-compiler.min.js?v=28af56ac5027aa8d0ab96086ec1da10b (127.0.0.1) 0.74ms
I this is my 000-default.conf
<VirtualHost *:80>
ProxyHTMLEnable On
ProxyPreserveHost On
ServerName bokeh.example.blah
ProxyPass / http://127.0.0.1:5006/sliders
ProxyPassReverse / http://127.0.0.1:5006/sliders
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>
Interestingly, no apache error log is created when this happens.
Googling tells me that serving dynamic content with apache through a reverse proxy is sometimes problematic- this led me to add the 'ProxyHTMLEnable On' to my .conf file. If anyone has seen this sort of issue before and has advice it would be much appreciated!
I also notice that there seem to be a few places that indicate that documnetation about how to deploy bokeh apps is pending. For instance, the empty Deploying for Production section here, or this issue. Does information about this already exist and I'm just looking in the wrong place?
Thanks!
Update:
I now have a different error.
The previous error was due to the fact that I hadn't specified where bokeh keeps its static files. I did so by modifying '000-default.conf' to this:
<VirtualHost *:80>
ServerName bokeh.example.blah
ProxyPreserveHost On
ProxyPass /sliders/ws ws://127.0.0.1:5100/sliders/ws
ProxyPassReverse /sliders/ws ws://127.0.0.1:5100/sliders/ws
ProxyPass /sliders http://127.0.0.1:5100/sliders/
ProxyPassReverse /sliders http://127.0.0.1:5100/sliders/
<Directory />
Require all granted
Options -Indexes
</Directory>
Alias /static /usr/local/lib/python2.7/dist-packages/bokeh/server/static
<Directory /usr/local/lib/python2.7/dist-packages/bokeh/server/static>
# directives to effect the static directory
Options +Indexes
</Directory>
</VirtualHost>
As before, I can load only the title of the HTML page, but not the interactive plot. Now, the java console gives me this error:
WebSocket connection to 'ws://bokeh.example.blah/sliders/ws?bokeh-protocol-version=1.0&bokeh-session-id=pyrxrMepBAnQyAdl4UpACinDdhxcegz0SWtNnonrLngM' failed: Error during WebSocket handshake: Unexpected response code: 500
bokeh.min.js?v=50ffae6…:62 Bokeh: Failed to connect to Bokeh server Error: Could not open websocket
I would really appreciate advice!