We have an app that uploads automatically generated SSL certificate to our NGINX load balancers. One time the we had this issue that a "bad certificate" got uploaded and then a automated nginx reload is thereafter executed, our server went offline for a while causing DNS issues (DNS not found) for our server domain. Causing a huge downtime to our clients.
However it is a feature / function in our application to allow apps to upload SSL cerficate and our backend server installs it automatically, is there a way to tell to ignore bad NGINX conf files and crt/key's altogether? Looking at the before logs I can remember that I saw something like SSL handshake error before the incident.
Here's how our main nginx-jelastic.conf looks like:
######## HTTP SECTION PROTOTYPE ########
http {
server_tokens off ;
### other settings hidden for simplicity
include /etc/nginx/conf.d/*.conf;
}
######## TCP SECTION PROTOTYPE ########
So what I am thinking if it's possible for nginx to just ignore all bad NGINX conf files that is located there. Here's a sample of what gets uploaded in the conf.d folder:
#
www.example-domain.com HTTPS server configuration
#
server {
listen 443 ssl;
server_name www.example-domain.com;
ssl_certificate /var/lib/nginx/ssl/www.example-domain.com.crt;
ssl_certificate_key /var/lib/nginx/ssl/www.example-domain.com.key;
access_log /var/log/nginx/localhost.access_log main;
error_log /var/log/nginx/localhost.error_log info;
proxy_temp_path /var/nginx/tmp/;
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
location / {
set $upstream_name common;
include conf.d/ssl.upstreams.inc;
proxy_pass http://$upstream_name;
proxy_next_upstream error;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Host $http_host;
proxy_set_header X-Forwarded-For $http_x_forwarded_for;
proxy_set_header X-URI $uri;
proxy_set_header X-ARGS $args;
proxy_set_header Refer $http_refer;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
For some reason the certificate and key indicated in the configuration could be wrong, that that is going to wreck the nginx server and since our domain is pointed to this server via A record, it us a total disaster if the nginx fails as DNS issues happens and it could take 24-48 hours for DNS to get back.
Related
I installed Harbor in a server inside the company farm and I can use it without problem through https://my-internal-server.com/harbor.
I tried to add the reverse proxy rules to Apache to access it through the public server for harbor, v2, chartrepo, service endpoints, like https://my-public-server.com/harbor, but this doesn't work.
For example:
ProxyPass /harbor https://eslregistry.eng.it/harbor
ProxyPassReverse /harbor https://eslregistry.eng.it/harbor
I also set in harbor.yaml:
external_url: https://my-public-server.com
When I try to access to https://my-public-server.com/harbor with the browser I see a Loading... page and 404 errors for static resources because it tries to get them with this GET:
https://my-public-server.com/scripts.a459d5a2820e9a99.js
How can I configure it to work?
You should pass the whole domain, not only the path. Take a look at the official Nginx config to have an idea how this might look like.
upstream harbor {
server harbor_proxy_ip:8080;
}
server {
listen 443 ssl;
server_name harbor.mycomp.com;
ssl_certificate /etc/nginx/conf.d/mycomp.com.crt;
ssl_certificate_key /etc/nginx/conf.d/mycomp.com.key;
client_max_body_size 0;
chunked_transfer_encoding on;
location / {
proxy_pass http://harbor/;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffering off;
proxy_request_buffering off;
}
Note that you should disable proxy or buffering
I created a Node.js site that uses Google authentication. The site is used by 100+ users concurrently which affect the performance. So I understand that Nginx could help in scaling the site by creating multiple instance of the Node.js app in multiple ports and then we use Nginx as a load balancer.
So, I configured Nginx, but the issue is that it dose not seem to work with Google authentication. I am able to see the first page of my site and I am able to to login via Google but it is dose not work after this point.
Any suggestions to what could be missing to make this work.
This is my configuration file:
upstream my_app
{
least_conn; # Use Least Connections strategy
server ip:3001; # NodeJS Server 2 I changed the actual ip
server ip:3002; # NodeJS Server 3
server ip:3003; # NodeJS Server 4
server ip:3004; # NodeJS Server 5
keepalive 256;
}
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
expires epoch;
add_header Cache-Control "no-cache, public, must-revalidate, proxy-revalidate";
server_name ip;
access_log /var/log/nginx/example.com-access.log;
error_log /var/log/nginx/example.com-error.log error;
# Browser and robot always look for these
# Turn off logging for them
location = /favicon.ico { log_not_found off; access_log off; }
location = /robots.txt { log_not_found off; access_log off; }
# pass the request to the node.js server
# with some correct headers for proxy-awareness
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_buffers 8 16k;
proxy_buffer_size 32k;
proxy_pass http://my_app ;
proxy_redirect off ;
add_header Pragma "no-cache";
# Handle Web Socket connections
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
I just started learning about nginx, I checked when the upstream have only one ip address and it is working. i.e it works as a reverse proxy but not as a load balancer and my guess is due to google authentication nature.
And the error I receive in the error log is connection refused.
Thanks.
I figure out what was wrong. least_conn load balancing technique was not the right to choose since it dose not persist session. I changed it to
hash $remote_addr or hash_ip and it is working.
I have a master nginx server deciding on the incoming server name where to route requests to. For two secondary servers this master nginx server is also holding ssl certificates and keys. The 3rd server is holding his own certificates and keys because there is a frequent update process for those.
My question is now how I can configure the master nginx server to forward all requests to server 3 which are coming in for this server. I cannot copy the certificates and keys from server 3 to the master server as they change too often.
Try to proxy the tcp traffic instead of the http traffic
stream {
server {
listen SRC_IP:SRC_PORT;
proxy_pass DST_IP:DST_PORT;
}
}
for more details refer to the nginx documentation
https://docs.nginx.com/nginx/admin-guide/load-balancer/tcp-udp-load-balancer/
Here's a configuration that might work. Proxy through the master and forward everything to Server3. Use the ssl port but turn ssl off.
server {
listen 443;
server_name myserver.mydomain.whatever;
ssl off;
access_log /var/log/nginx/myserver.access.log;
error_log /var/log/nginx/myserver.error.og;
keepalive_timeout 60;
location / {
set $fixed_destination $http_destination;
if ( $http_destination ~* ^https(.*)$ )
{
set $fixed_destination http$1;
}
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Destination $fixed_destination;
# Fix the “It appears that your reverse proxy set up is broken" error.
# might need to explicity set https://localip:port
proxy_pass $fixed_destination;
# force timeout if backend died.
proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
proxy_read_timeout 90;
proxy_redirect http:// https://;
}
}
I am using Ghost as a blogging platform and nginx as a reverse proxy for ghost as detailed in the documentation. Ghost is installed in a subdirectory and is served over the domain http://example.com/blog whereas the static website is served over example.com
I have set up SSL on my server and want to serve the ghost login page (example.com/blog/ghost) over SSL while serving the rest of the pages over normal HTTP. However if I use forceAdminSSL:true and try to go to http://example.com/blog/ghost it should automatically redirect me to https://example.com/blog/ghost. Instead I'm redirected to https://example.com/ghost and end up with 404 error. The only work around I have found that works is to use foreAdminSSL:{redirect:false} which is clumsy because then I have to manually type https in the address bar instead of http.
How do I server Ghost Admin panel over ssl while ghost is installed in a subdirectory? I guess this has something to do with configuration in nginx.
My nginx config block
server {
listen 80;
listen 443 ;
server_name *.example.com;
server_name example.com;
ssl on;
ssl_certificate /etc/nginx/ssl/certificate.crt;
ssl_certificate_key /etc/nginx/ssl/key.key;
location ^~/blog {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header X-NginX-Proxy true;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:2786;
proxy_redirect off;
}
location / {
root "/home/ubuntu/somedirectory/";
index index.html;
}
I think you haven't entered the config URL while setting up Ghost correctly.
You can do this by running the following commands:
ghost config URL https://my-domain.com/blog/
ghost restart
If this doesn't solve the problem, you can check out a detailed tutorial, solving this issue, on my blog here
I'm flummoxed.
I have a server that is primarily running couchdb over ssl (using nginx to proxy the ssl connection) but also has to serve some apache stuff.
Basically I want everything that DOESN'T start /www to be sent to the couchdb backend. If a url DOES start /www then it should be mapped to the local apache server on port 8080.
My config below works with the exception that I'm getting prompted for authentication on the /www paths as well. I'm a bit more used to configuring Apache than nginx, so I suspect I'm mis-understanding something, but if anyone can see what is wrong from my configuration (below) I'd be most grateful.
To clarify my use scenario;
https://my-domain.com/www/script.cgi should be proxied to
http://localhost:8080/script.cgi
https://my-domain.com/anythingelse should be proxied to
http://localhost:5984/anythingelse
ONLY the second should require authentication. It is the authentication issue that is causing problems - as I mentioned, I am being challenged on https://my-domain.com/www/anything as well :-(
Here's the config, thanks for any insight.
server {
listen 443;
ssl on;
# Any url starting /www needs to be mapped to the root
# of the back end application server on 8080
location ^~ /www/ {
proxy_pass http://localhost:8080/;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# Everything else has to be sent to the couchdb server running on
# port 5984 and for security, this is protected with auth_basic
# authentication.
location / {
auth_basic "Restricted";
auth_basic_user_file /path-to-passwords;
proxy_pass http://localhost:5984;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Ssl on;
}
}
Maxim helpfully answered this for me by mentioning that browsers accessing the favicon would trigger this behaviour and that the config was correct in other respects.