I am setting a reverse proxy using kubernetes nginx-ingress, but I don't know how to add nginx parameters to the configuration, specifically: proxy_ssl_server_name. How do I set ingress parameters in yaml configurations?
I already tried using the server-snippet annotation, but it seems like it's not adding the parameter to the nginx.conf file in the cluster pods.
Here is the current code for the reverse proxy:
kind: Service
apiVersion: v1
metadata:
name: formstack
namespace: serves
spec:
type: ExternalName
externalName: fluidsignal.formstack.com
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: formstack
namespace: serves
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/upstream-vhost: "fluidsignal.formstack.com"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
tls:
- hosts:
- fluidattacks.com
secretName: fluidattacks-cert
rules:
- host: fluidattacks.com
http:
paths:
- backend:
serviceName: formstack
servicePort: 443
path: /forms(.*)
After setting up the proxy, I get a 502 Bad Gateway error from Nginx. After looking at the pods logs, I see I'm getting the following openssl error: SSL: error:14094410:SSL routines:ssl3_read_bytes:sslv3 alert handshake failure:SSL alert number 40, which is why I want to add the parameter I mentioned before.
I just figured out that I was indeed using the right annotation: nginx.ingress.kubernetes.io/server-snippet,
But I needed to add an extra parameter: proxy_ssl_name
Adding the following code fixed the problem:
nginx.ingress.kubernetes.io/server-snippet: |
proxy_ssl_name fluidsignal.formstack.com;
proxy_ssl_server_name on;
Everything seems to be working fine now :D
Related
I have following problem. I have deployment and service, frontend config and ingres as follows (skiping deployment as it is not really interesting):
---
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
name: backend-config
spec:
healthCheck:
type: HTTP
requestPath: /readiness
port: 8080
---
apiVersion: v1
kind: Service
metadata:
annotations:
cloud.google.com/app-protocols: '{"http":"HTTP"}'
cloud.google.com/backend-config: '{"default": "backend-config"}'
name: app
labels:
app: app
spec:
type: ClusterIP
selector:
app: app
ports:
- name: http
port: 80
targetPort: http
protocol: TCP
---
apiVersion: networking.gke.io/v1beta1
kind: FrontendConfig
metadata:
name: frontend-config
spec:
redirectToHttps:
enabled: true
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: 'app-ip'
networking.gke.io/v1beta1.FrontendConfig: 'frontend-config'
ingress.gcp.kubernetes.io/pre-shared-cert: 'ssl-certificate'
kubernetes.io/ingress.allow-http: 'false'
labels:
app: app
spec:
# tls:
# - secretName: tls-secret
rules:
- host: myhost.com
http:
paths:
- path: /*
pathType: Prefix
backend:
service:
name: app
port:
name: http
as you can see there is static IP address and ssl-certificate, which I registered with GCP.
By this configuration I am getting mostly
OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to ...
if I use curl and "Remote host terminated the handshake" with java clients. But sporadically packets come through. It has something to do with change of replicas. As you can see above I also tried to use kubernetes secret before. Had the same effect.
Does anybody had the same problem or maybe somebody has any clue what I am doing wrong?
Just to prevent the question with ssl. As you can see certificate is added as self managed certificates and is valid. Also it worked within other environment. In the cert part is complete chain stored till the root certificate.
Thank you in advance.
UPDATE:
I am using wildcard certificate. Means I have sub.domain.com covered by *.domain.com certificate. Tried to change configuration in ingress like this:
spec:
tls:
- hosts:
- telematics.tranziit.com
secretName: tranziit-tls-secret
Here how the certificate looks in GCP:
Certificate in GCP
with no success - same effect. I used this certificate as self managed when I was using VMs and normal external load balancer - no problem.
UPDATE 2:
In between I removed completely backend config and frontend config. So service and ingress are looking as follows:
---
apiVersion: v1
kind: Service
metadata:
name: app-service
labels:
app: app-service
spec:
type: ClusterIP
selector:
app: app
ports:
- name: http
port: 80
targetPort: http
protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: 'app-ip'
kubernetes.io/ingress.allow-http: 'false'
labels:
app: app
spec:
tls:
- hosts:
- sub.domain.com
secretName: tls-secret
rules:
- host: sub.domain.com
http:
paths:
- path: /api
pathType: Prefix
backend:
service:
name: app-service
port:
name: http
names are changed of course. But the process was as follows:
As soon as ingress was green I started several times curl -v https://mysubdomain-address and I have got at first several answers as expected with expected responses and in verbose log of curl I could see that handshakes are done. But then after third time or so I have got the issue again.
curl -v https://mydomain/path
* Trying XX.XXX.XXX.XX:443...
* Connected to mydomain (34.149.251.22) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to mydomain:443
* Closing connection 0
curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to mydomain:443
That what I mean that it is no stable result. I can see that packets comming through also because this interface receives messages and pushes them to pub/sub and on the other end I have a function. So I can see that the function gets invoked as following:
Function Invocations
I really do not know what shall I try else. Only thing would be to change to managed non-wildcard certificate from GCP. It is IMHO only thing to try.
It looks like the answer was - use NodePort as it described in documentation. Since I have changed service type to NodePort ingress is running stable. Unfortunatelly, google statement that there is no limitation and ingress can work with NodePort or with ClusterIP through proxies is not realy confirmed. In my particular case ingress was created with ClusterIP backend but was instable in particular during the tls handshake.
Thanks to #boredabdel for mental support :)
I need advice on how to configure nginx ingress controller and keycloak on eks kubernetes with TLS.
My first attempt was to get keycloak working without TLS. This works fine. But when I add TLS to the ingress definition, keycloak gives a Invalid parameter: redirect_uri error.
This is the working sample:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: keycloak-tls-test
namespace: keycloak-tls-test
spec:
rules:
- host: keycloak.test.myhops.com
http:
paths:
- backend:
serviceName: keycloak
servicePort: 80
When I add the tls part, I get the aforementioned error. See yaml file below.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: keycloak-tls-test
namespace: keycloak-tls-test
spec:
tls:
- hosts:
- keycloak.test.myhops.com
secretName: test-myhops-tls
rules:
- host: keycloak.test.myhops.com
http:
paths:
- backend:
serviceName: keycloak
servicePort: 80
Any suggestions on how to solve this? One online suggestion was to add https://keycloak.test.myhops.com/* to the Valid Redirect URL for the security-admin-console client in the master realm. This solved the invalid redirect_uri error, but then I only got blank pages. I could not find hints in the keycloak logging.
I figured out what caused the problem. It seems that when keycloak is used behind a reverse proxy that terminates the TLS, then keycloak needs proxy-address-forwarding to be set. For the docker container this means that the env var PROXY_ADDRESS_FORWARDING=true does the trick. So Ingress was correct and the keycloak deployment needed an extra environment variable.
Please not that ingress-nginx already set the correct headers for this to work:
X-Forwarded-For
X-Forwarded-Proto
Docker documentation
[keycloak documentation2
I have a Spring Boot application responsible for getting citizens and a NGINX Ingress controller configured with ssl-passthrough to expose my cluster to outside.
When I do:
https://myhostname.com/citizens
It perfectly works. This is the configuration I am using:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: myhostname.com
http:
paths:
- path: /
backend:
serviceName: myservice
servicePort: 443
But I would like to have something like:
https://myhostname.com/myservice/citizens
It would be something similar to math Simple fanout approach that is described in the kubernetes docs:
https://kubernetes.io/docs/concepts/services-networking/ingress/#simple-fanout
Therefore, in my Spring application I have configured a contextPath so when locally I do:
https://localhost:8080/myservice/citizens
I get the desired result
For NGINX I have set up the following:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: myhostname.com
http:
paths:
- path: /myservice
backend:
serviceName: myservice
servicePort: 443
What am I doing wrong?
Solution:
I've been researching a little and I've realized that the approach that I was trying to achieve is not possible with ssl-passthrough as the proxy is blind and it doesn't know the path to where route the traffic.
Therefore, the solution would be to use subdomains and not paths, because thanks to the SNI protocol, the NGINX controller will know to which hostname the client is trying to connect.
The final solution would be something similar to this
If your backend is expecting the whole path /myservice/citizens you can just use it to point it to the service without the rewrite-target directive, since it only dictates how URIs are going to be treated in the backend:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
rules:
- host: myhostname.com
http:
paths:
- path: /myservice/citizens
backend:
serviceName: myservice
servicePort: 443
Be aware that this assumes that your service is able to redirect requests to 8080, which is your backend listening port, so it has to match it.
I am trying to setup oauth2_proxy on kubernetes to secure one of my single page application and I am using github as the authentication provider. I found this link useful and I followed this for doing my setup.
[https://alikhil.github.io/2018/05/oauth2-proxy-for-kubernetes-services/]
I have created an application on github and used my DNS Name in place of the HomePageURL and CallBackURL (https://auth.example.com replaced with https://example.com) because I do not have TLS secrets generated for auth.example.com. Rather I have TLS certificates generated for example.com because this domain belongs to me. I was getting error in nginx-controller that the certificate belongs to example.com and not to auth.example.com as these URLs have been used in defining the example Ingress and oauth proxy ingress and this was the basis for me to do the before mentioned chang.
My Ingresses looks like this
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: oauth2-proxy
annotations:
kubernetes.io/tls-acme: "true"
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: oauth2-proxy
servicePort: 4180
path: /oauth2
tls:
- hosts:
- example.com
secretName: oauth-proxy-tls
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: oauth-main-ingress
annotations:
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
ingress.kubernetes.io/auth-url: https://example.com/oauth2/auth
ingress.kubernetes.io/auth-signin: https://example/oauth2/start?rd=https://$host$request_uri$is_args$args
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: example-service
servicePort: 80
path: /
tls:
- hosts:
- example.com
secretName: tls-secret
I want that whenever I click example.com it should display page for github authentication but in my case its directly giving the response which a service should give after successful authentication. I am not being asked to provide credentials. Also, I am getting error in my ingress controller logs as 7 controller.go:753] Error obtaining Endpoints for Service "default/oauth2-proxy": no object matching key "default/oauth2-proxy" in local store
Also, I tried replacing using nginx.ingress.kubernetes.io/auth-url: http://oauth2-proxy.oauth-proxy.svc.cluster.local:4180/oauth2/auth as mentioned in the link but it did not work for me. Can some one explain why is oauth2_proxy not asking for authentication and ingress is serving the requests directly without asking for authentication?
The annotation declared in the oauth-main-ingress yaml is incorrect.
As per the kubernetes/nginx-ingress documentation the annotation for external auth-url should be
nginx.ingress.kubernetes.io/auth-url: http://<oauth-service-url>
instead of
ingress.kubernetes.io/auth-url: http://<oauth-service-url>
I have configured an ingress on a kubernetes cluster from Google Cloud. I followed the instructions as is on this doc.
My manifest:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: web-proxy
annotations:
ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
tls:
- hosts:
- foo.bar.com
secretName: foo-bar-secret
rules:
- host: foo.bar.com
http:
paths:
- backend:
serviceName: foo-bar-service
servicePort: 80
path: /*
From the documentation, it specifies that ssl-redirect
is by default true if tls is configured but it doesn't work. I've also tried to set force-ssl-redirect to true(default is false), still no luck.
I'm confused if there is actually a bug or I'm doing something wrong, because when I run kubectl describe ing web-proxy, it displays that annotations were set:
Annotations:
force-ssl-redirect: true
[...]
Any suggestions?
EDIT 1:
I was using the 1.8.3.gke.0 version but when I upgraded to 1.8.4.gke.0 this seems to have solved the problem.
I'll keep this post updated while I test more annotations(like: nginx.ingress.kubernetes.io/from-to-www-redirect)