I have an application that has react in the front-end and a node service in the back-end. The app is deployed in the GKE cluster. Both the apps are exposed as a NodePort Service, and the fan out ingress path is done as follows :
- host: example.com
http:
paths:
- backend:
serviceName: frontend-service
servicePort: 3000
path: /*
- backend:
serviceName: backend-service
servicePort: 5000
path: /api/*
I have enabled authentication using IAP for both services. When enabling IAP for both the kubernetes services, new Client Id and Client Secret is created individually. But I need to provide authentication for the back-end API from the front-end, since they have 2 different accounts, its not possible, i.e when I call the back-end API service from the front-end the authentication fails because the cookies provided from the FE does not match in the back-end service.
What is the best way to handle this scenario. Is there a way to use the same client credentials for both these services and if so, Is that the right way to do it or Is there a way to authenticate the Rest API using IAP directly.
If IAP is setup using BackendConfig, then you can have two separate BackendConfig objects for frontend and backend applications but both of them use the same secrete (secretName) for oauthclientCredentials.
For frontend app
apiVersion: cloud.google.com/v1beta1
kind: BackendConfig
metadata:
name: frontend-iap-config
namespace: namespace-1
spec:
iap:
enabled: true
oauthclientCredentials:
secretName: common-iap-oauth-credentials
For backend app
apiVersion: cloud.google.com/v1beta1
kind: BackendConfig
metadata:
name: backend-iap-config
namespace: namespace-1
spec:
iap:
enabled: true
oauthclientCredentials:
secretName: common-iap-oauth-credentials
Then refer these BackendConfigs from respective kubernetes service objects
Related
I am currently trying to figure out how to enable istio to use a client certificate to authenticate to an external https service that requires client authentication. The client is a pod deployed in a kubernetes cluster that has istio installed. It currently accesses the external service using http, and cannot be changed. I know and have verified that istio can perform TLS origination so that the client can still use http to refer to the service, and istio will perform the TLS connection. But if the service also requires client certificate authentication, is there a way for me to configure istio to utilize a given certificate to do that?
I have tried by creating a ServiceEntry as described in some tutorials, as well as DestinationRules for that ServiceEntry. Is there a configuration in the DestinationRule, or elsewhere that will allow me to do that?
This is my current attempt. The hostname that requires client authentication is app.k8s.ssg-masamune.com. I have already verified that all the certificates I'm using appear to work through curl.
The certificates though are signed by a custom CA.
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: external-svc-https
spec:
hosts:
- api.dropboxapi.com
- www.googleapis.com
- developers.facebook.com
- app.k8s.ssg-masamune.com
- bookinfo.k8s.ssg-masamune.com
- edition.cnn.com
- artifactory.pds-centauri.com
location: MESH_EXTERNAL
ports:
- number: 80
name: http
protocol: HTTP
targetPort: 443
resolution: DNS
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: app-dr
spec:
host: app.k8s.ssg-masamune.com
trafficPolicy:
portLevelSettings:
- port:
number: 80
tls:
mode: SIMPLE
credentialName: app-secret
insecureSkipVerify: true
sni: app.k8s.ssg-masamune.com
subjectAltNames:
- app
Here's the situation: I have a bunch of applications that I am deploying to my k8s cluster:
a hasura backend (exposed with an nginx ingress)
some serverless functions hosted on an openfaas instance
an authentication service
a vuejs frontend application served on an nginx docker image (exposed with an nginx ingress)
My kubernetes cluster is hosted on Jelastic. I have enabled SSL with a nginx load-balancer in front of my cluster:
Inside of my cluster, I have the usual nginx-ingress-controller.
Now, I am busy trying to implement authentication properly. Following this advice, here's what I'm doing:
I have an auth server (fusionauth) deployed inside the cluster with service type ClusterIP
I have an openfaas function that performs user login: it accepts a username, a password, and an application id; it returns a JWT and issues a session cookie; the function talks to fusionauth and gets the JWT and refresh token
I have a hasura action login that calls the login openfaas function
I have a frontend application calling the hasura action login
Basically, my frontend served by an nginx docker image on the k8s cluster talks to the hasura api which talks to the internal openfaas login function which talks to the internal fusionauth. Both the frontend and the hasura api have an ingress defined:
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: admin-ui
namespace: services-108-staging
labels:
app.kubernetes.io/instance: admin-ui
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: admin-ui
app.kubernetes.io/version: '0.0'
helm.sh/chart: admin-ui-0.0.0
annotations:
meta.helm.sh/release-name: admin-ui
meta.helm.sh/release-namespace: services-108-staging
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Access-Control-Allow-Origin: $http_origin";
nginx.ingress.kubernetes.io/cors-allow-credentials: 'true'
nginx.ingress.kubernetes.io/cors-allow-methods: 'PUT, GET, POST, OPTIONS'
nginx.ingress.kubernetes.io/enable-cors: 'true'
spec:
rules:
- host: admin-staging.my-env.the-jelastic-host.com
http:
paths:
- path: /
backend:
serviceName: admin-ui
servicePort: 4000
---
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: api
namespace: services-108-staging
labels:
app.kubernetes.io/instance: api
app.kubernetes.io/managed-by: Helm
app.kubernetes.io/name: api
app.kubernetes.io/version: '0.0'
helm.sh/chart: api-0.0.0
annotations:
kubernetes.io/ingress.class: nginx
meta.helm.sh/release-name: api
meta.helm.sh/release-namespace: services-108-staging
spec:
rules:
- host: api-staging.my-env.the-jelastic-host.com
http:
paths:
- path: /
backend:
serviceName: api
servicePort: 8080
Only the frontal nginx load-balancer can access both the admin-staging.my-env.the-jelastic-host.com and the api-staging.my-env.the-jelastic-host.com URLs. The two applications are served through the frontal load-balancer on other domains, like admin-staging.my-host.com and api-staging.my-host.com respectively. The frontal load-balancer re-routes its incoming requests like this (e.g. for the admin-staging url):
server {
server_name admin-staging.my-host.com;
listen 443 ssl;
[...]
location / {
[...]
proxy_pass http://admin-staging.my-env.the-jelastic-host.com;
}
}
My goal is to get the session cookie stored in my frontend application. Currently, while I get the Set-Cookie header upon calling the login graphql mutation:
I don't get it in my frontend application's cookies:
I don't know exactly what I should set where to make that session cookie appear in my frontend's storage. Everything is served over https: the hasura api and the frontend application. Should I set a particular domain in my cookie? The cookie is issued from inside of the k8s cluster, from my serverless function hosted on
auth.services-108-staging.svc.cluster.local
(that's the result of the .Net Core call to Request.Host.ToUriComponent()). Should I set a particular domain in my cookie? Should I setup session affinity? On what ingress should I define session affinity? Is that enough to define session affinity in my cluster? What about the nginx load-balancer in front of my cluster? In essence, that load-balancer routes requests to the k8s cluster. Do I have to setup something special there?
Looks like session affinity is a good option to try first.
Could you please try to add the following annotations to all your ingress rules and check if that can help you to solve the issue?
nginx.ingress.kubernetes.io/affinity cookie
nginx.ingress.kubernetes.io/affinity-mode persistent
nginx.ingress.kubernetes.io/session-cookie-name hasura-backend
nginx.ingress.kubernetes.io/session-cookie-change-on-failure true
Alternatively, you can try to eliminate https2http termination that most likely is a reason that your FE application doesn’t see the cookie.
On a screenshot, it is seen that Set-Cookie header has a secure flag set. In order to eliminate SSL termination, you can attach Public IP addresses to your worker nodes of K8s, then activate SSL annotation in ingress and have a direct https connection to your frontend.
The Set-Cookie comes from a different domain, so the browser does nothing about it. The key phrase for your problem is "cross-origin set cookie".
I created a basic react/express app with IAP authentication and deployed to google app engine and everything works as expected. Now i'm moving from app engine deployment to kubernetes, all things work except the user authentication with IAP on kubernetes. How do I enable IAP user authentication with kubernetes?
Do I have to create a kubernetes secret to get user authentication to work? https://cloud.google.com/iap/docs/enabling-kubernetes-howto
Authentication code in my server.js https://cloud.google.com/nodejs/getting-started/authenticate-users#cloud-identity-aware-proxy
In order for Cloud IAP to be working with Kubernetes, you will need a group of one or more GKE instances, served by an HTTPS load balancer. The load balancer should be created automatically when you create an Ingress object in a GKE cluster.
Also required for enabling Cloud IAP in GKE: a domain name registered to the address of your load balancer and an App code to verify that all requests have an identity.
Once these requirements have been achieved, you can move forward with enabling Cloud IAP on Kubernetes Engine. This includes the steps to set up Cloud IAP access and creating OAuth credentials.
You will need to create a Kubernetes Secret to configure BackendConfig for Cloud IAP. The BackendConfig uses a Kubernetes Secret to wrap the OAuth client that you create when enabling the Cloud IAP.
You need to add an Ingress and enable IAP on the backend service
Create a BackendConfig object:
apiVersion: cloud.google.com/v1
kind: BackendConfig
metadata:
name: app-bc
spec:
iap:
enabled: true
oauthclientCredentials:
secretName: oauth-client-secret
attached it to the service:
apiVersion: v1
kind: Service
metadata:
name: app-service
annotations:
cloud.google.com/backend-config: '{"default": "app-bc"}'
cloud.google.com/neg: '{"ingress": true}'
spec:
type: ClusterIP
ports:
- port: 80
targetPort: 80
selector:
app: nginx
and then create the ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: app-ingress
spec:
tls:
- secretName: ingress-tls-secret
rules:
- host: ""
http:
paths:
- path: /*
pathType: ImplementationSpecific
backend:
service:
name: app-service
port:
number: 80
You can find the full tutorial here hodo.dev/posts/post-26-gcp-iap/
We are connecting Ambassador (API Gateway - https://www.getambassador.io/) to Google VM instance group via a load balancer where in http/2 is enabled. This requires ssl must be enabled. There is no proper information on how to connect Ambassador to an SSL enabled end system.
We tried connecting to Google VM instance from an Ambassador pod which is running in kubernetes via normal http service as per the suggestion - https://github.com/datawire/ambassador/issues/585. But could not find a way to connect to an SSL enabled endpoint by providing SSL certificate.
kind: Service
apiVersion: v1
metadata:
name: a-b-service
annotations:
getambassador.io/config: |
---
apiVersion: ambassador/v1
kind: Mapping
name: a-b-mapping
grpc: True
headers:
lang: t
prefix: /a.Listener/
rewrite: /a.Listener/
service: http://<ip>:<port>/
timeout_ms: 60000
We want to connect to an SSL enabled Google VM instance group via loadbalancing. Also, how to provide SSL certificate to this
kind: Service
apiVersion: v1
metadata:
name: a-b-service
annotations:
.....
service: https://<ip>:443/ <---- https with ssl
timeout_ms: 60000
Can someone suggest how to achieve this?
Ambassador has a fair amount of documentation around TLS, so probably your best bet is to check out https://www.getambassador.io/reference/core/tls/ and start from there.
The short version is specifying a service that starts with https:// is enough for Ambassador to originate TLS, but if you want to control the originating certificate, you need to reference a TLSContext as well.
I'm trying to expose a SignalR hub hosted in a Kubernetes (Azure) pod. Basically, the authentication and the handshake steps work fine, but when I trigger some action, all clients connected via the k8s Ingress doesn't receive the message. Has anybody experienced this issue or just have shared SignalR hubs through Kubernetes - Ingress?
ingress.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: endpoints
annotations:
kubernetes.io/ingress.class: addon-http-application-routing
ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.org/websocket-services: "myservice"
spec:
rules:
- host: api.[MY-DOMAIN].com
http:
paths:
- backend:
serviceName: myservice
servicePort: 80
path: /myservice
Try:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/affinity: cookie
nginx.ingress.kubernetes.io/session-cookie-hash: sha1
nginx.ingress.kubernetes.io/session-cookie-name: REALTIMESERVERID
I wrote a sample project a while back, if you want a working example: DenisBiondic/RealTimeMicroservices
As a side note, consider using Azure SignalR Service, it should remove many headaches (also in the example above).
Not familiar with SignalR but there could be a couple of things.
The nginx Ingress might be stripping some http headers that SignalR needs. Are you familiar with the http headers that the SignalR is supposed to receive?
After authenticating, is it possible that SignalR hub is trying to speak TLS? I see that you have this running on port 80 with not TLS. You would have to configure something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: endpoints
annotations:
kubernetes.io/ingress.class: addon-http-application-routing
ingress.kubernetes.io/ssl-redirect: "false" <== you may need to remove
nginx.ingress.kubernetes.io/ssl-redirect: "false" <== you may need to remove
nginx.org/websocket-services: "myservice"
spec:
rules:
- host: api.[MY-DOMAIN].com
http:
paths:
- backend:
serviceName: myservice
servicePort: 80
path: /myservice
tls:
- secretName: <your-tls-certs>
Hope it helps!