We are using - ingress-controller(nginx) as elb on aws.
for the server we have nodeJs app(express) on kubernetes.
Also we store the session on redis.
On client side the - the client didn't got an cookies from server.
Please need help with the configuration that solve this bug.
The app.js:
app.enable('trust proxy');
app.use(session({
//store: new RedisStore({ client: redisClient }),
secret: 'To-change!',
domain: sessionDomain,
resave: true,
proxy: true,
saveUninitialized: true,
cookie: {
sameSite: 'none',
secure: true,
httpOnly: true,
maxAge: 3600 * 1000 * 24
}
}));
The ingress.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: basic-routing
annotations:
nginx.ingress.kubernetes.io/use-regex: "true"
kubernetes.io/ingress.class: nginx
spec:
ingressClassName: nginx
rules:
- host: my-domain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: myapp-webapp
port:
number: 80
please help me to solve this case
Related
I am using Kubernetes v1.21.5 on docker. The following is my ingress YAML file
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: triver.dev
http:
paths:
- path: /service/account/?(.*)
pathType: Prefix
backend:
service:
name: auth-srv
port:
number: 3000
I am running an auth srv image and the following is its YAML file
kind: Deployment
metadata:
name: auth-depl
spec:
replicas: 1
selector:
matchLabels:
app: auth
template:
metadata:
labels:
app: auth
spec:
containers:
- name: auth
image: triver/auth
env:
- name: MONGO_URI
value: 'mongodb://mongo-srv:27017/auth'
- name: JWT_KEY
valueFrom:
secretKeyRef:
name: jwt-secret
key: JWT_KEY
---
apiVersion: v1
kind: Service
metadata:
name: auth-srv
spec:
selector:
app: auth
ports:
- name: auth
protocol: TCP
port: 3000
targetPort: 3000
but when I try to send a request to any route that I created in auth service for example I created a triver.dev/service/account/signup post express router for a user to signup. When I try to send a post request to the route through Postman it gives an error (404) of ECONNRefused. Couldn't send a request. Why is it happening? (My postman is working fine. It's not an issue on the postman end.)
What am I doing wrong
The app works but I just can't access the route. It's definitely an ingress issue. Can someone help me, please? This is a very important project.
This is what show up when I use the command 'kubectl get ingress'
Everything works fine when I run the application using skaffold dev.
it's due to you have not mentioned the hostname into the ingress, also make sure your ingress controller is running
example ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: hello-world.info
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: web
port:
number: 8080
host: hello-world.info
Reference : https://kubernetes.io/docs/tasks/access-application-cluster/ingress-minikube/
you can also checkout : https://kubernetes.io/docs/concepts/services-networking/ingress/
if you have set default backend set in ingress and host is not issue
make sure you are sending the request on HTTP instead of HTTPS
nginx.ingress.kubernetes.io/ssl-redirect: "false"
as you are not using the certificate SSL/TLS so please try the URL with HTTP
I have two services in Kubernetes which are exposed through nginx controller. Service a wants to invoke content on domain b but at the same time both services need to be authenticated through Google using the oauth-proxy service.
So I have managed to enable CORS and a can invoke b without any issues. But the problem is when I include the authentication as well, I am constantly getting:
Access to manifest at 'https://accounts.google.com/o/oauth2/auth?access_type=offline&approval_prompt=force&client_id=<obscured>.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fa.example.com%2Foauth2%2Fcallback&response_type=code&scope=profile+email&state=<obscured>manifest.json' (redirected from 'https://a.example.com/manifest.json') from origin 'https://a.example.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
Here are the ingresses:
Service a
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0
name: a
spec:
rules:
- host: a.example.com
http:
paths:
- backend:
service:
name: a-svc
port:
number: 8080
path: /
pathType: Prefix
tls:
- hosts:
- a.example.com
secretName: a-tls
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0
labels:
k8s-app: oauth2-proxy
name: a-oauth
spec:
rules:
- host: a.example.com
http:
paths:
- backend:
service:
name: oauth2-proxy
port:
number: 4180
path: /oauth2
pathType: Prefix
tls:
- hosts:
- a.example.com
secretName: a-oauth-tls
Service b
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
kubernetes.io/tls-acme: "true"
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0
name: b
spec:
rules:
- host: b.example.com
http:
paths:
- backend:
service:
name: b-svc
port:
number: 8080
path: /
pathType: Prefix
tls:
- hosts:
- b.example.com
secretName: b-tls
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
nginx.ingress.kubernetes.io/whitelist-source-range: 0.0.0.0/0
labels:
k8s-app: oauth2-proxy
name: b-oauth
spec:
rules:
- host: b.example.com
http:
paths:
- backend:
service:
name: oauth2-proxy
port:
number: 4180
path: /oauth2
pathType: Prefix
tls:
- hosts:
- b.example.com
secretName: b-oauth-tls
Obviously, there is only one difference between these two and that is the cors annotations nginx.ingress.kubernetes.io/enable-cors: "true" in the service b ingresses.
I am not sure of what is causing the issue but I am guessing that the authentication done against Google in service a is not being passed to the CORS request so that the service b can also be authenticate with the same token/credentials.
What am I doing wrong and how can I resolve this?
Based on the documentation, it looks like you lack of annotations for using HTTP.
Try to add fragment like this in service configuration file:
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS"
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Access-Control-Allow-Origin: $http_origin";
But it can be a problem with CORS. In such case you should add the following line:
--cors-allowed-origins=["http://*"]
to /etc/default/kube-apiserver or /etc/kubernetes/manifests/kube-apiserver.yaml file (depending on the location your kube-apiserver configuration file).
After that restart kube-apiserver.
See also this similar question.
So, it turns out that the whole Kubernetes and Nginx config was correct, so the solution was implementing the usage of the saved cookie on client side when invoking a CORS request to the second service.
Essentially, this was already answered here: Set cookies for cross origin requests
Excerpt from the answer:
Front-end (client): Set the XMLHttpRequest.withCredentials flag to
true, this can be achieved in different ways depending on the
request-response library used:
jQuery 1.5.1 xhrFields: {withCredentials: true}
ES6 fetch() credentials: 'include'
axios: withCredentials: true
I'm following the guide and I'm able to access my website via HTTP and HTTPs however redirect is not working for me, any ideas on what might be wrong?
# IngresRoute
---
kind: IngressRoute
apiVersion: traefik.containo.us/v1alpha1
metadata:
name: whoami
namespace: default
spec:
entryPoints:
web:
address: :80
http:
redirections:
entryPoint:
to: websecure
scheme: https
permanent: true
websecure:
address: :443
routes:
- match: Host(`hello.mydomain.io`)
kind: Rule
services:
- name: whoami
port: 80
tls: {}
I use Docker compose, so might not be spot on for you. But suggestion is to add a scheme redirect middleware to your dynamic config file.
http:
middlewares:
https_redirect:
redirectScheme:
scheme: https
permanent: true
Or just add the middleware to your service if you don't have access to the Traefik configs.
I prefer the dynamic config, because then you can register it on any service as required using https_redirect#file.
You do need a router per entrypoint though, using this method. And register the middleware on only the http router.
I'm sure there are other, better ways. But if you need some apps automatically redirecting and some not this is the best solution I've found thusfar.
I have a working configuration which uses Ingress objects instead of IngressRoute , but I hope this will help some people :
Thus, here is a working configuration :
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
name: redirect
namespace: some_namespace
spec:
redirectScheme:
scheme: https
permanent: true
and
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: wordpress
namespace: your_app_namespace
annotations:
kubernetes.io/ingress.class: traefik
traefik.ingress.kubernetes.io/router.middlewares: some_namespace-redirect#kubernetescrd
spec:
tls:
- secretName: your_certificate
hosts:
- www.your_website.com
rules:
- host: www.your_website.com
http:
paths:
- path: /
backend:
service:
name: your_service
port:
number: 80
pathType: ImplementationSpecific
So the trick is to :
define a Middleware object (in any namespace you want, but that may be in the same one as your app)
reference it in traefik.ingress.kubernetes.io/router.middlewares with the syntax <NAMESPACE>-<NAME>#kubernetescrd (where NAMESPACE and NAME are those of the Middleware object)
I have problems with Kubernetes. I try to deploy my service for two days now bu I'm doing something wrong.
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "forbidden: User \"system:anonymous\" cannot get path \"/\": No policy matched.",
"reason": "Forbidden",
"details": {
},
"code": 403
}
Does anybody knows what the problem could be?
Here is also my yaml file:
# Certificate
apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
name: ${APP_NAME}
spec:
secretName: ${APP_NAME}-cert
dnsNames:
- ${URL}
- www.${URL}
acme:
config:
- domains:
- ${URL}
- www.${URL}
http01:
ingressClass: nginx
issuerRef:
name: ${CERT_ISSUER}
kind: ClusterIssuer
---
# Ingress
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ${APP_NAME}
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: 'true'
nginx.ingress.kubernetes.io/from-to-www-redirect: 'true'
spec:
tls:
- secretName: ${APP_NAME}-cert
hosts:
- ${URL}
- www.${URL}
rules:
- host: ${URL}
http:
paths:
- backend:
serviceName: ${APP_NAME}-service
servicePort: 80
---
# Service
apiVersion: v1
kind: Service
metadata:
name: ${APP_NAME}-service
labels:
app: ${CI_PROJECT_NAME}
spec:
selector:
name: ${APP_NAME}
app: ${CI_PROJECT_NAME}
ports:
- name: http
port: 80
targetPort: http
---
# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: ${APP_NAME}
labels:
app: ${CI_PROJECT_NAME}
spec:
replicas: ${REPLICAS}
revisionHistoryLimit: 0
selector:
matchLabels:
app: ${CI_PROJECT_NAME}
template:
metadata:
labels:
name: ${APP_NAME}
app: ${CI_PROJECT_NAME}
spec:
containers:
- name: webapp
image: eu.gcr.io/my-site/my-site.com:latest
imagePullPolicy: Always
ports:
- name: http
containerPort: 80
env:
- name: COMMIT_SHA
value: ${CI_COMMIT_SHA}
livenessProbe:
tcpSocket:
port: 80
initialDelaySeconds: 30
timeoutSeconds: 1
readinessProbe:
tcpSocket:
port: 80
initialDelaySeconds: 5
timeoutSeconds: 1
resources:
requests:
memory: '16Mi'
limits:
memory: '64Mi'
imagePullSecrets:
- name: ${REGISTRY_PULL_SECRET}
Can anybody help me with this? I'm stuck and I've no idea what could be the problem. This is also my first Kubernetes project.
"message": "forbidden: User \"system:anonymous\" cannot get path \"/\": No policy matched.",
.. means just what it says: your request to the kubernetes api was not authenticated (that's the system:anonymous part), and your RBAC configuration does not tolerate the anonymous user making any requests to the API
No one here is going to be able to help you straighten out that problem, because fixing that depends on a horrific number of variables. Perhaps ask your cluster administrator to provide you with the correct credentials.
I have explained it in this post. You will need ServiceAccount, ClusterRole and RoleBinding. You can find explanation in this article. Or as Matthew L Daniel mentioned in the Kubernetes documentation.
If you still have problems, provide the method/tutorial you have used to deploy the cluster (as "Gitlab Kubernetes integration" does not tell much on the method you have used).
I've been struggling for a while trying to get HTTPS access to my Elasticsearch cluster in Kubernetes.
I think the problem is that Kubernetes doesn't like the TLS certificate I'm trying to use, which is why it's not passing it all the way through to the browser.
Everything else seems to work, since when I accept the Kubernetes Ingress Controller Fake Certificate, the requests go through as expected.
In my attempt to do this I've set up:
The cluster itself
An nginx-ingress controller
An ingress resource
Here's the related yaml:
Cluster:
apiVersion: v1
kind: Service
metadata:
creationTimestamp: 2018-08-03T03:20:47Z
labels:
run: my-es
name: my-es
namespace: default
resourceVersion: "3159488"
selfLink: /api/v1/namespaces/default/services/my-es
uid: 373047e0-96cc-11e8-932b-42010a800043
spec:
clusterIP: 10.63.241.39
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 9200
selector:
run: my-es
sessionAffinity: None
type: ClusterIP
status:
loadBalancer: {}
The ingress resource
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST, OPTIONS
nginx.ingress.kubernetes.io/cors-origins: http://localhost:3425 https://mydomain.ca
https://myOtherDomain.ca
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/rewrite-target: /
creationTimestamp: 2018-08-12T08:44:29Z
generation: 16
name: es-ingress
namespace: default
resourceVersion: "3159625"
selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/es-ingress
uid: ece0071d-9e0b-11e8-8a45-42001a8000fc
spec:
rules:
- http:
paths:
- backend:
serviceName: my-es
servicePort: 8080
path: /
tls:
- hosts:
- mydomain.ca
secretName: my-tls-secret
status:
loadBalancer:
ingress:
- ip: 130.211.179.225
The nginx-ingress controller:
apiVersion: v1
kind: Service
metadata:
creationTimestamp: 2018-08-12T00:41:32Z
labels:
app: nginx-ingress
chart: nginx-ingress-0.23.0
component: controller
heritage: Tiller
release: nginx-ingress
name: nginx-ingress-controller
namespace: default
resourceVersion: "2781955"
selfLink: /api/v1/namespaces/default/services/nginx-ingress-controller
uid: 755ee4b8-9dc8-11e8-85a4-4201a08000fc
spec:
clusterIP: 10.63.250.256
externalTrafficPolicy: Cluster
ports:
- name: http
nodePort: 32084
port: 80
protocol: TCP
targetPort: http
- name: https
nodePort: 31182
port: 443
protocol: TCP
targetPort: https
selector:
app: nginx-ingress
component: controller
release: nginx-ingress
sessionAffinity: None
type: LoadBalancer
status:
loadBalancer:
ingress:
- ip: 35.212.6.131
I feel like I'm missing something basic, because it doesn't seem like it should be this hard to expose something this simple...
To get my certificate, I just requested one for mydomain.ca from godaddy.
Do I need to somehow get a certificate using my ingress resource's cluster IP as the common name?
It doesn't seem possible to verify ownership of an IP.
I've seen people mention ways for Kubernetes to automatically create certificates for ingress resources, but those seem to be self signed.
Here are some logs from the nginx-controller:
This one is talking about a PEM with the tls-secret, but it's only a warning.
{
insertId: "1kvvhm7g1q7e0ej"
labels: {
compute.googleapis.com/resource_name: "fluentd-gcp-v2.0.17-5b82n"
container.googleapis.com/namespace_name: "default"
container.googleapis.com/pod_name: "nginx-ingress-controller-58f57fc597-zl25s"
container.googleapis.com/stream: "stderr"
}
logName: "projects/project-7d320/logs/nginx-ingress-controller"
receiveTimestamp: "2018-08-14T02:58:42.135388365Z"
resource: {
labels: {
cluster_name: "my-elasticsearch-cluster"
container_name: "nginx-ingress-controller"
instance_id: "2341889542400230234"
namespace_id: "default"
pod_id: "nginx-ingress-controller-58f57fc597-zl25s"
project_id: "project-7d320"
zone: "us-central1-a"
}
type: "container"
}
severity: "WARNING"
textPayload: "error obtaining PEM from secret default/my-tls-cert: error retrieving secret default/my-tls-cert: secret default/my-tls-cert was not found"
timestamp: "2018-08-14T02:58:37Z"
}
I have a few occurences of this handshake error, which may be a result of the last warning...
{
insertId: "148t6rfg1xmz978"
labels: {
compute.googleapis.com/resource_name: "fluentd-gcp-v2.0.17-5b82n"
container.googleapis.com/namespace_name: "default"
container.googleapis.com/pod_name: "nginx-ingress-controller-58f57fc597-zl25s"
container.googleapis.com/stream: "stderr"
}
logName: "projects/project-7d320/logs/nginx-ingress-controller"
receiveTimestamp: "2018-08-14T15:55:52.438035706Z"
resource: {
labels: {
cluster_name: "my-elasticsearch-cluster"
container_name: "nginx-ingress-controller"
instance_id: "2341889542400230234"
namespace_id: "default"
pod_id: "nginx-ingress-controller-58f57fc597-zl25s"
project_id: "project-7d320"
zone: "us-central1-a"
}
type: "container"
}
severity: "ERROR"
textPayload: "2018/08/14 15:55:50 [crit] 1548#1548: *860 SSL_do_handshake() failed (SSL: error:1417D18C:SSL routines:tls_process_client_hello:version too low) while SSL handshaking, client: 127.0.0.1, server: 0.0.0.0:442"
timestamp: "2018-08-14T15:55:50Z"
}
The above logs make it seem like my tls secret isnt working, but when I run kubectl describe ingress, it says my secret terminates.
aaronmw#project-7d320:~$ kubectl describe ing
Name: es-ingress
Namespace: default
Address: 130.221.179.212
Default backend: default-http-backend:80 (10.61.3.7:8080)
TLS:
my-tls-secret terminates mydomain.ca
Rules:
Host Path Backends
---- ---- --------
*
/ my-es:8080 (<none>)
Annotations:
Events: <none>
I figured it out!
What I ended up doing was adding a default ssl certificate to my nginx-ingress controller on creation using the following command
helm install --name nginx-ingress --set controller.extraArgs.default-ssl-certificate=default/search-tls-secret stable/nginx-ingress
Once I had that, it was passing the cert as expected, but I still had the wrong cert as the CN didn't match my load balancer IP.
So what I did was:
Make my load balancer IP static
Add an A record to my domain, to map a subdomain to that IP
Re-key my cert to match that new subdomain
And I'm in business!
Thanks to #Crou, who's comment reminded me to look at the logs and got me on the right track.