Why does a VirtualNode require podSelector.matchLabels and serviceDiscovery.dns?
Aren't they doing the same thing - identifying which Pods (IPs) are member of this virtual node?
The AWS App Mesh Controller For K8s uses podSelector to match every Pod to a VirtualNode (API Design) in the Mesh as long as it does not have an appmesh.k8s.aws/sidecarInjectorWebhook: disabled annotation defined.
Related
I have two Api's on the same cluster and when I run the get services I get the following.
dh-service ClusterIP 10.233.48.45 <none> 15012/TCP 70d
api-service ClusterIP 10.233.54.208 <none> 15012/TCP
Now I want to make a Api call from one API to the other, When I do it using the Ingress address for the two Images I get 404 Not Found.
What address should I use for my post calls? Will the cluster ip work ?
I want to make a Api call from one API to the other
If they are in the same namespace and you use http, you can use:
http://dh-service
http://api-service
to access them.
If e.g. the api-service is located in a different namespace e.g. blue-namespace you can access it with:
http://api-service.blue-namespace
See more on DNS for Services and Pods
I have the following situation:
Application uses S3 to store data in Amazon. Application is deployed as a pod in kubernetes. Sometimes some of developers messes with access data for S3 (eg. user/password) and application fails to connect to S3 - but pod starts normally and kills previous pod version that worked OK (since all readiness and aliveness probes are OK). I thought of adding S3 probe to readiness - in order to execute HeadBucketRequest on S3 and if this one succeeds it is able to connect to S3. The problem here is that these requests cost money, and I really need them only on start of the pod.
Are there any best-practices related to this one?
If you (quote) "... really need them [the probes] only on start of the pod" then look into adding a startup probe.
In addition to what startup probes help with - pods that take longer time to start - a startup probe will make it possible to verify a condition only at pod startup time.
Readiness and liveness prove as for checking the health of POD or container while running. You scenario is quite wired but with Readiness & liveness probe it wont work as it fire on internal and which cost money.
in this case you might can use the lifecycle hook :
containers:
- image: MAGE_NAME
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "script.sh"]
which will run the hook at starting of the container you can keep shell file inside the POD or image.
inside shell file you can right logic if 200 response move a head and container get started.
https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/
We have several pods (as service/deployments) in our k8s workflow that are dependent on each other, such that if one goes into a CrashLoopBackOff state, then all these services need to be redeployed.
Instead of having to manually do this, is there a programatic way of handling this?
Of course we are trying to figure out why the pod in question is crashing.
If these are so tightly dependant on each other, I would consider these options
a) Rearchitect your system to be more resilient towards failure and tolerate, if a pod is temporary unavailable
b) Put all parts into one pod as separate containers, making the atomic design more explicit
If these don't fit your needs, you can use the Kubernetes API to create a program that automates the task of restarting all dependent parts. There are client libraries for multiple languages and integration is quite easy. The next step would be a custom resource definition (CRD) so you can manage your own system using an extension to the Kubernetes API.
First thing to do is making sure that pods are started in correct sequence. This can be done using initContainers like that:
spec:
initContainers:
- name: waitfor
image: jwilder/dockerize
args:
- -wait
- "http://config-srv/actuator/health"
- -wait
- "http://registry-srv/actuator/health"
- -wait
- "http://rabbitmq:15672"
- -timeout
- 600s
Here your pod will not start until all the services in a list are responding to HTTP probes.
Next thing you may want to define liveness probe that periodically executes curl to the same services
spec:
livenessProbe:
exec:
command:
- /bin/sh
- -c
- curl http://config-srv/actuator/health &&
curl http://registry-srv/actuator/health &&
curl http://rabbitmq:15672
Now if any of those services fail - you pod will fail liveness probe, be restarted and wait for services to become back online.
That's just an example how it can be done. In your case checks can be different of course.
I believe I have followed this guide:
https://medium.com/#hobochild/installing-cert-manager-on-a-gcloud-k8s-cluster-d379223f43ff
which, has me install the without-rbac version of cert-manager from this repo:
https://github.com/jetstack/cert-manager
however when the cert-manager pod boots up it starts spamming this error:
leaderelection.go:224] error retrieving resource lock cert-manager/cert-manager-controller: configmaps "cert-manager-controller" is forbidden: User "system:serviceaccount:cert-manager:default" cannot get configmaps in the namespace "cert-manager": Unknown user "system:serviceaccount:cert-manager:default"
Hoping someone has some ideas.
The errors seem to be coming from RBAC. If you're running this in minikube you can grant the default service account in the cert-manager namespace the proper rights by running:
kubectl create clusterrolebinding cert-manager-cluster-admin --clusterrole=cluster-admin --serviceaccount=cert-manager:default
After creating the role binding, cert-manager should complete its startup.
You should use the 'with-rbac.yaml' variant if you are installing in GKE, unless you have explicitly disabled RBAC on the GKE cluster!
This should resolve the issues you're seeing here, as by the looks of your error message, you do have RBAC enabled!
How does the openshift count the resource quota consumed by specific builder image? (There may be multiple images)
It is created by sti builder, but not openshift cluster itself (k8s exactly).
I know the quota is equal to the sti builder, but would like to know how to count it if we customized the quota (and if I can do that). It looks like the cluster can't count the resource quota (cpu/memory, etc)
Together with quota you can define a scope. See OpenShift Origin: quota scopes.
The relevant scope for build and deployment pods is NonTerminating.
Adding this scope to quota definition will constrain it to only build and deployment pods (pods that have spec.activeDeadlineSeconds is nil according to docs)
Example definition:
apiVersion: v1
kind: ResourceQuota
metadata:
name: slow-builds-and-deployments
spec:
hard:
pods: "2"
limits.cpu: "1"
limits.memory: "1Gi"
scopes:
- NotTerminating
The Terminating scope on the other side will be applied other pods (pods with spec.activeDeadlineSeconds >= 0).