I have setup a kubernetes cluster from scratch. This just means I did not use services provided by others, but used the k8s installer it self. Before we used to have other clusters, but with providers and they give you tls cert and key for auth, etc. Now this cluster was setup by myself, I have access via kubectl:
$ kubectl get all
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 21h
$
I also tried this and I can add a custom key, but then when I try to query via curl I get pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" at the cluster scope.
I can not figure out where can I get the cert and key for a user to authenticate using the API for tls auth. I have tried to understand the official docs, but I have got nowhere. Can someone help me find where those files are or how to add or get certificates that i can use for the rest API?
Edit1: my .kube.config file looks like this:
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LS0t(...)=
server: https://private_IP:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin#kubernetes
current-context: kubernetes-admin#kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: LS0tLS(...)Qo=
client-key-data: LS0(...)tCg==
It works from the localhost just normally.
In the other hand I noticed something. From the localhost I can access the cluster by generating the token using this method.
Also notice that for now I do not mind about creating multiple roles for multiple users, etc. I just need access to the API from remote and can be using "default" authentication or roles.
Now when I try to do the same from remote I get the following:
I tried using that config to run kubectl get all from remote, it runs for a while and then ends in Unable to connect to the server: dial tcpprivate_IP:6443: i/o timeout.
This happens because the config has private_IP, then I changed the IP to Public_IP:6443 and now get the following : Unable to connect to the server: x509: certificate is valid for some_private_IP, My_private_IP, not Public_IP:6443
Keep present that this is and AWS ec2 instance with elastic IP (You can think of Elastic IP as just a public IP on a traditional setup, but this public ip is on your public router and then this router routes requests to your actual server on private network). For AWS fans like I said, I can not use the EKS service here.
So how do I get this to be able to use the Public IP?
It seems your main problem is the TLS server certificate validation.
One option is to tell kubectl to skip the validation of the server certificate:
kubectl --insecure-skip-tls-verify ...
This has obviously the potential to be "insecure", but that depends on your use case
Another option is to recreate the cluster with the public IP address added to the server certificate. And it should also be possible to recreate only the certificate with kubeadm without recreating the cluster. Details about the latter two points can be found in this answer.
You need to setup RBAC for the user. define roles and rolebinding. follow the link for reference -> https://docs.bitnami.com/kubernetes/how-to/configure-rbac-in-your-kubernetes-cluster/
I am trying to send my app to a Google Cloud Cluster using the kubectl command behind a corporative proxy that needs a certificate ".crt" file to be used when doing HTTPS requests.
I already ran the gcloud container clusters get-credentials... command and it also asked for a certificate. I followed the given instructions by Google and I configured my certificate file without any issue and it worked.
But when I try the kubectl get pods I am getting the following message:
"Unable to connect to the server: x509: certificate signed by unknown authority"
How can I configure my certificate file to be used by the kubectl command?
I did a search about this subject but I found too difficult steps. Could I just run something like this:
kubectl --set_ca_file /path/to/my/cert
Thank you
The short answer up to what I know is no.
here[1] you can see the step by step of how to get this done in the easiest way I found so far, is not a one line way but is the closest to that.
after having your cert files you need to run this:
gcloud compute ssl-certificates create test-ingress-1 \ --certificate [FIRST_CERT_FILE] --private-key [FIRST_KEY_FILE]
then you need to create your YAML file with the configuration (in the link there are two examples)
run this command:
kubectl apply -f [NAME_OF_YOUR_FILE].yaml
[1] https://cloud.google.com/kubernetes-engine/docs/how-to/ingress-multi-ssl
i'm getting an error when running kubectl one one machine (windows)
the k8s cluster is running on CentOs 7 kubernetes cluster 1.7
master, worker
Here's my .kube\config
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: REDACTED
server: https://10.10.12.7:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: system:node:localhost.localdomain
name: system:node:localhost.localdomain#kubernetes
current-context: system:node:localhost.localdomain#kubernetes
kind: Config
preferences: {}
users:
- name: system:node:localhost.localdomain
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
the cluster is built using kubeadm with the default certificates on the pki directory
kubectl unable to connect to server: x509: certificate signed by unknown authority
One more solution in case it helps anyone:
My scenario:
using Windows 10
Kubernetes installed via Docker Desktop ui 2.1.0.1
the installer created config file at ~/.kube/config
the value in ~/.kube/config for server is https://kubernetes.docker.internal:6443
using proxy
Issue: kubectl commands to this endpoint were going through the proxy, I figured it out after running kubectl --insecure-skip-tls-verify cluster-info dump which displayed the proxy html error page.
Fix: just making sure that this URL doesn't go through the proxy, in my case in bash I used export no_proxy=$no_proxy,*.docker.internal
So kubectl doesn't trust the cluster, because for whatever reason the configuration has been messed up (mine included). To fix this, you can use openssl to extract the certificate from the cluster
openssl.exe s_client -showcerts -connect IP:PORT
IP:PORT should be what in your config is written after server:
Copy paste stuff starting from -----BEGIN CERTIFICATE----- to -----END CERTIFICATE----- (these lines included) into a new text file, say... myCert.crt If there are multiple entries, copy all of them.
Now go to .kube\config and instead of
certificate-authority-data: <wrongEncodedPublicKey>`
put
certificate-authority: myCert.crt
(it assumes you put myCert.crt in the same folder as the config file)
If you made the cert correctly it will trust the cluster (tried renaming the file and it no longer trusted afterwards).
I wish I knew what encoding certificate-authority-data uses, but after a few hours of googling I resorted to this solution, and looking back I think it's more elegant anyway.
Run:
gcloud container clusters get-credentials standard-cluster-1 --zone us-central1-a --project devops1-218400
here devops1-218400 is my project name. Replace it with your project name.
I got the same error while running $ kubectl get nodes as a root user. I fixed it by exporting kubelet.conf to environment variable.
$ export KUBECONFIG=/etc/kubernetes/kubelet.conf
$ kubectl get nodes
For my case, its simple worked by adding --insecure-skip-tls-verify at end of kubectl commands, for single time.
Sorry I wasn't able to provide this earlier, I just realized the cause:
So on the master node we're running a kubectl proxy
kubectl proxy --address 0.0.0.0 --accept-hosts '.*'
I stopped this and voila the error was gone.
I'm now able to do
kubectl get nodes
NAME STATUS AGE VERSION
centos-k8s2 Ready 3d v1.7.5
localhost.localdomain Ready 3d v1.7.5
I hope this helps those who stumbled upon this scenario.
I my case I resolved this issue copying the kubelet configuration to my home kube config
cat /etc/kubernetes/kubelet.conf > ~/.kube/config
This was happening because my company's network does not allow self signing certificates through their network. Try switching to a different network
For those of you that were late to the thread like I was and none of these answers worked for you I may have the solution:
When I copied over my .kube/config file to my windows 10 machine (with kubectl installed) I didn't change the IP address from 127.0.0.1:6443 to the master's IP address which was 192.168.x.x. (running windows 10 machine connecting to raspberry pi cluster on the same network). Make sure that you do this and it may fix your problem like it did mine.
On GCP
check: gcloud version
-- localMacOS# gcloud version
Run:
--- localMacOS# gcloud container clusters get-credentials 'clusterName' \ --zone=us-'zoneName'
Get clusterName and zoneName from your console -- here: https://console.cloud.google.com/kubernetes/list?
ref: .x509 #market place deployments on GCP #Kubernetes
I got this because I was not connected to the office's VPN
In case of the error you should export all the kubecfg which contains the certs. kops export kubecfg "your cluster-name and export KOPS_STATE_STORE=s3://"paste your S3 store" .
Now you should be able to access and see the resources of your cluster.
This is an old question but in case that also helps someone else here is another possible reason.
Let's assume that you have deployed Kubernetes with user x. If the .kube dir is under the /home/x user and you connect to the node with root or y user it will give you this error.
You need to switch to the user profile so kubernetes can load the configuration from the .kube dir.
Update: When copying the ~/.kube/config file content on a local pc from a master node make sure to replace the hostname of the loadbalancer with a valid IP. In my case the problem was related to the dns lookup.
Hope this helps.
I am using a Docker container to run a bunch of services, all of those services make use of certificates to communicate to each other.
When starting up those services there is one in concrete that complains with the following error
> discovery_1 | INFO ttn: Got public keys for token validation
> discovery_1 | DEBUG Connected to gRPC server Address=localhost:1900
> discovery_1 | FATAL Could not start client for gRPC proxy error=x509: certificate is valid for discovery, not localhost
> ttnbackbone_discovery_1 exited with code 1
I have created the certificate for "discovery" user but still Docker runs it for the localhost, in some way, which I don't understand... I have also followed this tutorial of certificates usage from Docker but still I have the same error.
What can I do further?
THanks in advance,
REgards!
I encountered this today. x509 certificates have a Common Name attribute that some software use to match the DNS hostname of a server. Here was my error with a certificate with CN of localhost and a DNS hostname of docker1-staging:
error during connect: Get https://docker1-staging:2376/v1.26/containers/json: x509: certificate is valid for localhost, not docker1-staging
I'll have to regenerate the certificate used by the Docker server and make sure it has a CN value of docker1-staging. You'll have to do the same with a CN value of localhost.
We have created a private docker registry in artifactory.
Our artifactory is a standalone installation and have Nginx as a webserver.
SSL certificates are trusted and works fine.
on docker client, I have copied the ca.crt to /etc/docker/certs.d/:5001/
while am trying to login or push images from my docker client i see below error.
[root#cds-dev-test ~]# docker login artifactory.host:5001
Username: raj
Password:
Email: raj#gmail.com
Error response from daemon: invalid registry endpoint
https://artifactory.host:5001/v0/: unable to ping registry endpoint
v2 ping attempt failed with error: Get https://artifactory.host:5001/v2/: Tunnel Connection Failed
v1 ping attempt failed with error: Get artifactory.host:5001/v1/_ping: Tunnel Connection Failed. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add --insecure-registry artifactory.host:5001 to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/artifactory.host:5001/ca.crt
my docker version is 1.9.1 and artifactory versioin 4.4.3.
It works when i use --insecure-registry option but not the secure way. We have all trusted certs in place, still see the error.
I have tried using proxy settings on docker client and also without proxy... always the same error.
Any help guys?
I figured it out.
I have proxy settings under my docker deamon. I have added No_Proxy and it works fine.
FYI....
so people if you are using trusted CA cert, and your network is behind proxy, make sure your docker services file doesnt have proxy settings, if it does add No-proxy=artifactory.host.
/etc/systemd/system/docker.service.d/http-proxy.conf
Thanks