I'm trying some basic examples to request data from the web, however all requests to different hosts result in an SSL error: x509: certificate signed by unknown authority. Note: I'm not behind a proxy and no forms of certificate interception is happening, as using curl or the browser works without problems.
The code sample I'm currently working with is:
package main
import (
"fmt"
"net/http"
"io/ioutil"
"os"
)
func main() {
response, err := http.Get("https://google.com")
if err != nil {
fmt.Printf("%s\n", err)
os.Exit(1)
} else {
defer response.Body.Close()
contents, err := ioutil.ReadAll(response.Body)
if err != nil {
fmt.Printf("%s", err)
os.Exit(1)
}
fmt.Printf("%s\n", string(contents))
}
}
Edit: Code is run on Arch linux kernel 4.9.37-1-lts.
Edit 2: Apparently /etc/ssl/certs/ca-certificates.crt had a difference between the version on my system, by (re)moving the certificate and re-installing the ca-certificates-utils package manually, the issue was solved.
Based on your error, I'm assuming you are using Linux?
It's likely that you will have to install ca-certificates on the machine your program is running on.
On Ubuntu, you would execute something like this:
sudo apt-get install ca-certificates
Related
I'm starting to learn golang and I'm trying to make a simple http client that will get a list of virtual machines from one of our oVirt clusters. The API that I'm trying to access has a self-signed certificate (auto generated during the cluster installation) and golang's http.client encounters a problem when serializing the time from the certificate. Below you can find the code and the output.
package main
import (
"fmt"
"io/ioutil"
"net/http"
"crypto/tls"
)
func do_request(url string) ([]byte, error) {
// ignore self signed certificates
transCfg := &http.Transport{
TLSClientConfig: &tls.Config {
InsecureSkipVerify: true,
},
}
// http client
client := &http.Client{Transport: transCfg}
// request with basic auth
req, _ := http.NewRequest("GET", url, nil)
req.SetBasicAuth("user","pass")
resp, err := client.Do(req)
// error?
if err != nil {
fmt.Printf("Error : %s", err)
return nil, err
}
defer resp.Body.Close()
body, _ := ioutil.ReadAll(resp.Body)
return []byte(body), nil
}
func main() {
body, _ := do_request("https://ovirt.example.com/")
fmt.Println("response Status:", string(body))
}
and the error when I'm trying to compile:
$ go run http-get.go
Error : Get https://ovirt.example.com/: tls: failed to parse certificate from server: asn1: time did not serialize back to the original value and may be invalid: given "141020123326+0000", but serialized as "141020123326Z"response Status:
Is there any way to ignore this verification? I tried making a request using other programming languages (python, ruby) and skipping insecure certificates seems to be enough.
Thank you!
PS: I know the proper solution is to change the certificate with a valid one, but for the moment I cannot do this.
Unfortunately, you've encountered an error that you cannot get around in Go. This is buried deep in the cypto/x509 and encoding/asn1 packages without a way to ignore. Specifically, asn1.parseUTCTime is expecting the time format to be "0601021504Z0700", but your server is sending "0601021504+0000". Technically, that is a known format but encoding/asn1 does not support it.
There are only 2 solutions that I can come up with that do not require a code change for golang.
1) Edit the encoding/asn1 package in your go src directory and then rebuild all the standard packages with go build -a
2) Create your own customer tls, x509 and asn1 packages to use the format your server is sending.
Hope this helps.
P.S. I've opened an issue with the Go developers to see if it can resolved by them at some later point Issue Link
Possible ASN1 UtcTime Formats.
I want to receive a TCP connection over TLS. I want to validate client certificate and use it to authenticate the client to my application.
Go has the standard crypto/tls package. It can validate client/server certificates. But I can't find way to get details of the remote (client) certificate, like the common name.
Have to call crypto/tls/Conn.Handshake.
Then you can read peer certificate:
tlsconn.ConnectionState().PeerCertificates[0].Subject.CommonName
Following code may help you get your answer
package main
import (
"crypto/tls"
"fmt"
"log"
)
func main() {
conf := &tls.Config{
InsecureSkipVerify: true,
}
conn, err := tls.Dial("tcp", "www.google.com:443", conf)
if err != nil {
log.Println("Error in Dial", err)
return
}
defer conn.Close()
certs := conn.ConnectionState().PeerCertificates
for _, cert := range certs {
fmt.Printf("Issuer Name: %s\n", cert.Issuer)
fmt.Printf("Expiry: %s \n", cert.NotAfter.Format("2006-January-02"))
fmt.Printf("Common Name: %s \n", cert.Issuer.CommonName)
}
}
When working with crypto/tls you can query any Conn object for ConnectionState:
func (c *Conn) ConnectionState() ConnectionState
The ConnectionState struct contains information about the client certificate:
type ConnectionState struct {
PeerCertificates []*x509.Certificate // certificate chain presented by remote peer
}
The x509.Certificate should be pretty straightforward to work with.
Before the server requests for client authentication, you have to configure the connection with the server certificate, client CA (otherwise you will have to verify the trust chain manually, you really don't want that), and tls.RequireAndVerifyClientCert. For example:
// Load my SSL key and certificate
cert, err := tls.LoadX509KeyPair(settings.MyCertificateFile, settings.MyKeyFile)
checkError(err, "LoadX509KeyPair")
// Load the CA certificate for client certificate validation
capool := x509.NewCertPool()
cacert, err := ioutil.ReadFile(settings.CAKeyFile)
checkError(err, "loadCACert")
capool.AppendCertsFromPEM(cacert)
// Prepare server configuration
config := tls.Config{Certificates: []tls.Certificate{cert}, ClientCAs: capool, ClientAuth: tls.RequireAndVerifyClientCert}
config.NextProtos = []string{"http/1.1"}
config.Rand = rand.Reader
There is an easier way to do that:
func renewCert(w http.ResponseWriter, r *http.Request) {
if r.TLS != nil && len(r.TLS.PeerCertificates) > 0 {
cn := strings.ToLower(r.TLS.PeerCertificates[0].Subject.CommonName)
fmt.Println("CN: %s", cn)
}
}
I'm creating a Go TCP server (NOT http/s) and I'm trying to configure it to use SSL. I have a StartCom free SSL certificate which I am trying to use to accomplish this. My server code looks like this:
cert, err := tls.LoadX509KeyPair("example.com.pem", "example.com.key")
if err != nil {
fmt.Println("Error loading certificate. ",err)
}
trustCert, err := ioutil.ReadFile("sub.class1.server.ca.pem")
if err != nil {
fmt.Println("Error loading trust certificate. ",err)
}
validationCert, err := ioutil.ReadFile("ca.pem")
if err != nil {
fmt.Println("Error loading validation certificate. ",err)
}
certs := x509.NewCertPool()
if !certs.AppendCertsFromPEM(validationCert) {
fmt.Println("Error installing validation certificate.")
}
if !certs.AppendCertsFromPEM(trustCert) {
fmt.Println("Error installing trust certificate.")
}
sslConfig := tls.Config{RootCAs: certs,Certificates: []tls.Certificate{cert}}
service := ":5555"
tcpAddr, error := net.ResolveTCPAddr("tcp", service)
if error != nil {
fmt.Println("Error: Could not resolve address")
} else {
netListen, error := tls.Listen(tcpAddr.Network(), tcpAddr.String(), &sslConfig)
if error != nil {
fmt.Println(error)
} else {
defer netListen.Close()
for {
fmt.Println("Waiting for clients")
connection, error := netListen.Accept()
I've tried switching around the order of the certs, not including some certs, etc. but the output from openssl s_client -CApath /etc/ssl/certs/ -connect localhost:5555 remains essentially the same, verify error:num=20:unable to get local issuer certificate. See here for full output. I seem to be doing something wrong with the intermediate certificates, but I have no idea what. I have been working on this for a few days, lots of googling and SO'ing, but nothing seemed to quite fit my situation. I have set up many certificates in Apache and HAProxy, but this really has me stumped.
The RootCAs field is for clients verifying server certificates. I assume you only want to present a cert for verification, so anything you need should be loaded into the Certificates slice.
Here is a minimal example:
cert, err := tls.LoadX509KeyPair("example.com.pem", "example.com.key")
if err != nil {
log.Fatal("Error loading certificate. ", err)
}
tlsCfg := &tls.Config{Certificates: []tls.Certificate{cert}}
listener, err := tls.Listen("tcp4", "127.0.0.1:5555", tlsCfg)
if err != nil {
log.Fatal(err)
}
defer listener.Close()
for {
log.Println("Waiting for clients")
conn, err := listener.Accept()
if err != nil {
log.Fatal(err)
}
go handle(conn)
}
Even though you're not using HTTPS, it may still be useful to walk through the server setup starting at http.ListenAndServeTLS.
I have the following certificate hierarchy:
Root-->CA-->3 leaf certificates
The entire chain has both serverAuth and clientAuth as extended key usages explicitly defined.
In my go code, I create a tls.Config object like so:
func parseCert(certFile, keyFile string) (cert tls.Certificate, err error) {
certPEMBlock , err := ioutil.ReadFile(certFile)
if err != nil {
return
}
var certDERBlock *pem.Block
for {
certDERBlock, certPEMBlock = pem.Decode(certPEMBlock)
if certDERBlock == nil {
break
}
if certDERBlock.Type == "CERTIFICATE" {
cert.Certificate = append(cert.Certificate, certDERBlock.Bytes)
}
}
// Need to flip the array because openssl gives it to us in the opposite format than golang tls expects.
cpy := make([][]byte, len(cert.Certificate))
copy(cpy, cert.Certificate)
var j = 0
for i := len(cpy)-1; i >=0; i-- {
cert.Certificate[j] = cert.Certificate[i]
j++
}
keyData, err := ioutil.ReadFile(keyFile)
if err != nil {
return
}
block, _ := pem.Decode(keyData)
if err != nil {
return
}
ecdsaKey, err := x509.ParseECPrivateKey(block.Bytes)
if err != nil {
return
}
cert.PrivateKey = ecdsaKey
return
}
// configure and create a tls.Config instance using the provided cert, key, and ca cert files.
func configureTLS(certFile, keyFile, caCertFile string) (tlsConfig *tls.Config, err error) {
c, err := parseCert(certFile, keyFile)
if err != nil {
return
}
ciphers := []uint16 {
tls.TLS_RSA_WITH_AES_256_CBC_SHA,
tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,
}
certPool := x509.NewCertPool()
buf, err := ioutil.ReadFile(caCertFile)
if nil != err {
log.Println("failed to load ca cert")
log.Fatal(seelog.Errorf("failed to load ca cert.\n%s", err))
}
if !certPool.AppendCertsFromPEM(buf) {
log.Fatalln("Failed to parse truststore")
}
tlsConfig = &tls.Config {
CipherSuites: ciphers,
ClientAuth: tls.RequireAndVerifyClientCert,
PreferServerCipherSuites: true,
RootCAs: certPool,
ClientCAs: certPool,
Certificates: []tls.Certificate{c},
}
return
}
certFile is the certificate chain file and keyFile is the private key file. caCertFile is the truststore and consists of just the root certificate
So basically, here is what I expect to have inside of my tls.Config object that comes out of this function:
RootCAs: Just the root certificate from caCertFile
ClientCAs: Again, just the root certificate from caCertFile, same as RootCAs
Certificates: A single certificate chain, containing all of the certificates in certFile, ordered to be leaf first.
Now, I have 3 pieces here. A server, a relay, and a client. The client connects directly to the relay, which in turn forwards the request to the server. All three pieces use the same configuration code, of course using different certs/keys. The caCertFile is the same between all 3 pieces.
Now, if I stand up the server and the relay and connect to the relay from my browser, all goes well, so I can assume that the connection between relay and server is fine. The issue comes about when I try to connect my client to the relay. When I do so, the TLS handshake fails and the following error is returned:
x509: certificate signed by unknown authority
On the relay side of things, I get the following error:
http: TLS handshake error from : remote error: bad certificate
I am really at a loss here. I obviously have something setup incorrectly, but I am not sure what. It's really weird that it works from the browser (meaning that the config is correct from relay to server), but it doesn't work with the same config from my client.
Update:
So if I add InsecureSkipVerify: true to my tls.Config object on both the relay and the client, the errors change to:
on the client: remote error: bad certificate
and on the relay: http: TLS handshake error from : tls: client didn't provide a certificate
So it looks like the client is rejecting the certificate on from the server (the relay) due to it being invalid for some reason and thus never sending its certificate to the server (the relay).
I really wish go had better logging. I can't even hook into this process to see what, exactly, is going on.
When you say
Need to flip the array because openssl gives it to us in the opposite format than golang tls expects.
I have used certificates generated by openssl and had no problem opening them with:
tls.LoadX509KeyPair(cert, key)
Anyway, the error message bad certificate is due to the server not managing to match the client-provided certificate against its RootCAs. I have also had this problem in Go using self-signed certificats and the only work-around I've found is to install the caCertFile into the machines system certs, and use x509.SystemCertPool() instead of x509.NewCertPool().
Maybe someone else will have another solution?
Beside what beldin0 suggested.
I have tried another way to do this.
caCertPool := x509.NewCertPool()
caCertPool.AppendCertsFromPEM(crt)
client := &http.Client{
//some config
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
RootCAs: caCertPool,
},
},
}
Here, the variable "crt" is the content in your certificate.
Basically, you just add it into your code(or read as a config file).
Then everything would be fine.
I'm trying to establish a TLS connection with the use of a self signed server certificate.
I generated the certificate with this example code: http://golang.org/src/pkg/crypto/tls/generate_cert.go
My relevant client code looks like that:
// server cert is self signed -> server_cert == ca_cert
CA_Pool := x509.NewCertPool()
severCert, err := ioutil.ReadFile("./cert.pem")
if err != nil {
log.Fatal("Could not load server certificate!")
}
CA_Pool.AppendCertsFromPEM(severCert)
config := tls.Config{RootCAs: CA_Pool}
conn, err := tls.Dial("tcp", "127.0.0.1:8000", &config)
if err != nil {
log.Fatalf("client: dial: %s", err)
}
And the relevant server code like that:
cert, err := tls.LoadX509KeyPair("./cert.pem", "./key.pem")
config := tls.Config{Certificates: []tls.Certificate{cert}}
listener, err := tls.Listen("tcp", "127.0.0.1:8000", &config)
for {
conn, err := listener.Accept()
if err != nil {
log.Printf("server: accept: %s", err)
break
}
log.Printf("server: accepted from %s", conn.RemoteAddr())
go handleConnection(conn)
}
Because the server certificate is self signed is use the same certificate for the server and the clients CA_Pool however this does not seem to work since i always get this error:
client: dial: x509: certificate signed by unknown authority
(possibly because of "x509: invalid signature: parent certificate
cannot sign this kind of certificate" while trying to verify
candidate authority certificate "serial:0")
What's my mistake?
It finally worked with the go built in x509.CreateCertificate,
the problem was that I did not set the IsCA:true flag,
I only set the x509.KeyUsageCertSign which made creating the self signed certificate work, but crashed while verifying the cert chain.
The problem is that you need a CA certificate in the server-side config, and this CA must have signed the server's certificate.
I have written some Go code that will generate a CA certificate, but it hasn't been reviewed by anyone and is mostly a toy for playing around with client certs. The safest bet is probably to use openssl ca to generate and sign the certificate. The basic steps will be:
Generate a CA Certificate
Generate a Server key
Sign the Server key with the CA certificate
Add the CA Certificate to the client's tls.Config RootCAs
Set up the server's tls.Config with the Server key and signed certificate.
Kyle, is correct. This tool will do what you want and it simplifies the entire process:
https://github.com/deckarep/EasyCert/releases (only OSX is supported since it uses the openssl tool internally)
and the source:
https://github.com/deckarep/EasyCert
Basically with this tool it will generate a bundle of files but you will need the three that it outputs when it's done.
a CA root cer file
a Server cer file
a Server key file
In my case, the certificate I appended was not encoded correctly in pem format.
If using keytools, ensure to append -rfc while exporting the certificate from keystore, pem encoded could be opened in a text editor to display:
-----BEGIN CERTIFICATE-----
MIIDiDCCAnCgAwIBAgIEHKSkvDANBgkqhkiG9w0BAQsFADBi...
I saw the same error when using mysql client in Go:
Failed to connect to database: x509: cannot validate certificate for 10.111.202.229 because it doesn't contain any IP SANs
and setting InsecureSkipVerify to true (to skip verification of certificate) resolved it for me:
https://godoc.org/crypto/tls#Config
The following code worked for me:
package main
import (
"fmt"
"github.com/go-sql-driver/mysql"
"github.com/jinzhu/gorm"
"crypto/tls"
"crypto/x509"
"io/ioutil"
"log"
)
func main() {
rootCertPool := x509.NewCertPool()
pem, err := ioutil.ReadFile("/usr/local/share/ca-certificates/ccp-root-ca.crt")
if err != nil {
log.Fatal(err)
}
if ok := rootCertPool.AppendCertsFromPEM(pem); !ok {
log.Fatal("Failed to append root CA cert at /usr/local/share/ca-certificates/ccp-root-ca.crt.")
}
mysql.RegisterTLSConfig("custom", &tls.Config{
RootCAs: rootCertPool,
InsecureSkipVerify: true,
})
db, err := gorm.Open("mysql", "ccp-user:I6qnD6zNDmqdDLXYg3HqVAk2P#tcp(10.111.202.229:3306)/ccp?tls=custom")
defer db.Close()
}
You need to use the InsecureSkipVerify flag, refer to https://groups.google.com/forum/#!topic/golang-nuts/c9zEiH6ixyw.
The related code of this post (incase the page is offline):
smtpbox := "mail.foo.com:25"
c, err := smtp.Dial(smtpbox)
host, _, _ := net.SplitHostPort(smtpbox)
tlc := &tls.Config{
InsecureSkipVerify: true,
ServerName: host,
}
if err = c.StartTLS(tlc); err != nil {
fmt.Printf(err)
os.Exit(1)
}
// carry on with rest of smtp transaction
// c.Auth, c.Mail, c.Rcpt, c.Data, etc