i am trying to make https request with golang.
conf := &tls.Config{
InsecureSkipVerify: true,
MinVersion:tls.VersionTLS10,
}
//TLS connection
tlsCon, err := tls.Dial("tcp", "youtube.com:443", conf)
if err != nil {
fmt.Println("SSL Error : " + err.Error())
return
}
defer tlsCon.Close()
state := tlsCon.ConnectionState()
fmt.Println("SSL ServerName : " + state.ServerName)
fmt.Println("SSL Handshake : ", state.HandshakeComplete)
fmt.Println("SSL Mutual : ", state.NegotiatedProtocolIsMutual)
request = '
CONNECT youtube.com:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:51.0) Gecko/20100101 Firefox/51.0
Proxy-Connection: close
Connection: close
Host: youtube.com:443
'
n, err = io.WriteString(tlsCon, request)
if err != nil {
fmt.Println("SSL Write error :", err.Error(), n)
}
n, err = tlsCon.Read(data)
if err != nil {
fmt.Println("SSL Read error : " + err.Error())
return
}
this is my code , state.HandshakeComplete, state.NegotiatedProtocolIsMutual returns true and state.ServerName returns = ''
when i write data to tls connection no error returns also tls connection never reply my request.
any help appricated. thanks.
Related
I'm doing a post request and I get a 200 OK response. I also receive the headers. However, the body keeps coming back empty.
There should be a body, when I run it in postman the body shows up. What am I missing here?
func AddHealthCheck(baseURL string, payload HealthCheck, platform string, hostname string) (string, error) {
url := fmt.Sprintf(baseURL+"add-healthcheck/%s/%s", platform, hostname)
//convert go struct to json
jsonPayload, err := json.Marshal(payload)
if err != nil {
log.Error("[ADD HEALTH CHECK] Could not convert go struct to json : ", err)
return "", err
}
// Create client & set timeout
client := &http.Client{}
client.Timeout = time.Second * 15
// Create request
req, err := http.NewRequest("POST", url, bytes.NewBuffer(jsonPayload))
if err != nil {
log.Error("[ADD HEALTH CHECK] Could not create request : ", err)
return "", err
}
req.Header.Set("Content-Type", "application/json")
// Fetch Request
resp, err := client.Do(req)
if err != nil {
log.Error("[ADD HEALTH CHECK] Could not fetch request : ", err)
return "", err
}
defer resp.Body.Close()
// Read Response Body
respBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Error("[HEALTH CHECK] Could not read response body : ", err)
return "", err
}
fmt.Println("response Status : ", resp.Status)
fmt.Println("response Headers : ", resp.Header)
fmt.Println("response Body : ", string(respBody))
return string(respBody), nil
}
I have confirmed locally that your code, as shown, should work.
Here is the code I used:
package main
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"time"
)
func main() {
http.HandleFunc("/", handler)
go func(){
http.ListenAndServe(":8080", nil)
}()
AddHealthCheck()
}
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hi there")
}
func panicError(err error) {
if err != nil {
panic(err)
}
}
func AddHealthCheck() (string, error) {
//convert go struct to json
payload := "bob"
jsonPayload, err := json.Marshal(payload)
panicError(err)
// Create client & set timeout
client := &http.Client{}
client.Timeout = time.Second * 15
// Create request
req, err := http.NewRequest("POST", "http://localhost:8080", bytes.NewBuffer(jsonPayload))
panicError(err)
req.Header.Set("Content-Type", "application/json")
// Fetch Request
resp, err := client.Do(req)
panicError(err)
defer resp.Body.Close()
// Read Response Body
respBody, err := ioutil.ReadAll(resp.Body)
panicError(err)
fmt.Println("response Status : ", resp.Status)
fmt.Println("response Headers : ", resp.Header)
fmt.Println("response Body : ", string(respBody))
return string(respBody), nil
}
The code above is just a slightly stripped down version of your code, and it outputs the body of the response. (Note that I provide a server here to receive the post request and return a response)
The server is simply not sending you a body. You can confirm this with something like wireshark.
If you are getting a body back using postman, you must be sending a different request in postman than in go. It can sometimes be tough to see what is the difference, as both go and postman can sometimes add headers behind the scenes that you don't see. Again, something like wireshark can help here.
Or if you have access to the server, you can add logs there.
I have sites https://a-b-c.com and https://www.x-y-z.com running behind a reverse proxy on ports 4444 and 5555 respectively.
I had them configured to use letsencrypt tls certificates, but now I get an error when using the reverse proxy and I think I need to use a &tls.config{} that includes their certificates but I don't know how to set it up.
My ReverseProxy looks like:
director := func(req *http.Request) {
log.Println(req.Host)
switch req.Host {
case "a-b-c-.com":
req.URL.Host = "localhost:4444"
req.URL.Scheme = "https"
case "x-y-z.com":
req.URL.Host = "localhost:5555"
req.URL.Scheme = "https"
}
proxy := &httputil.ReverseProxy{Director: director}
proxy.Transport = &http.Transport{
Proxy: http.ProxyFromEnvironment,
Dial: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
}).Dial,
TLSHandshakeTimeout: 10 * time.Second,
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
log.Fatalln(http.ListenAndServe(":443", proxy))
You don't have to set up your reverse proxy. What you need to do is to set up HTTP server that you attach your reverse proxy to.
Configure TLS listener by passing two pairs of cert/keys:
abcCrt, err := tls.LoadX509KeyPair("a-b-c.crt", "a-b-c.key")
if err != nil {
panic(err)
}
xyzCrt, err := tls.LoadX509KeyPair("x-y-z.crt", "z-y-z.key")
if err != nil {
panic(err)
}
tlsConfig := &tls.Config{Certificates: []tls.Certificate{abcCrt, xyzCrt}}
ln, err := tls.Listen("tcp", ":443", tlsConfig)
if err != nil {
panic(err)
}
http.Serve(ln, proxy)
I am trying to authenticate a user by calling a REST api, before I proxy the request to a remote server.
However, I find that if I make the api call before proxy to remote server, the request will fail with the following error:
http: proxy error: http: ContentLength=139 with Body length 0.
If I remove the api call before proxy to remote server, the request can get through and return correct response.
My middleware is as following:
func AuthMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// the api call to external auth server
user_id, err := authenticate(r.FormValue("username"), r.FormValue("password"))
if err != nil {
http.Error(w, err.Error(), 401)
return
}
next.ServeHTTP(w, r)
})
}
My reverse proxy is as following:
func NewReverseProxy(target *url.URL) *httputil.ReverseProxy {
director := func(req *http.Request) {
req.URL.Scheme = target.Scheme
req.URL.Host = target.Host
req.URL.Path = target.Path
targetQuery := target.RawQuery
if targetQuery == "" || req.URL.RawQuery == "" {
req.URL.RawQuery = targetQuery + req.URL.RawQuery
} else {
req.URL.RawQuery = targetQuery + "&" + req.URL.RawQuery
}
if _, ok := req.Header["User-Agent"]; !ok {
// explicitly disable User-Agent so it's not set to default value
req.Header.Set("User-Agent", "")
}
}
return &httputil.ReverseProxy{Director: director}
}
And I am using Chi for routing
r.Use(AuthMiddleware)
r.Post("/", NewReverseProxy(targets).ServeHTTP)
What is the issue with this implementation?
If you do not care for the body anymore you can set the contentlength of the request to 0, reflecting the current state of the body:
func AuthMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// the api call to external auth server
user_id, err := authenticate(r.FormValue("username"), r.FormValue("password"))
if err != nil {
http.Error(w, err.Error(), 401)
return
}
r.ContentLength = 0
next.ServeHTTP(w, r)
})
}
I have certificate.pem that I use to perform client authentication with a remote server. When I access the server, normally Chrome pops up, asks if I want to use that certificate, I say yes, then I'm authenticated. I'm trying to figure out why it's not sending the certificate with the dialer when I call it programmatically:
type DialerHelper func() (io.ReadWriter, error)
func DialIt(addr string, port uint16, config *tls.Config) (Dialer, error) {
address := fmt.Sprintf("%s:%d", addr, port)
return DialerHelper(func() (io.ReadWriter, error) {
return tls.Dial("tcp", address, config)
}), nil
}
caPool := x509.NewCertPool()
cert, err := ioutil.ReadFile("certificate.pem")
if err != nil {
panic(err)
}
ok := caPool.AppendCertsFromPEM(cert)
if !ok {
panic(ok)
}
tlsconfig := &tls.Config{
InsecureSkipVerify: true,
RootCAs: caPool, }
tlsconfig.BuildNameToCertificate()
DialIt("some.address.com", 443, tlsconfig)
I keep getting an error from the server saying there is no client certificate supplied. Am I sending the SSL certificate correctly to the remote server? I'm not an expert with SSL.
Edit: this is the functionality I'm trying to replicate: curl -k --cert /home/me/.ssh/certificate.pem
If the server is using a cert generated from your own Certificate Authority, then the following code will do the trick.
I've never tried Client Cert Authentication in an environment where the server cert is from a public CA, so I'm not sure how you'd achieve that. Perhaps just leaving out setting config.RootCAs.
func loadCertificates(caFileName, certFileName, keyFileName string) (tls.Certificate, *x509.CertPool, error) {
myCert, err := tls.LoadX509KeyPair(certFileName, keyFileName)
if err != nil {
return tls.Certificate{}, nil, err
}
ca, err := ioutil.ReadFile(caFileName)
if err != nil {
return tls.Certificate{}, nil, err
}
certPool := x509.NewCertPool()
if !certPool.AppendCertsFromPEM(ca) {
return tls.Certificate{}, nil, errors.New("Failed appending certs")
}
return myCert, certPool, nil
}
func GetClientTlsConfiguration(caFileName, certFileName, keyFileName string) (*tls.Config, error) {
config := &tls.Config{}
myCert, certPool, err := loadCertificates(caFileName, certFileName, keyFileName)
if err != nil {
return nil, err
}
config.Certificates = make([]tls.Certificate, 1)
config.Certificates[0] = myCert
config.RootCAs = certPool
config.ClientCAs = certPool
return config, nil
}
tlsConfig, err := config.GetClientTlsConfiguration("ca.crt", "client.crt", "client.key")
if err != nil {
log.Fatalf("Error loading tls config - %v", err)
}
client := &http.Client{Transport: &http.Transport{TLSClientConfig: tlsConfig}}
client.Get(.....)
In Go I'm using NewSingleHostReverseProxy to perform a reverse proxy, however I need to confirm the SSL certificates of the host site, to make sure I have the correct secure certificate... any ideas how I should do this? Should I be doing this with the handler or transport? I'm new to Go and still getting my head around it.
proxy := httputil.NewSingleHostReverseProxy(&url.URL{
Scheme: "https",
Host: "sha256.badssl.com",
})
http.ListenAndServe("127.0.0.1:80", proxy)
To access the certificate, you will have get access to the ConnectionState. The easiest way to do that is to provide your own version of DialTLS. In there you connect to the server using net.Dial, do the TLS handshake and then you are free to verify.
package main
import (
"crypto/tls"
"log"
"net"
"net/http"
"net/http/httputil"
"net/url"
)
func main() {
proxy := httputil.NewSingleHostReverseProxy(&url.URL{
Scheme: "https",
Host: "sha256.badssl.com",
})
// Set a custom DialTLS to access the TLS connection state
proxy.Transport = &http.Transport{DialTLS: dialTLS}
// Change req.Host so badssl.com host check is passed
director := proxy.Director
proxy.Director = func(req *http.Request) {
director(req)
req.Host = req.URL.Host
}
log.Fatal(http.ListenAndServe("127.0.0.1:3000", proxy))
}
func dialTLS(network, addr string) (net.Conn, error) {
conn, err := net.Dial(network, addr)
if err != nil {
return nil, err
}
host, _, err := net.SplitHostPort(addr)
if err != nil {
return nil, err
}
cfg := &tls.Config{ServerName: host}
tlsConn := tls.Client(conn, cfg)
if err := tlsConn.Handshake(); err != nil {
conn.Close()
return nil, err
}
cs := tlsConn.ConnectionState()
cert := cs.PeerCertificates[0]
// Verify here
cert.VerifyHostname(host)
log.Println(cert.Subject)
return tlsConn, nil
}
To tweak the SSL to the Reverse Host, it is possible to set the transport
options. So if you want to skip the verify you can set it like this.
proxy := httputil.NewSingleHostReverseProxy(&url.URL{
Scheme: "https",
Host: "sha256.badssl.com",
})
proxy.Transport = &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}
http.ListenAndServe("127.0.0.1:80", proxy)