What I'm trying to do:
Build a package (later usage) that provides a method to execute a get-request to any page through a given socks5 proxy.
My problem:
When ever I try to request a page with SSL (https) I get the following error:
Error executing request Get https://www.xxxxxxx.com: socks connect tcp 83.234.8.214:4145->www.xxxxxxx.com:443: EOF
However requesting http://www.google.com is working fine. So there must be a problem with the SSL connection. Can't imagine why this isn't working as I'm not very experienced with SSL-connections. End of file makes no sense to me.
My current code:
func main() {
// public socks5 - worked when I created this question
proxy_addr := "83.234.8.214:4145"
// With this address I get the error
web_addr := "https://www.whatismyip.com"
// Requesting google works fine
//web_addr := "http://www.google.com"
dialer, err := proxy.SOCKS5("tcp", proxy_addr, nil, proxy.Direct)
handleError(err, "error creating dialer")
httpTransport := &http.Transport{}
httpClient := &http.Client{Transport: httpTransport}
httpTransport.DialTLS = dialer.Dial
req, err := http.NewRequest("GET", web_addr, nil)
handleError(err, "error creating request")
httpClient.Timeout = 5 * time.Second
resp, err := httpClient.Do(req)
handleError(err, "error executing request")
defer resp.Body.Close()
b, err := ioutil.ReadAll(resp.Body)
handleError(err, "error reading body")
fmt.Println(string(b))
}
func handleError(err error, msg string) {
if err != nil {
log.Fatal(err)
}
}
So what am I missing in here to deal with ssl-connections?
Thank you very much.
Edit 1:
In case someone would think this is an issue with whatismyip.com I've done some more tests:
https://www.google.com
EOF error
https://stackoverflow.com
EOF error
https://www.youtube.com/
EOF error
Connection between your program and your socks5 proxy goes not through SSL/TLS
So you should change line
httpTransport.DialTLS = dialer.Dial
to
httpTransport.Dial = dialer.Dial
I checked https://www.whatismyip.com and https://www.google.com.
URLs are downloaded fine.
For test I set up 3proxy service on my server, test your code with fixed line and check 3proxy logs.
All made requests was in proxy server logs.
If you need more help - please let me know, I'll help
Things to notice:
Socks5 proxies need to support SSL connections.
The code from the question won't work with this answer as the proxy (used in the code) isn't supporting SSL connections.
Related
I find myself needing to set up a WebSocket connection in a hostile environment in which a firewall sniffs SNI information from TLS which I'd rather it didn't. In my particular case, the WebSocket server does not use SNI for request handling, so as such, the SNI part of the handshake could be safely removed.
My question then becomes: In the golang.org WebSocket package, golang.org/x/net/websocket, what is the simplest way to strip SNI information while retaining validation of the provided chain?
The best I have been able to come up with is to simply replace the hostname of the URL to be dialled with its corresponding IP. This causes crypto/tls to never add the problematic SNI information, but, in the solution I was able to come up with, a custom validator ends up having to be provided to validate the chain:
func dial(url string, origin string) (*websocket.Conn, error) {
// Use system resolver to get IP of host
hostRegExp := regexp.MustCompile("//([^/]+)/")
host := hostRegExp.FindStringSubmatch(url)[1]
addrs, err := net.LookupHost(host)
if err != nil {
return fmt.Errorf("Could not resolve address of %s: %v", host, err)
}
ip := addrs[0]
// Replace the hostname in the given URL with its IP instead
newURL := strings.Replace(url, host, ip, 1)
config, _ := websocket.NewConfig(newURL, origin)
// As we have removed the hostname, the Go TLS package will not know what to
// validate the certificate DNS names against, so we have to provide a custom
// verifier based on the hostname we threw away.
config.TlsConfig = &tls.Config{
InsecureSkipVerify: true,
VerifyPeerCertificate: verifier(host),
}
return websocket.DialConfig(config)
}
func verifier(host string) func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
return func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
// For simplicity, let us only consider the case in which the first certificate is the one
// to validate, and in which it is signed directly by a CA, with no parsing of
// intermediate certificates required.
opts := x509.VerifyOptions{
DNSName: host,
}
rawCert := rawCerts[0]
cert, err := x509.ParseCertificate(rawCert)
if err != nil {
return err
}
_, err = cert.Verify(opts)
return err
}
}
This totally works but seems rather clunky. Is there a simpler approach? (Ideally one that is not specific to WebSocket applications but works for TLS in general; the exact same idea as above could be applied to HTTPS.)
I'm trying to set up some logging/monitoring for a Golang application. I want to be alerted if an SSL handshake error occurs. In Java I looked for the string "javax.net.ssl.SSLHandshakeException". Is there an equivalent in Golang for when something goes wrong with an SSL handshake?
Golang has no exceptions, but funcs return errors. This code would set err if the handshake fails:
conf := &tls.Config{}
tlsCon, err := tls.Dial("tcp", "example.com:443", conf)
if err != nil { // err is set when handshake fails
fmt.Println(err.Error())
return
}
tlsCon.Close()
i am trying to make one get in https://www.google.com using proxy with authentication, i already passing header parameter
Proxy-Authorization
but proxy server return
Proxy Authentication Required
code:
package main
import (
"crypto/tls"
"encoding/base64"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"os"
)
func main() {
req, _ := http.NewRequest("GET", "https://www.google.com.br", nil)
req.Header.Set("Host", "www.google.com.br")
proxyURL := url.URL{
Host: "IP-HERE:PORT-HEERE"}
transport := &http.Transport{
Proxy: http.ProxyURL(&proxyURL),
TLSClientConfig: &tls.Config{},
}
client := &http.Client{Transport: transport}
req.RequestURI = ""
auth := fmt.Sprintf("USER:PASSWORD")
basic := "Basic " + base64.StdEncoding.EncodeToString([]byte(auth))
req.Header.Add("Proxy-Authorization", basic)
resp, err := client.Do(req)
if err != nil {
fmt.Printf("erro: %s", err)
}
fmt.Printf("code: %s", resp.StatusCode)
htmlData, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Println(os.Stdout, string(htmlData))
}
I have to pass another parameter?
When i perform one get in http://www.google.com.br, without https.. proxy authentication with success. why?
I found the solution. Golang don't pass header parameters in method CONNECT, then parameter Proxy-Authorization aren't sent to proxy server.
to resolve this, field ProxyConnectHeader was added to struct Transport. but this change until released, this change are only in master.
to test this new field using golang from master, i made on project in github and worked.
link to golang path
link to project in github that test this new field
I'm starting to learn golang and I'm trying to make a simple http client that will get a list of virtual machines from one of our oVirt clusters. The API that I'm trying to access has a self-signed certificate (auto generated during the cluster installation) and golang's http.client encounters a problem when serializing the time from the certificate. Below you can find the code and the output.
package main
import (
"fmt"
"io/ioutil"
"net/http"
"crypto/tls"
)
func do_request(url string) ([]byte, error) {
// ignore self signed certificates
transCfg := &http.Transport{
TLSClientConfig: &tls.Config {
InsecureSkipVerify: true,
},
}
// http client
client := &http.Client{Transport: transCfg}
// request with basic auth
req, _ := http.NewRequest("GET", url, nil)
req.SetBasicAuth("user","pass")
resp, err := client.Do(req)
// error?
if err != nil {
fmt.Printf("Error : %s", err)
return nil, err
}
defer resp.Body.Close()
body, _ := ioutil.ReadAll(resp.Body)
return []byte(body), nil
}
func main() {
body, _ := do_request("https://ovirt.example.com/")
fmt.Println("response Status:", string(body))
}
and the error when I'm trying to compile:
$ go run http-get.go
Error : Get https://ovirt.example.com/: tls: failed to parse certificate from server: asn1: time did not serialize back to the original value and may be invalid: given "141020123326+0000", but serialized as "141020123326Z"response Status:
Is there any way to ignore this verification? I tried making a request using other programming languages (python, ruby) and skipping insecure certificates seems to be enough.
Thank you!
PS: I know the proper solution is to change the certificate with a valid one, but for the moment I cannot do this.
Unfortunately, you've encountered an error that you cannot get around in Go. This is buried deep in the cypto/x509 and encoding/asn1 packages without a way to ignore. Specifically, asn1.parseUTCTime is expecting the time format to be "0601021504Z0700", but your server is sending "0601021504+0000". Technically, that is a known format but encoding/asn1 does not support it.
There are only 2 solutions that I can come up with that do not require a code change for golang.
1) Edit the encoding/asn1 package in your go src directory and then rebuild all the standard packages with go build -a
2) Create your own customer tls, x509 and asn1 packages to use the format your server is sending.
Hope this helps.
P.S. I've opened an issue with the Go developers to see if it can resolved by them at some later point Issue Link
Possible ASN1 UtcTime Formats.
How can I check the fingerprints of the server SSL/TLS certificates during a http request in golang?
This ruby code shows what I want to do in Go:
#verify_callback = proc do |preverify_ok, store_context|
if preverify_ok and store_context.error == 0
certificate = OpenSSL::X509::Certificate.new(store_context.chain[0])
fingerprint = Digest::SHA1.hexdigest(certificate.to_der).upcase.scan(/../).join(":")
$valid_fingerprints.include?(fingerprint)
else
false
end
end
In general the process of generating a certificate fingerprint in Go is pretty simple. If you already have an x509.Certificate struct, stored in cert, all you need to do is
sha1Fingerprint := sha1.Sum(cert.Raw)
Getting certificates from an HTTP response struct after the request is complete is also pretty easy (use resp.TLS.PeerCertificates), but it doesn't seem like that's what you need.
If you need access to the server's certificate at TLS connection set up time, I think you'll need to create your own http.Transport and hand it a custom implementation of DialTLS. You'd then use that transport when configuring an http.Client to make your outbound requests.
Within your custom DialTLS func you'd have access to connection state information like the server's certificate chain, and you could perform the SHA1 fingerprint generation from there.
You probably shouldn't implement certificate checking yourself, but let the net/http do the checking based on the valid CAs you provide. Also, usually working directly with fingerprints isn't worth the trouble.
For example, this is how you set up a HTTPS server that requires clients to authenticate by using a certificate. The client certificate must be signed by the CA, or the SSL/TLS handshake stops.
// Server's own certificate & key
cert, err := tls.LoadX509KeyPair("server.crt", "server.key")
if err != nil {
panic(err)
}
// Load the CA certificate(s)
capool := x509.NewCertPool()
cacert, err := ioutil.ReadFile("ca.crt")
if err != nil {
panic(err)
}
capool.AppendCertsFromPEM(cacert)
// Server configuration
config := tls.Config{Certificates: []tls.Certificate{cert}, ClientCAs: capool, ClientAuth: tls.RequireAndVerifyClientCert}
config.NextProtos = []string{"http/1.1"}
config.Rand = rand.Reader // Strictly not necessary, should be default
// TLS web server
myTLSWebServer := &http.Server{Addr: "myaddress", TLSConfig: &config, Handler: nil}
// .. proceed with setting handlers etc
http.HandleFunc("/", myHandler)
// Bind to port and start the server up
conn, err := net.Listen("tcp", settings.ServiceAddress)
if err != nil {
panic(err)
}
tlsListener := tls.NewListener(conn, &config)
myTLSWebServer.Serve(tlsListener)
Reading the documentation for tls.Config will show you that by changing the parameters (ClientAuth, ClientCAs, Certificates, RootCAs) you can easily select different modes for checking the certificates. You usually get failures returned in error.
If you really insist on checking fingerprints, you can retrieve the TLS status from Request TLS *tls.ConnectionState. I think you should probably use the Signature from that struct for fingerprinting.. Off the top of my head, something roughly along the lines of
func lol(r *http.Request) {
tls := r.TLS
if tls != nil {
// Try the first one for simplicity
cert := tls.PeerCertificates[0]
signature := cert.Signature
// Do something with the signature
}
}
should do the trick.