How can I fix a TLS handshake error using Go on Heroku? - ssl

I have a simple proxy that listens to HTTP and HTTPS connections on the same port.
However I'm seeing weird TLS handshake errors that seem to be caused by the Heroku router:
heroku[router]: sock=backend at=error code=H18 desc="Server Request Interrupted" method=GET path="/favicon.ico" host=banana.camp request_id=f56144c8-e480-476a-90b8-429b490f1ff5 fwd="24.67.185.77" dyno=web.1 connect=0ms service=1ms status=503 bytes=7 protocol=https
http: TLS handshake error from 10.81.159.108:19578: tls: first record does not look like a TLS handshake
http: TLS handshake error from 172.17.117.25:36924: EOF
Is there any way to fix or ignore them? Any clues are greatly appreciated.
Thanks!
func InitServer() error {
m := autocert.Manager{
Cache: autocert.DirCache(*StorageDir),
Prompt: autocert.AcceptTOS,
HostPolicy: func(ctx context.Context, host string) error {
return nil
},
}
errchan := make(chan error)
s := &http.Server{
Addr: ":" + os.Getenv("PORT"),
TLSConfig: &tls.Config{GetCertificate: m.GetCertificate},
Handler: ServeHTTP(),
}
go (func() {
errchan <- http.ListenAndServe(":"+os.Getenv("PORT"), m.HTTPHandler(nil))
})()
errchan <- s.ListenAndServeTLS("", "")
return <-errchan
}
func ServeHTTP() http.Handler {
return http.HandlerFunc(func(res http.ResponseWriter, req *http.Request) {
if upstreams, ok := Hosts[req.Host]; ok {
forwarder, _ := forward.New(forward.PassHostHeader(true))
loadbalancer, _ := roundrobin.New(forwarder)
for _, upstream := range upstreams {
if url, err := url.Parse(upstream); err == nil {
loadbalancer.UpsertServer(url)
} else {
log.Fatal("InitHostsList: ", err)
}
}
loadbalancer.ServeHTTP(res, req)
return
}
http.Error(res, "The request service couldn't be found here", http.StatusNotImplemented)
})
}

TLS termination is handled by the Heroku router and can't be done at the app level ¯_(ツ)_/¯

You can't listen to both http and https on the same port, and those log-errors are cases when you have http clients try to connect to the app.

By default, the golang http server only supports listening to http or https traffic on a given port (not both). To listen to https traffic, you need to use http.ListenAndServeTLS.
If you are directing both http and https traffic at at point configured for only one, you will end up seeing errors.
There are third-party solution (like cmux) that support hosting both secure and insecure traffic on the same port. Not as easy as separating the ports, but if it's something you want to do, they have an example configuration in the README.

Related

Http get request through socks5 | EOF error

What I'm trying to do:
Build a package (later usage) that provides a method to execute a get-request to any page through a given socks5 proxy.
My problem:
When ever I try to request a page with SSL (https) I get the following error:
Error executing request Get https://www.xxxxxxx.com: socks connect tcp 83.234.8.214:4145->www.xxxxxxx.com:443: EOF
However requesting http://www.google.com is working fine. So there must be a problem with the SSL connection. Can't imagine why this isn't working as I'm not very experienced with SSL-connections. End of file makes no sense to me.
My current code:
func main() {
// public socks5 - worked when I created this question
proxy_addr := "83.234.8.214:4145"
// With this address I get the error
web_addr := "https://www.whatismyip.com"
// Requesting google works fine
//web_addr := "http://www.google.com"
dialer, err := proxy.SOCKS5("tcp", proxy_addr, nil, proxy.Direct)
handleError(err, "error creating dialer")
httpTransport := &http.Transport{}
httpClient := &http.Client{Transport: httpTransport}
httpTransport.DialTLS = dialer.Dial
req, err := http.NewRequest("GET", web_addr, nil)
handleError(err, "error creating request")
httpClient.Timeout = 5 * time.Second
resp, err := httpClient.Do(req)
handleError(err, "error executing request")
defer resp.Body.Close()
b, err := ioutil.ReadAll(resp.Body)
handleError(err, "error reading body")
fmt.Println(string(b))
}
func handleError(err error, msg string) {
if err != nil {
log.Fatal(err)
}
}
So what am I missing in here to deal with ssl-connections?
Thank you very much.
Edit 1:
In case someone would think this is an issue with whatismyip.com I've done some more tests:
https://www.google.com
EOF error
https://stackoverflow.com
EOF error
https://www.youtube.com/
EOF error
Connection between your program and your socks5 proxy goes not through SSL/TLS
So you should change line
httpTransport.DialTLS = dialer.Dial
to
httpTransport.Dial = dialer.Dial
I checked https://www.whatismyip.com and https://www.google.com.
URLs are downloaded fine.
For test I set up 3proxy service on my server, test your code with fixed line and check 3proxy logs.
All made requests was in proxy server logs.
If you need more help - please let me know, I'll help
Things to notice:
Socks5 proxies need to support SSL connections.
The code from the question won't work with this answer as the proxy (used in the code) isn't supporting SSL connections.

Stripping SNI information from TLS WebSocket connections

I find myself needing to set up a WebSocket connection in a hostile environment in which a firewall sniffs SNI information from TLS which I'd rather it didn't. In my particular case, the WebSocket server does not use SNI for request handling, so as such, the SNI part of the handshake could be safely removed.
My question then becomes: In the golang.org WebSocket package, golang.org/x/net/websocket, what is the simplest way to strip SNI information while retaining validation of the provided chain?
The best I have been able to come up with is to simply replace the hostname of the URL to be dialled with its corresponding IP. This causes crypto/tls to never add the problematic SNI information, but, in the solution I was able to come up with, a custom validator ends up having to be provided to validate the chain:
func dial(url string, origin string) (*websocket.Conn, error) {
// Use system resolver to get IP of host
hostRegExp := regexp.MustCompile("//([^/]+)/")
host := hostRegExp.FindStringSubmatch(url)[1]
addrs, err := net.LookupHost(host)
if err != nil {
return fmt.Errorf("Could not resolve address of %s: %v", host, err)
}
ip := addrs[0]
// Replace the hostname in the given URL with its IP instead
newURL := strings.Replace(url, host, ip, 1)
config, _ := websocket.NewConfig(newURL, origin)
// As we have removed the hostname, the Go TLS package will not know what to
// validate the certificate DNS names against, so we have to provide a custom
// verifier based on the hostname we threw away.
config.TlsConfig = &tls.Config{
InsecureSkipVerify: true,
VerifyPeerCertificate: verifier(host),
}
return websocket.DialConfig(config)
}
func verifier(host string) func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
return func(rawCerts [][]byte, verifiedChains [][]*x509.Certificate) error {
// For simplicity, let us only consider the case in which the first certificate is the one
// to validate, and in which it is signed directly by a CA, with no parsing of
// intermediate certificates required.
opts := x509.VerifyOptions{
DNSName: host,
}
rawCert := rawCerts[0]
cert, err := x509.ParseCertificate(rawCert)
if err != nil {
return err
}
_, err = cert.Verify(opts)
return err
}
}
This totally works but seems rather clunky. Is there a simpler approach? (Ideally one that is not specific to WebSocket applications but works for TLS in general; the exact same idea as above could be applied to HTTPS.)

how to disable responding to http1 requests in nginx or apache or caddy?

I have a http2 server but by default it responds http1 requests.
I want to stop my server responding to http1 requests?
Most browsers might use alpn or npn. is there a possibility to advertise only http2 ? or a custom list of application protocols ?
In Caddy, if you're comfortable modifying the source code, you can make the following changes in caddyhttp/httpserver/server.go.
Change the line that says:
var defaultALPN = []string{"h2", "http/1.1"}
…so that it says:
var defaultALPN = []string{"h2"}
This will prevent it from advertising HTTP/1.1 via ALPN.
Then add this code to the beginning of the ServeHTTP method:
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
if !r.ProtoAtLeast(2, 0) {
if hj, ok := w.(http.Hijacker); ok {
conn, _, err := hj.Hijack()
if err == nil {
conn.Close()
}
}
return
}
…
}
This will immediately close the connection without sending headers if any protocol earlier than HTTP/2 is used.
You could use something like below rule to block all http1 requests. But it is not advisable to do this.
if ($server_protocol ~* "HTTP/1*") {
return 444;
}

Access HTTPS via HTTP proxy with basic authentication

I am trying to make a GET request against an HTTPS URL using proxy with username/password authorization (auth is required by the proxy not the website).
Here's what I do:
package main
import (
"crypto/tls"
"encoding/base64"
"fmt"
"net/http"
"net/http/httputil"
"net/url"
)
func main() {
ua := "Mozilla/5.0 (Windows NT 6.1"
basic := "Basic " + base64.StdEncoding.EncodeToString([]byte("username:mypass"))
req, err := http.NewRequest("GET", "https://api.ipify.org/", nil)
proxyUrl, _ := url.Parse("http://myproxy.com:9999")
fmt.Println(basic) // Basic dXNlcm5hbWU6bXlwYXNz
req.Header.Add("Proxy-Authorization", basic)
req.Header.Add("User-Agent", ua)
bb, _ := httputil.DumpRequest(req, false)
fmt.Println(string(bb))
/*
Get / HTTP/1.1
Host: api.ipify.org
Proxy-Authorization: Basic dXNlcm5hbWU6bXlwYXNz
User-Agent: Mozilla/5.0 (Windows NT 6.1
*/
client := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
Proxy: http.ProxyURL(proxyUrl),
},
}
resp, err := client.Do(req)
fmt.Println(err) // Proxy Authentication Required
fmt.Println(resp) // <nil>
}
The catch is that when I try to do a request to an HTTP (not HTTPS) site it goes fine, but if I make HTTPS request it fails (see above the message).
I tested the proxy with my browser (FireFox) and everything goes well, I looked for the headers through firebug and added everything relevant to the request (above). I've double-triple-checked the basic value and everything else but without any luck.
So does some one have any idea why this happens or at least how do I research the problem?
The last thing to add is that I can use a public HTTP proxy (that doesn't require any auth) in this case and problems seem to start when auth enters in this process (the error also suggests that).
P.S. Unfortunately I cannot share the proxy IP, port and username cause it is against their policy.
package main
import (
"crypto/tls"
"fmt"
"net/url"
"net/http"
)
func main() {
req, err := http.NewRequest("GET", "https://api.ipify.org/", nil)
proxyUrl, _ := url.Parse("http://username:password#127.0.0.1:9999")
client := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
Proxy: http.ProxyURL(proxyUrl),
},
}
_, err = client.Do(req)
fmt.Println(err)
}

Golang ReverseProxy with Apache2 SNI/Hostname error

i am writing my own ReverseProxy in Go.The ReverseProxy should connect my go-webserver and my apache2 webserver. But when I run my reverseproxy on another IP-Adress then my Apache2 webserver I got following error in my apache-logfile, when the reverseproxy sends the request to apache.
"Hosname xxxx provided via sni and hostname xxxx2 provided via http are different"
My Reverse Proxy and apache-webserver running on https.
Here some code:
func (p *Proxy) directorApache(req *http.Request) {
mainServer := fmt.Sprintf("%s:%d", Config.HostMain, Config.PortMain)
req.URL.Scheme = "https"
req.URL.Host = mainServer
}
func (p *Proxy) directorGo(req *http.Request) {
goServer := fmt.Sprintf("%s:%d", Config.GoHost, Config.GoPort)
req.URL.Scheme = "http"
req.URL.Host = goServer
}
func (p *Proxy) ServeHTTP(rw http.ResponseWriter, req *http.Request) {
fmt.Println(req.URL.Path)
if p.isGoRequest(req) {
fmt.Println("GO")
p.goProxy.ServeHTTP(rw, req)
return
}
p.httpProxy.ServeHTTP(rw, req)
}
func main() {
var configPath = flag.String("conf", "./configReverse.json", "Path to the Json config file.")
flag.Parse()
proxy := New(*configPath)
cert, err := tls.LoadX509KeyPair(Config.PathCert, Config.PathPrivateKey)
if err != nil {
log.Fatalf("server: loadkeys: %s", err)
}
config := tls.Config{InsecureSkipVerify: true, Certificates: []tls.Certificate{cert}}
listener, err := net.Listen("tcp",
net.JoinHostPort(proxy.Host, strconv.Itoa(proxy.Port)))
if err != nil {
log.Fatalf("server: listen: %s", err)
}
log.Printf("server: listening on %s")
proxy.listener = tls.NewListener(listener, &config)
serverHTTPS := &http.Server{
Handler: proxy.mux,
TLSConfig: &config,
}
if err := serverHTTPS.Serve(proxy.listener); err != nil {
log.Fatal("SERVER ERROR:", err)
}
}
Perhaps someone has a idea about that issue.
Short example
Say you're starting an HTTP request to https://your-proxy.local. Your request handler takes the http.Request struct and rewrites its URL field to https://your-apache-backend.local.
What you have not considered, is that the original HTTP request also contained a Host header (Host: your-proxy.local). When passing that same request to http://your-apache-backend.local, the Host header in that request still says Host: your-proxy.local. And that's what Apache is complaining about.
Explanation
As you're using TLS with Server Name Indication (SNI), the request hostname will not only be used for DNS resolution, but also to select the SSL certificate that should be used to establish the TLS connection. The HTTP 1.1 Host header on the other hand is used to distinguish several virtual hosts by Apache. Both names must match. This issue is also mentioned in the Apache HTTPD wiki:
SNI/Request hostname mismatch, or SNI provides hostname and request doesn't.
This is a browser bug. Apache will reject the request with a 400-type error.
Solution
Also rewrite the Host header. If you want to preserve the original Host header, you can store it in an X-Forwarded-Host header (that's a non-standard header, but it's widely used in reverse proxies):
func (p *Proxy) directorApache(req *http.Request) {
mainServer := fmt.Sprintf("%s:%d", Config.HostMain, Config.PortMain)
req.URL.Scheme = "https"
req.URL.Host = mainServer
req.Header.Set("X-Forwarded-Host", req.Header().Get("Host"))
req.Host = mainServer
}