The following the example works very nicely for HTTP: Send a chunked HTTP response from a Go server
As soon as I add TLS, I see that the responses are no longer chunked:
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
flusher, ok := w.(http.Flusher)
if !ok {
panic("expected http.ResponseWriter to be an http.Flusher")
}
w.Header().Set("X-Content-Type-Options", "nosniff")
for i := 1; i <= 10; i++ {
fmt.Fprintf(w, "Chunk #%d\n", i)
flusher.Flush() // Trigger "chunked" encoding and send a chunk...
time.Sleep(500 * time.Millisecond)
}
})
log.Print("Listening on localhost:8080")
log.Fatal(http.ListenAndServeTLS(":8080", "<CERT_FILE>", "<KEY_FILE>", nil))
}
Any ideas why this might be?
log.Infof("Protocol Version: %s", request.Proto)
Confirms its using HTTP/2.0
Thanks Vorsprung
Related
I'm using the Godog library to implement some cucumbers tests for my api code, right now I'm only testing one endpoint but I'm hitting an error where it looks like it's expecting to have a server open. I created a httptest server that listens to port 8080 but the tests are failing with a 404.
If I run my cucumber in debug mode they work but if I use the run test command they fail cos the expect an open port dial tcp localhost:8080. Could someone point me to the right direction since I quite don't know where I'm failing.
This is my godog_test
`
func mockServer() *httptest.Server {
router := mux.NewRouter()
u, _ := url.Parse("http://localhost:8080")
l, _ := net.Listen("tcp", u.Host)
server := httptest.NewUnstartedServer(router)
_ = server.Listener.Close()
server.Listener = l
server.Start()
return server
}
func killMockServer(server *httptest.Server) {
server.Close()
}
func TestFeatures(t *testing.T) {
suite := godog.TestSuite{
TestSuiteInitializer: InitializeTestSuite,
ScenarioInitializer: InitializeScenario,
Options: &godog.Options{
Format: "pretty",
Paths: []string{"features"},
TestingT: t,
},
}
if suite.Run() != 0 {
t.Fatal("non-zero status returned, failed to run feature tests")
}
}
func InitializeTestSuite(ctx *godog.TestSuiteContext) {
var server *httptest.Server
ctx.BeforeSuite(func() {
server = mockServer()
})
ctx.AfterSuite(func() {
fmt.Println("shutting down everything")
killMockServer(server)
})
}
`
Post step that I'm testing
`
func iCallPOSTTo(path string) error {
req, err := json.Marshal(reqBody)
if err != nil {
return err
}
request, err := http.NewRequest(
http.MethodPost,
endpoint+path,
bytes.NewReader(reqBody),
)
res, err := http.DefaultClient.Do(request)
if err != nil {
return err
}
resBody, err := io.ReadAll(res.Body)
if err != nil {
return err
}
res.Body.Close()
[REDACTED]
return nil
}
`
I tried using a mock server to open port 8080 since at first I was receiving a connection refused error, after that I'm getting a 404 which means that my test is not reaching my actual function that processes the post request. I'm not sure if the mock server is the correct approach on this case.
I want to take limit and offset values from my frontend. For that, I have written the following routing path
func (s *Server) stockRoutes() {
s.r.Route("/stock", func(r chi.Router) {
r.Get("/{limit}{offset}", s.ListStocks(s.ctx))
r.Route("/{id}", func(r chi.Router) {
r.Get("/", s.GetStock(s.ctx))
r.Put("/", s.UpdateStockDetails(s.ctx))
})
})
}
I am handling the request in the following fashion. I am parsing the values of limit and offset
func (s *Server) ListStocks(ctx context.Context) http.HandlerFunc {
return func(rw http.ResponseWriter, r *http.Request) {
param, _ := strconv.Atoi(chi.URLParam(r, "limit"))
param2, _ := strconv.Atoi(chi.URLParam(r, "offset"))
limit := int32(param)
offset := int32(param2)
arg := db.ListStocksParams{
Limit: limit,
Offset: offset,
}
stocks, err := s.store.ListStocks(ctx, arg)
if err != nil {
http.Error(rw, "error returning list of stocks", http.StatusInternalServerError)
return
}
log.Printf("%+v", stocks)
json.NewEncoder(rw).Encode(stocks)
}
}
Using postman, I am sending a request in the following way http://localhost:8000/stock?limit=5&offset=0.
Can anyone help me understand what I am doing wrong?
Here api to go which should load the file when posting a request of the form
curl -X POST -d "url = http: //site.com/file.txt" http: // localhost: 8000 / submit
But 404 gets out, what's the reason?
Or how to download files via POST in API?
func downloadFile(url string) Task {
var task Task
resp, err := http.Get(url)
if err != nil {
fmt.Println("Error while downloading")
}
defer resp.Body.Close()
filename := strings.Split(url, "/")[len(strings.Split(url, "/"))-1]
fmt.Println(filename)
out, err := os.Create(filename)
if err != nil {
fmt.Println("Error while downloading")
}
defer out.Close()
_, err = io.Copy(out, resp.Body)
fmt.Println("Error while downloading")
}
func submit(c *gin.Context) {
c.Header("Content-Description", "File Transfer")
c.Header("Content-Transfer-Encoding", "binary")
url := c.Param("url")
fmt.Println("url " + url)
task := downloadFile(url)
hashFile(task.ID)
c.JSON(200, task.ID)
}
func main() {
router := gin.Default()
router.POST("/submit/:url", submit)
}
HTTP status 404 means the server couldn't find the requested URL. This appears to make perfect sense given your curl command. You appear to be requesting the URL http://localhost:8000/submit, but your application only has a single route:
router.POST("/submit/:url", submit)
This route requires a second URL segment after /submit, such as /submit/foo.
I'm doing a post request and I get a 200 OK response. I also receive the headers. However, the body keeps coming back empty.
There should be a body, when I run it in postman the body shows up. What am I missing here?
func AddHealthCheck(baseURL string, payload HealthCheck, platform string, hostname string) (string, error) {
url := fmt.Sprintf(baseURL+"add-healthcheck/%s/%s", platform, hostname)
//convert go struct to json
jsonPayload, err := json.Marshal(payload)
if err != nil {
log.Error("[ADD HEALTH CHECK] Could not convert go struct to json : ", err)
return "", err
}
// Create client & set timeout
client := &http.Client{}
client.Timeout = time.Second * 15
// Create request
req, err := http.NewRequest("POST", url, bytes.NewBuffer(jsonPayload))
if err != nil {
log.Error("[ADD HEALTH CHECK] Could not create request : ", err)
return "", err
}
req.Header.Set("Content-Type", "application/json")
// Fetch Request
resp, err := client.Do(req)
if err != nil {
log.Error("[ADD HEALTH CHECK] Could not fetch request : ", err)
return "", err
}
defer resp.Body.Close()
// Read Response Body
respBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Error("[HEALTH CHECK] Could not read response body : ", err)
return "", err
}
fmt.Println("response Status : ", resp.Status)
fmt.Println("response Headers : ", resp.Header)
fmt.Println("response Body : ", string(respBody))
return string(respBody), nil
}
I have confirmed locally that your code, as shown, should work.
Here is the code I used:
package main
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"time"
)
func main() {
http.HandleFunc("/", handler)
go func(){
http.ListenAndServe(":8080", nil)
}()
AddHealthCheck()
}
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hi there")
}
func panicError(err error) {
if err != nil {
panic(err)
}
}
func AddHealthCheck() (string, error) {
//convert go struct to json
payload := "bob"
jsonPayload, err := json.Marshal(payload)
panicError(err)
// Create client & set timeout
client := &http.Client{}
client.Timeout = time.Second * 15
// Create request
req, err := http.NewRequest("POST", "http://localhost:8080", bytes.NewBuffer(jsonPayload))
panicError(err)
req.Header.Set("Content-Type", "application/json")
// Fetch Request
resp, err := client.Do(req)
panicError(err)
defer resp.Body.Close()
// Read Response Body
respBody, err := ioutil.ReadAll(resp.Body)
panicError(err)
fmt.Println("response Status : ", resp.Status)
fmt.Println("response Headers : ", resp.Header)
fmt.Println("response Body : ", string(respBody))
return string(respBody), nil
}
The code above is just a slightly stripped down version of your code, and it outputs the body of the response. (Note that I provide a server here to receive the post request and return a response)
The server is simply not sending you a body. You can confirm this with something like wireshark.
If you are getting a body back using postman, you must be sending a different request in postman than in go. It can sometimes be tough to see what is the difference, as both go and postman can sometimes add headers behind the scenes that you don't see. Again, something like wireshark can help here.
Or if you have access to the server, you can add logs there.
I am trying to authenticate a user by calling a REST api, before I proxy the request to a remote server.
However, I find that if I make the api call before proxy to remote server, the request will fail with the following error:
http: proxy error: http: ContentLength=139 with Body length 0.
If I remove the api call before proxy to remote server, the request can get through and return correct response.
My middleware is as following:
func AuthMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// the api call to external auth server
user_id, err := authenticate(r.FormValue("username"), r.FormValue("password"))
if err != nil {
http.Error(w, err.Error(), 401)
return
}
next.ServeHTTP(w, r)
})
}
My reverse proxy is as following:
func NewReverseProxy(target *url.URL) *httputil.ReverseProxy {
director := func(req *http.Request) {
req.URL.Scheme = target.Scheme
req.URL.Host = target.Host
req.URL.Path = target.Path
targetQuery := target.RawQuery
if targetQuery == "" || req.URL.RawQuery == "" {
req.URL.RawQuery = targetQuery + req.URL.RawQuery
} else {
req.URL.RawQuery = targetQuery + "&" + req.URL.RawQuery
}
if _, ok := req.Header["User-Agent"]; !ok {
// explicitly disable User-Agent so it's not set to default value
req.Header.Set("User-Agent", "")
}
}
return &httputil.ReverseProxy{Director: director}
}
And I am using Chi for routing
r.Use(AuthMiddleware)
r.Post("/", NewReverseProxy(targets).ServeHTTP)
What is the issue with this implementation?
If you do not care for the body anymore you can set the contentlength of the request to 0, reflecting the current state of the body:
func AuthMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// the api call to external auth server
user_id, err := authenticate(r.FormValue("username"), r.FormValue("password"))
if err != nil {
http.Error(w, err.Error(), 401)
return
}
r.ContentLength = 0
next.ServeHTTP(w, r)
})
}