I wrote this very basic parser that goes through Reddit JSON and am curious how I can specifically manage an error in Go.
For example I have this "Get" method for a link:
func Get(reddit string) ([]Item, error) {
url := fmt.Sprintf("http://reddit.com/r/%s.json", reddit)
resp, err := http.Get(url)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, err
}
/*
* Other code here
*/
}
How can I handle, say, a 404 error from the StatusCode? I know I can test for the 404 error itself:
if resp.StatusCode == http.StatusNotfound {
//do stuff here
}
But is there a way I can directly manage the resp.StatusCode != http.StatusOK without having to write a bunch of if statements? Is there a way I can use err in a switch statement?
Firstly note that http.Get doesn't return an error for an HTTP return which isn't 200. The Get did its job successfully even when the server gave it a 404 error. From the docs
A non-2xx response doesn't cause an error.
Therfore in your code, err will be nil when you call this which means it will return err=nil which probably isn't what you want.
if resp.StatusCode != http.StatusOK {
return nil, err
}
This should do what you want
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("HTTP Error %d: %s", resp.StatusCode, resp.Status)
}
Which will return an error for any kind of HTTP error, with a message as to what it was.
Sure you can:
package main
import "fmt"
func main() {
var err error
switch err {
case nil:
fmt.Println("Is nil")
}
}
https://play.golang.org/p/4VdHW87wCr
Related
I am trying to make a function where it hosts your file on the anonfiles.com website using the anonfiles API. Even thought I am correctly using the api, it always returns nil. Response is missing message.
func host(file string) {
fileBytes, err := ioutil.ReadFile(file)
if err != nil {
fmt.Println("\033[1;31mCommand > Host: Could not read file,", err, "\033[0m")
return
}
url := "https://api.anonfiles.com/upload"
request, err := http.NewRequest("POST", url, bytes.NewBuffer(fileBytes))
if err != nil {
fmt.Println("\033[1;31mCommand > Host: Could not post request,", err, "\033[0m")
return
}
request.Header.Set("Content-Type", "application/octet-stream")
client := &http.Client{}
response, err := client.Do(request)
if err != nil {
fmt.Println("\033[1;31mCommand > Host: Could not send request,", err, "\033[0m")
return
}
defer response.Body.Close()
body, err := ioutil.ReadAll(response.Body)
if err != nil {
fmt.Println("\033[1;31mCommand > Host: Could not read response,", err, "\033[0m")
return
}
var result map[string]interface{}
err = json.Unmarshal(body, &result)
if err != nil {
fmt.Println("\033[1;31mCommand > Host: Could not parse response,", err, "\033[0m")
return
}
if response.StatusCode == 200 {
if result["url"] == nil {
fmt.Println("\033[1;31mCommand > Host: Response is missing URL\033[0m")
return
}
fmt.Println("File hosted successfully:", result["url"].(string))
} else {
if result["message"] == nil {
fmt.Println("\033[1;31mCommand > Host: Response is missing message\033[0m")
return
}
fmt.Println("\033[1;31mCommand > Host:\033[0m", result["message"].(string))
}
}
I'd thought I'd take a moment to expand those comments into an answer.
First, as we we've already discussed, you're not using the correct API to upload files. If we modify your code to show the complete response body, like this:
client := &http.Client{}
response, err := client.Do(request)
if err != nil {
fmt.Println("\033[1;31mCommand > Host: Could not send request,", err, "\033[0m")
return
}
defer response.Body.Close()
body, err := ioutil.ReadAll(response.Body)
if err != nil {
fmt.Println("\033[1;31mCommand > Host: Could not read response,", err, "\033[0m")
return
}
fmt.Printf("BODY:\n%s\n", body)
We see the following:
{
"status": false,
"error": {
"message": "No file chosen.",
"type": "ERROR_FILE_NOT_PROVIDED",
"code": 10
}
}
We're getting this error because you're not providing the file parameter in a multipart/form-data request. The post to which I linked earlier has several examples of sending a multipart request; I've tested a couple of them and they seem to work as expected.
You're also making incorrect assumptions about the response returned by the API. If we make a successful request using curl and capture the response JSON, we find that it looks like this:
{
"status": true,
"data": {
"file": {
"url": {
"full": "https://anonfiles.com/k8cdobWey7/test_txt",
"short": "https://anonfiles.com/k8cdobWey7"
},
"metadata": {
"id": "k8cdobWey7",
"name": "test.txt",
"size": {
"bytes": 12,
"readable": "12 B"
}
}
}
}
}
Note that there is no response["url"] or response["message"]. If you want the URL for the uploaded file, you need to get response["data"]["file"]["url"]["full"] (or ["short"]).
Similarly, we can see examples of the error response above, which looks like this:
{
"status": false,
"error": {
"message": "No file chosen.",
"type": "ERROR_FILE_NOT_PROVIDED",
"code": 10
}
}
That's not result["message"]; that's result["error"]["message"].
Because you're unmarshalling into a map[string]interface, getting at these nested keys is going to be a bit of a pain. I found it easiest to create Go structs for the above responses, and just unmarshal into an appropriately typed variable.
That gets me the following types:
type (
AnonFilesUrl struct {
Full string `json:"full"`
Short string `json:"short"`
}
AnonFilesMetadata struct {
ID string `json:"id"`
Name string `json:"name"`
Size struct {
Bytes int `json:"bytes"`
Readable string `json:"readable"`
} `json:"size"`
}
AnonFilesData struct {
File struct {
URL AnonFilesUrl `json:"url"`
Metadata AnonFilesMetadata `json:"metadata"`
} `json:"file"`
}
AnonFilesError struct {
Message string
Type string
Code int
}
AnonFilesResponse struct {
Status bool `json:"status"`
Data AnonFilesData `json:"data"`
Error AnonFilesError `json:"error"`
}
)
And then unmarshalling the response looks like:
var result AnonFilesResponse
err = json.Unmarshal(body, &result)
And we can ask for fields like:
fmt.Printf("URL: %s\n", result.Data.File.URL.Full)
I'm getting 400 Bad Request for frappe.cloud API, when I'm trying to call it using golang code using http.NewRequest, this API is working fine when I check it using postman. following is the API
https://xxxx.frappe.cloud/api/resource/Item?fields=["name","item_name","item_group","description"]&filters=[["Item","item_group","=","xxx Product"]]
If I use the same golang code to call same API with out filters it works fine. following is the working API
https://xxxx.frappe.cloud/api/resource/Item?fields=["name","item_name","item_group","description"]
code as follows
func FetchProperties(dataChannel models.DataChannel) (map[string]interface{}, error) {
thisMap := make(map[string][]map[string]interface{})
client := &http.Client{}
req, err := http.NewRequest("GET", dataChannel.APIPath, nil)
if err != nil {
commons.ErrorLogger.Println(err.Error())
return nil, err
}
eds, err := GetDecryptedEDSByEDSID(dataChannel.EDSId)
if err != nil {
commons.ErrorLogger.Println(err.Error())
return nil, &commons.RequestError{StatusCode: 400, Err: err}
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", eds.DataSource.Auth.Token)
response, err := client.Do(req)
if err != nil {
commons.ErrorLogger.Println(err.Error())
return nil, err
}
data, err := ioutil.ReadAll(response.Body)
if err != nil {
commons.ErrorLogger.Println(err.Error())
return nil, &commons.RequestError{StatusCode: 400, Err: err}
}
if response.StatusCode == 200 {
err = json.Unmarshal(data, &thisMap)
if err != nil {
commons.ErrorLogger.Println(err.Error())
return nil, &commons.RequestError{StatusCode: 400, Err: err}
}
return thisMap["data"][0], err
} else {
return nil, &commons.RequestError{StatusCode: response.StatusCode, Err: errors.New("getting " + strconv.Itoa(response.StatusCode) + " From Data channel API")}
}
Postman has an option to convert request to programming language equivalent.
Here is a working go code for sending the request. package main
import (
"fmt"
"strings"
"net/http"
"io/ioutil"
)
func main() {
url := "https://xxx.frappe.cloud/api/resource/Item?fields=%5B%22name%22,%22item_name%22,%22item_group%22,%22description%22%5D&filters=%5B%5B%22Item%22,%22item_group%22,%22=%22,%22xxx%20Product%22%5D%5D%0A"
method := "GET"
payload := strings.NewReader(`{
"payload": {},
"url_key": "",
"req_type": ""
}`)
client := &http.Client {
}
req, err := http.NewRequest(method, url, payload)
if err != nil {
fmt.Println(err)
return
}
req.Header.Add("Content-Type", "application/json")
req.Header.Add("Cookie", "full_name=foo; sid=secret_sid; system_user=yes; user_id=foobar; user_image=")
res, err := client.Do(req)
if err != nil {
fmt.Println(err)
return
}
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(body))
}
I am trying to write a simple admission controller for pod naming (validation) but for some reason I am generating a wrong response.
Here is my code:
package main
import (
"fmt"
"encoding/json"
"io/ioutil"
"net/http"
"github.com/golang/glog"
// for Kubernetes
"k8s.io/api/admission/v1beta1"
"k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"regexp"
)
type myValidServerhandler struct {
}
// this is the handler fuction from the HTTP server
func (gs *myValidServerhandler) serve(w http.ResponseWriter, r *http.Request) {
var Body []byte
if r.Body != nil {
if data , err := ioutil.ReadAll(r.Body); err == nil {
Body = data
}
}
if len(Body) == 0 {
glog.Error("Unable to retrive Body from API")
http.Error(w,"Empty Body", http.StatusBadRequest)
return
}
glog.Info("Received Request")
// this is where I make sure the request is for the validation prefix
if r.URL.Path != "/validate" {
glog.Error("Not a Validataion String")
http.Error(w,"Not a Validataion String", http.StatusBadRequest)
return
}
// in this part the function takes the AdmissionReivew and make sure in is in the right
// JSON format
arRequest := &v1beta1.AdmissionReview{}
if err := json.Unmarshal(Body, arRequest); err != nil {
glog.Error("incorrect Body")
http.Error(w, "incorrect Body", http.StatusBadRequest)
return
}
raw := arRequest.Request.Object.Raw
pod := v1.Pod{}
if err := json.Unmarshal(raw, &pod); err != nil {
glog.Error("Error Deserializing Pod")
return
}
// this is where I make sure the pod name contains the kuku string
podnamingReg := regexp.MustCompile(`kuku`)
if podnamingReg.MatchString(string(pod.Name)) {
return
} else {
glog.Error("the pod does not contain \"kuku\"")
http.Error(w, "the pod does not contain \"kuku\"", http.StatusBadRequest)
return
}
// I think the main problem is with this part of the code because the
// error from the events I getting in the Kubernetes namespace is that
// I am sending 200 without a body response
arResponse := v1beta1.AdmissionReview{
Response: &v1beta1.AdmissionResponse{
Result: &metav1.Status{},
Allowed: true,
},
}
// generating the JSON response after the validation
resp, err := json.Marshal(arResponse)
if err != nil {
glog.Error("Can't encode response:", err)
http.Error(w, fmt.Sprintf("couldn't encode response: %v", err), http.StatusInternalServerError)
}
glog.Infof("Ready to write response ...")
if _, err := w.Write(resp); err != nil {
glog.Error("Can't write response", err)
http.Error(w, fmt.Sprintf("cloud not write response: %v", err), http.StatusInternalServerError)
}
}
The code is working as expected except for a positive output (where the pod name meets the criteria)
there is another file with a main just grabbing the TLS files and starting the HTTP service.
so after a few digging I found what was wrong with my code
first this part
if podnamingReg.MatchString(string(pod.Name)) {
return
} else {
glog.Error("the pod does not contain \"kuku\"")
http.Error(w, "the pod does not contain \"kuku\"", http.StatusBadRequest)
return
}
by writing "return" twice I discarded the rest of the code and more so I haven't attached the request UID to the response UID and because I am using the v1 and not the v1beta1 I needed to adding the APIVersion in the response
so the rest of the code looks like :
arResponse := v1beta1.AdmissionReview{
Response: &v1beta1.AdmissionResponse{
Result: &metav1.Status{},
Allowed: false,
},
}
podnamingReg := regexp.MustCompile(`kuku`)
if podnamingReg.MatchString(string(pod.Name)) {
fmt.Printf("the pod %s is up to the name standard", pod.Name)
arResponse.Response.Allowed = true
}
arResponse.APIVersion = "admission.k8s.io/v1"
arResponse.Kind = arRequest.Kind
arResponse.Response.UID = arRequest.Request.UID
so I needed to add the 2 parts and make sure that in case the pod name is not up to standard then I need to return the right response
I made a model serving server with Python Tornado library and its sole purpose is to accept http request with payload and return result in json. The request can be made with either application/json or multipart/form-data.
To authenticate and authorise users, I made another server with Golang echo library. So all user requests should reach here before reaching my resource server.
Here I have a problem, because my program requires images as input, so users will dispatch their request with FormData. When it first hit my Golang server, I need to do the following steps
Read the form file.
Save it in local disk.
Load the file and save it in a byte buffer.
Initialise a multipart writer
Make a request to my resource server
Got result, return to user
I feel like this is redundant as I imagine there is a way to propagate those request directly to my resource server (after auth is done), without having to go through the I/O parts.
My code currently looks like this, at this point authentication is done through middleware. Is there a way to optimise this flow?
func (h Handler) ProcessFormData(c echo.Context) error {
// some validation
file, err := c.FormFile("file")
if err != nil {
return c.JSON(http.StatusBadRequest, response.Exception{
Code: errcode.InvalidRequest,
Detail: "Invalid file uploaded",
Error: err,
})
}
filePath, err := fileUtil.SaveNetworkFile(file)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
f, err := os.Open(filePath)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
defer f.Close()
fi, err := f.Stat()
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
var body bytes.Buffer
writer := multipart.NewWriter(&body)
part, err := writer.CreateFormFile("file", fi.Name())
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
if _, err := io.Copy(part, f); err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
writer.Close()
req, err := http.NewRequest("POST", fmt.Sprintf("%s", env.ResourceServer), &body)
req.Header.Set("Content-Type", writer.FormDataContentType())
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.APIRequestError,
Error: err,
})
}
client := &http.Client{}
res, err := client.Do(req)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.APIRequestError,
Detail: "Error when posting request to resource server",
Error: err,
})
}
defer res.Body.Close()
data, _ := ioutil.ReadAll(res.Body)
if res.StatusCode != 200 {
errorData := &model.PanicResponse{}
err := json.Unmarshal(data, errorData)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.UnmarshalError,
Error: err,
})
}
return c.JSON(res.StatusCode, errorData)
}
result := &model.SuccessResponse{}
err = json.Unmarshal(data, result)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.UnmarshalError,
Error: err,
})
}
if fileUtil.IsFileExists(filePath) {
fileUtil.DeleteFile(filePath)
}
// track and update usage
userData := c.Get("USER")
user := userData.(model.User)
db.UpdateUsage(h.Db, &user.ID)
return c.JSON(200, result)
}
Found a solution thanks to the comment from #cerise-limón
Essentially, I need just 2 lines
f, err := file.Open()
if _, err := io.Copy(part, f); err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
})
}
I'm doing a post request and I get a 200 OK response. I also receive the headers. However, the body keeps coming back empty.
There should be a body, when I run it in postman the body shows up. What am I missing here?
func AddHealthCheck(baseURL string, payload HealthCheck, platform string, hostname string) (string, error) {
url := fmt.Sprintf(baseURL+"add-healthcheck/%s/%s", platform, hostname)
//convert go struct to json
jsonPayload, err := json.Marshal(payload)
if err != nil {
log.Error("[ADD HEALTH CHECK] Could not convert go struct to json : ", err)
return "", err
}
// Create client & set timeout
client := &http.Client{}
client.Timeout = time.Second * 15
// Create request
req, err := http.NewRequest("POST", url, bytes.NewBuffer(jsonPayload))
if err != nil {
log.Error("[ADD HEALTH CHECK] Could not create request : ", err)
return "", err
}
req.Header.Set("Content-Type", "application/json")
// Fetch Request
resp, err := client.Do(req)
if err != nil {
log.Error("[ADD HEALTH CHECK] Could not fetch request : ", err)
return "", err
}
defer resp.Body.Close()
// Read Response Body
respBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Error("[HEALTH CHECK] Could not read response body : ", err)
return "", err
}
fmt.Println("response Status : ", resp.Status)
fmt.Println("response Headers : ", resp.Header)
fmt.Println("response Body : ", string(respBody))
return string(respBody), nil
}
I have confirmed locally that your code, as shown, should work.
Here is the code I used:
package main
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"time"
)
func main() {
http.HandleFunc("/", handler)
go func(){
http.ListenAndServe(":8080", nil)
}()
AddHealthCheck()
}
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hi there")
}
func panicError(err error) {
if err != nil {
panic(err)
}
}
func AddHealthCheck() (string, error) {
//convert go struct to json
payload := "bob"
jsonPayload, err := json.Marshal(payload)
panicError(err)
// Create client & set timeout
client := &http.Client{}
client.Timeout = time.Second * 15
// Create request
req, err := http.NewRequest("POST", "http://localhost:8080", bytes.NewBuffer(jsonPayload))
panicError(err)
req.Header.Set("Content-Type", "application/json")
// Fetch Request
resp, err := client.Do(req)
panicError(err)
defer resp.Body.Close()
// Read Response Body
respBody, err := ioutil.ReadAll(resp.Body)
panicError(err)
fmt.Println("response Status : ", resp.Status)
fmt.Println("response Headers : ", resp.Header)
fmt.Println("response Body : ", string(respBody))
return string(respBody), nil
}
The code above is just a slightly stripped down version of your code, and it outputs the body of the response. (Note that I provide a server here to receive the post request and return a response)
The server is simply not sending you a body. You can confirm this with something like wireshark.
If you are getting a body back using postman, you must be sending a different request in postman than in go. It can sometimes be tough to see what is the difference, as both go and postman can sometimes add headers behind the scenes that you don't see. Again, something like wireshark can help here.
Or if you have access to the server, you can add logs there.