echo c.Get("user") does not work in testing env - testing

I am attempting to test a golang API built on the echo framework/router. I have the following test.....
func TestLogout(t *testing.T) {
loadConfig()
db := stubDBs(t)
Convey("When you post to /logout", t, func() {
Convey("with a valid token, you should get aa success msg and be logged out", func() {
e := echo.New()
e.Use(middleware.JWTWithConfig(middleware.JWTConfig{
SigningKey: []byte("secret"),
TokenLookup: "query:user",
}))
req, err := http.NewRequest(echo.POST, "auth/logout", strings.NewReader(""))
if err == nil {
req.Header.Set(echo.HeaderAuthorization, fmt.Sprintf("Bearer %v", Token))
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
Logout(db, Models.Redis)(c)
body := GetJsonBody(rec.Body.String())
error := body.Path("error").Data()
msg := body.Path("error").Data().(string)
pw := body.Path("data.user.password").Data().(string)
token := body.Path("data.user.token").Data()
So(error, ShouldEqual, nil)
So(msg, ShouldEqual, "Logged Out")
So(rec.Code, ShouldEqual, 200)
So(pw, ShouldEqual, "")
So(token, ShouldEqual, nil)
}
})
})
}
and in the controller....
//Logout ...
func Logout(db *sqlx.DB, r *redis.Client) echo.HandlerFunc {
return func(c echo.Context) error {
var user Models.User
log.Println(c.Get("user")) //<-----This Logs NIL only on testing
if c.Get("user") == nil {
res := createAuthErrorResponse(user, "Invalid Token")
return c.JSON(http.StatusOK, res)
}
token := c.Get("user").(*jwt.Token)
err := Models.Redis.Del(token.Raw).Err()
if err != nil {
handleErr(err)
res := createAuthErrorResponse(user, "Token not in storage")
return c.JSON(http.StatusOK, res)
}
user.Password = ""
user.Token = null.StringFrom("")
res := createAuthSuccessResponse(user, "Logged Out.")
return c.JSON(http.StatusOK, res)
}
}
Any advice on how I can test this functionality in the cho framework? This is a behavior I would not expect (c.Get("user") does not act the same on testing as in live env).

Through much pain I was able to figure this out. Here is working code. Explanation to follow.
Convey("with a valid token, you should get a success msg and be logged out", func() {
e := echo.New()
req, err := http.NewRequest(echo.POST, "auth/logout", strings.NewReader(""))
if err == nil {
req.Header.Set(echo.HeaderAuthorization, fmt.Sprintf("Bearer %v", Token))
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
middleware.JWTWithConfig(middleware.JWTConfig{
SigningKey: []byte("secret"),
})(Logout(db, Models.Redis))(c)
body := GetJsonBody(rec.Body.String())
error := body.Path("error").Data()
msg := body.Path("msg").Data().(string)
pw := body.Path("data.user.password").Data().(string)
token := body.Path("data.user.token").Data()
So(error, ShouldEqual, nil)
So(msg, ShouldEqual, "Logged Out")
So(rec.Code, ShouldEqual, 200)
So(pw, ShouldEqual, "")
So(token, ShouldEqual, nil)
}
})
I had set my Header properly but was not executing the JWT echo middleware which meant that my token was not validated and made accesisible using the c.Get() command.
The key here is that testing echo middlware without running the server requires you to execute the middleware function, which then takes a handler function (this is your function you want to test) and then takes the context. So, the function should look as follows to test your middleware routes. If it is all caps you need to replace it with your specific case.
A: middleware.MIDDLEWARE_FUNC(NECESSARY_CONFIGS) // <--- this returns echo.MiddlewareFunc
B: (YOUR_HANDLER(req, res)) // <---echo.MiddlewareFunc takes an echo.HanlderFunc as its only argument and you pass your handler you test req, res
(c) // <--Finally, you call the resulting compose function with your test context
A(B())(c)

Related

How i can fetch all data by latitude and longitude from wigle api

I wrote simple script to receive all data from wigle api using wigleapiv2, definitely this endpoint /api/v2/network/search. But I faced the problem, that I can receive only 1000 unique ssid's. I'm changing URL every iteration, and put in URL previous page's searchAfter. How can I fix it and receive all data from certain latitude and longitude?
Here an example of first iteration Uri (https://api.wigle.net/api/v2/network/search?closestLat=12.9&closestLong=1.2&latrange1=1.9&latrange2=1.8&longrange1=1.2&longrange2=1.4)
And here an example of remaining iterations uris (https://api.wigle.net/api/v2/network/search?closestLat=12.9&closestLong=1.2&latrange1=1.9&latrange2=1.8&longrange1=1.2&longrange2=1.4&searchAfter=1976621348&first=1). For every iteration I'm changing searchAfter and first.
It would be great id someone can say me where I'm doing wrong:)
I've tried to using only first or search after parameters, but it has the same result. One mark that I noticed, that when I'm using only searchAfter param I can receive only 100 unique ssids, but when I'm using both (searchAfter and first) I can receive 1000 unique ssids.
Here my main.go code
var (
wg = sync.WaitGroup{}
receiveResp = make(chan []*response.WiFiNetworkWithLocation, 100)
)
func main() {
startTime := time.Now()
viper.AddConfigPath(".")
viper.SetConfigFile("config.json")
if err := viper.ReadInConfig(); err != nil {
log.Fatal("error trying read from config: %w", err)
}
u := user.NewUser(viper.GetString("users.user.username"), viper.GetString("users.user.password"))
db, err := postgres.NewPG()
if err != nil {
log.Fatalf("Cannot create postgres connection: %v", err)
}
postgres.WG.Add(1)
go getResponse(u)
go parseResponse(db)
postgres.WG.Wait()
fmt.Printf("Execution time: %v ", time.Since(startTime))
}
func getResponse(u *user.Creds) {
url := fmt.Sprintf("%s? closestLat=%s&closestLong=%s&latrange1=%s&latrange2=%s&longrange1=%s&longrange2=%s",
viper.GetString("wigle.url"),
viper.GetString("queries.closestLat"),
viper.GetString("queries.closestLong"),
viper.GetString("queries.latrange1"),
viper.GetString("queries.latrange2"),
viper.GetString("queries.longrange1"),
viper.GetString("queries.longrange2"),
)
j := 0
i := 0
for {
i++
fmt.Println(url)
req, err := http.NewRequest("GET", url, bytes.NewBuffer([]byte("")))
if err != nil {
log.Printf("Failed wraps request: %v", err)
continue
}
req.SetBasicAuth(u.Username, u.Password)
c := http.Client{}
resp, err := c.Do(req)
if err != nil {
log.Printf("Failed send request: %v", err)
continue
}
bytes, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Printf("Failed read response body: %v", err)
continue
}
var r response.NetSearchResponse
if err := json.Unmarshal(bytes, &r); err != nil {
log.Printf("Failed unmarshal: %v", err)
continue
}
receiveResp <- r.Results
fmt.Println(r.TotalResults, r.SearchAfter)
if r.SearchAfter == "" {
postgres.WG.Done()
return
}
url = fmt.Sprintf("%s? closestLat=%s&closestLong=%s&latrange1=%s&latrange2=%s&longrange1=%s&longrange2=%s&searchAfter=%s&first=%v" ,
viper.GetString("wigle.url"),
viper.GetString("queries.closestLat"),
viper.GetString("queries.closestLong"),
viper.GetString("queries.latrange1"),
viper.GetString("queries.latrange2"),
viper.GetString("queries.longrange1"),
viper.GetString("queries.longrange2"),
r.SearchAfter,
i,
)
j++
fmt.Println(j)
}
func parseResponse(db *sql.DB) {
for {
select {
case responses := <-receiveResp:
clearResponses := make([]response.WiFiNetworkWithLocation, 0, len(responses))
for _, val := range responses {
clearResponses = append(clearResponses, *val)
}
postgres.WG.Add(1)
go postgres.SaveToDB(db, "test", clearResponses)
}
}
}

Go how to use cookiejar for multiple requests?

I'm trying to make a cli for a site that has csrf, needing the csrf token to be sent on headers and on a form.
I can't seem to understand net/http.Client or net/http/cookieJar
This its even good practice? There's a better way of doing csrf login on Go ?
Thx in advance ^v^
This its my code:
package main
import (
"fmt"
"log"
"net/http"
"net/http/cookiejar"
"net/url"
"strings"
"time"
)
var (
httpClient = &http.Client{}
)
func main() {
jar, err := cookiejar.New(nil)
if err != nil {
log.Fatal(err)
}
httpClient = &http.Client{
Timeout: 30 * time.Second,
Jar: jar,
}
requestURL := "https://example.com/"
res, err := httpClient.Get(requestURL)
if err != nil {
log.Fatal(err)
}
log.Println(res.Cookies())
// stdout: cookie as expected
u := &url.URL{}
u.Parse(requestURL)
log.Println(httpClient.Jar.Cookies(u))
// stdout: []
form := make(url.Values)
/* ... */
req, err := http.NewRequest(http.MethodPost, requestURL, strings.NewReader(form.Encode()))
if err != nil {
fmt.Printf("client: could not create request: %s\n", err)
}
res, err = httpClient.Do(req)
if err != nil {
log.Fatal(err)
}
fmt.Println(req)
// stdout: cookie as expected
}

how to do something repeatedly in a database action?

I am writing a user authentication system in go.First of all I prompt user to signup the form with email, username and password. Then I send a confirmation link to users email. The user must also select a title for his blog.Which is prompted after the confirmation link is clicked. How to ensure that the user don't move to the home page without a title.
My ConfirmEmail function is below:
func ConfirmEmail(w http.ResponseWriter, r *http.Request){
err := r.ParseForm()
if err != nil{
log.Fatal("Unable to parse data")
}
token := r.Form.Get("token")
db.ConnectDB()
current_time := time.Now().Unix()
user_id := 0
var date_generated int64
var date_expires int64
var date_used int64
row := db.Db.QueryRow("Select user_id, date_generated, date_expires, date_used from Token where token = ?", token)
if err := row.Scan(&user_id, &date_generated, &date_expires, &date_used); err != nil{
if err == sql.ErrNoRows{
//todo: no such token provide a link to signup..
fmt.Println("No such rows..")
} else {
log.Fatal("Something went wrong:", err)
}
}
//reuse of the token...
if (date_used != 0){
http.Redirect(w,r, "/signup", http.StatusFound)
}
// use of expired token...
if(date_expires < current_time){
//todo: inform about the expired token and prompt for re confirmation..
fmt.Println("Token expired..")
} else{
//todo: Check for blog title, if null prompt.
var title string
var username string
if err := db.Db.QueryRow("select username, blogTitle from User where user_id = ?", user_id).Scan(&username, &title); err != nil{
if err == sql.ErrNoRows{
http.Redirect(w, r, "/signup", http.StatusFound)
}
}
//want to do this until title is not provided..
if len(title) == 0{
err = templates.ExecuteTemplate(w, "chose-title.html", struct {
Username string
Msg string
}{
Username: username,
Msg: "",
})
if err != nil {
log.Fatal("Unable to render provided template:",err)
}
return
}
_, err = db.Db.Exec("Update Token set date_used = ? where token=?",current_time, token)
if err != nil {
log.Fatal("Unable to update with given data")
}
_, err = db.Db.Exec("Update User set Verified = true where user_id=?",user_id)
if err != nil {
log.Fatal("Unable to update with given data")
} else {
http.Redirect(w, r, "/login", http.StatusFound)
}
}
}
The main problematic part is:(contains snippet from previous block)
if len(title) == 0{
err = templates.ExecuteTemplate(w, "chose-title.html", struct {
Username string
Msg string
}{
Username: username,
Msg: "",
})
if err != nil {
log.Fatal("Unable to render provided template:",err)
}
return
}
_, err = db.Db.Exec("Update Token set date_used = ? where token=?",current_time, token)
if err != nil {
log.Fatal("Unable to update with given data")
}
_, err = db.Db.Exec("Update User set Verified = true where user_id=?",user_id)
if err != nil {
log.Fatal("Unable to update with given data")
} else {
http.Redirect(w, r, "/login", http.StatusFound)
}
}
I can think of a while loop in this, but don't think that would be a feasible option. Is there any other workaround or workflow to check this.

400 Bad Request for frappe.cloud API

I'm getting 400 Bad Request for frappe.cloud API, when I'm trying to call it using golang code using http.NewRequest, this API is working fine when I check it using postman. following is the API
https://xxxx.frappe.cloud/api/resource/Item?fields=["name","item_name","item_group","description"]&filters=[["Item","item_group","=","xxx Product"]]
If I use the same golang code to call same API with out filters it works fine. following is the working API
https://xxxx.frappe.cloud/api/resource/Item?fields=["name","item_name","item_group","description"]
code as follows
func FetchProperties(dataChannel models.DataChannel) (map[string]interface{}, error) {
thisMap := make(map[string][]map[string]interface{})
client := &http.Client{}
req, err := http.NewRequest("GET", dataChannel.APIPath, nil)
if err != nil {
commons.ErrorLogger.Println(err.Error())
return nil, err
}
eds, err := GetDecryptedEDSByEDSID(dataChannel.EDSId)
if err != nil {
commons.ErrorLogger.Println(err.Error())
return nil, &commons.RequestError{StatusCode: 400, Err: err}
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", eds.DataSource.Auth.Token)
response, err := client.Do(req)
if err != nil {
commons.ErrorLogger.Println(err.Error())
return nil, err
}
data, err := ioutil.ReadAll(response.Body)
if err != nil {
commons.ErrorLogger.Println(err.Error())
return nil, &commons.RequestError{StatusCode: 400, Err: err}
}
if response.StatusCode == 200 {
err = json.Unmarshal(data, &thisMap)
if err != nil {
commons.ErrorLogger.Println(err.Error())
return nil, &commons.RequestError{StatusCode: 400, Err: err}
}
return thisMap["data"][0], err
} else {
return nil, &commons.RequestError{StatusCode: response.StatusCode, Err: errors.New("getting " + strconv.Itoa(response.StatusCode) + " From Data channel API")}
}
Postman has an option to convert request to programming language equivalent.
Here is a working go code for sending the request. package main
import (
"fmt"
"strings"
"net/http"
"io/ioutil"
)
func main() {
url := "https://xxx.frappe.cloud/api/resource/Item?fields=%5B%22name%22,%22item_name%22,%22item_group%22,%22description%22%5D&filters=%5B%5B%22Item%22,%22item_group%22,%22=%22,%22xxx%20Product%22%5D%5D%0A"
method := "GET"
payload := strings.NewReader(`{
"payload": {},
"url_key": "",
"req_type": ""
}`)
client := &http.Client {
}
req, err := http.NewRequest(method, url, payload)
if err != nil {
fmt.Println(err)
return
}
req.Header.Add("Content-Type", "application/json")
req.Header.Add("Cookie", "full_name=foo; sid=secret_sid; system_user=yes; user_id=foobar; user_image=")
res, err := client.Do(req)
if err != nil {
fmt.Println(err)
return
}
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(body))
}

Optimise multiple network request

I made a model serving server with Python Tornado library and its sole purpose is to accept http request with payload and return result in json. The request can be made with either application/json or multipart/form-data.
To authenticate and authorise users, I made another server with Golang echo library. So all user requests should reach here before reaching my resource server.
Here I have a problem, because my program requires images as input, so users will dispatch their request with FormData. When it first hit my Golang server, I need to do the following steps
Read the form file.
Save it in local disk.
Load the file and save it in a byte buffer.
Initialise a multipart writer
Make a request to my resource server
Got result, return to user
I feel like this is redundant as I imagine there is a way to propagate those request directly to my resource server (after auth is done), without having to go through the I/O parts.
My code currently looks like this, at this point authentication is done through middleware. Is there a way to optimise this flow?
func (h Handler) ProcessFormData(c echo.Context) error {
// some validation
file, err := c.FormFile("file")
if err != nil {
return c.JSON(http.StatusBadRequest, response.Exception{
Code: errcode.InvalidRequest,
Detail: "Invalid file uploaded",
Error: err,
})
}
filePath, err := fileUtil.SaveNetworkFile(file)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
f, err := os.Open(filePath)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
defer f.Close()
fi, err := f.Stat()
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
var body bytes.Buffer
writer := multipart.NewWriter(&body)
part, err := writer.CreateFormFile("file", fi.Name())
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
if _, err := io.Copy(part, f); err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
Error: err,
})
}
writer.Close()
req, err := http.NewRequest("POST", fmt.Sprintf("%s", env.ResourceServer), &body)
req.Header.Set("Content-Type", writer.FormDataContentType())
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.APIRequestError,
Error: err,
})
}
client := &http.Client{}
res, err := client.Do(req)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.APIRequestError,
Detail: "Error when posting request to resource server",
Error: err,
})
}
defer res.Body.Close()
data, _ := ioutil.ReadAll(res.Body)
if res.StatusCode != 200 {
errorData := &model.PanicResponse{}
err := json.Unmarshal(data, errorData)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.UnmarshalError,
Error: err,
})
}
return c.JSON(res.StatusCode, errorData)
}
result := &model.SuccessResponse{}
err = json.Unmarshal(data, result)
if err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.UnmarshalError,
Error: err,
})
}
if fileUtil.IsFileExists(filePath) {
fileUtil.DeleteFile(filePath)
}
// track and update usage
userData := c.Get("USER")
user := userData.(model.User)
db.UpdateUsage(h.Db, &user.ID)
return c.JSON(200, result)
}
Found a solution thanks to the comment from #cerise-limón
Essentially, I need just 2 lines
f, err := file.Open()
if _, err := io.Copy(part, f); err != nil {
return c.JSON(http.StatusInternalServerError, response.Exception{
Code: errcode.SystemError,
Detail: "Error when processing file",
})
}